chapter1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction \n",
    "\n",
    "## Overview\n",
    "\n",
    "The Community Data Management System is an object-oriented data management\n",
    "system, specialized for organizing multidimensional, gridded data used\n",
    "in climate analysis and simulation. \n",
    "\n",
    "CDMS is implemented as part of the Climate Data\n",
    "Analysis Tool (CDAT), which uses the Python language. The examples in\n",
    "this chapter assume some familiarity with the language and the Python\n",
    "Numpy module (https://www.numpy.org). A number of excellent tutorials\n",
    "on Python are available in books or on the Internet. For example, see\n",
    "the [Python Foundation's homepage](https://python.org).\n",
    "\n",
    "\n",
    "## Variables\n",
    "\n",
    "The basic unit of computation in CDMS is the variable. A variable is essentially a multidimensional data array, augmented with a domain, a set of attributes, and optionally a spatial and/or temporal coordinate system [(see Coordinate Axes)](https://cdms.readthedocs.io/en/readthedocstest/manual/cdms_1.html#coordinate-axes).  As a data array, a variable can\n",
    "be sliced to obtain a portion of the data, and can be used in arithmetic\n",
    "computations. For example, if ``u`` and ``v`` are variables representing\n",
    "the eastward and northward components of wind speed, respectively, and\n",
    "both variables are functions of time, latitude, and longitude, then the\n",
    "velocity for time 0 (first index) can be calculated as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import cdms2\n",
    "from cdms2 import MV2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# uncomment the line below to donwload \"clt.nc\"\n",
    "# !wget \"https://cdat.llnl.gov/cdat/sample_data/clt.nc\"\n",
    "f1=cdms2.open(\"https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/clt.nc\")\n",
    "u = f1('u')\n",
    "v = f1('v')\n",
    "from cdms2 import MV2\n",
    "print(u.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MV is a module that wraps numpy masked arrays.  It keeps the axes, grid and other attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "vel = MV2.sqrt(u[0]**2 + v[0]**2)\n",
    "print(vel.listattributes())\n",
    "print(\"units: \", vel.units)\n",
    "print(vel.getLevel())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Several Points:\n",
    "\n",
    "-  Square brackets represent the slice operator. Indexing starts at 0,\n",
    "   so ``u[0]`` selects from variable ``u`` for the first timepoint. The\n",
    "   result of this slice operation is another variable. The slice\n",
    "   operator can be multidimensional, and follows the syntax of Numpy\n",
    "   Python arrays. In this example, ``u[0:10,:,1]`` would retrieve data\n",
    "   for the first ten timepoints, at all latitudes, for the second\n",
    "   longitude.\n",
    "-  Variables can be used in computation. ``**`` is the Python\n",
    "   exponentiation operator.\n",
    "-  Arithmetic functions are defined in the ``cdms2.MV2`` module.\n",
    "-  Operations on variables carry along the corresponding metadata where\n",
    "   applicable. In the above example, ``vel`` has the same latitude and\n",
    "   longitude coordinates as ``u`` and ``v``, and the time coordinate is\n",
    "   the first time of ``u`` and ``v``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print u([0:10,:,1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# File I/O\n",
    "\n",
    "\n",
    "A variable can be obtained from a file or collection of files, or can be\n",
    "generated as the result of a computation. Files can be in any of the\n",
    "self- describing formats `netCDF`, `HDF`, `GrADS/GRIB` *(GRIB with a GrADS\n",
    "control file)*, or `PCMDI DRS`. (HDF and DRS support is optional, and is\n",
    "configured at the time CDMS is installed.) For instance, here is an example to read data\n",
    "from file clt.nc into variable u."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = cdms2.open('https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/clt.nc')\n",
    "u = f('u')\n",
    "u.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Data can be read by index or by world coordinate values. The following reads the `n-th` timepoint of `u` *(the syntax slice(i,j)* refers to indices `k` such that `i <= k < j`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n=0\n",
    "u0 = f('u',time=slice(n,n+1))\n",
    "u0.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To read u at time 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l = f('u',time=1.)\n",
    "l.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A variable can be written to a file follwoing the CF-1 convention with the write function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = cdms2.open('sample2.nc','w')\n",
    "g.write(u) \n",
    "g.close()\n",
    "!ncdump -h sample2.nc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Coordinate Axes\n",
    "\n",
    "\n",
    "A coordinate axis is a variable that represents coordinate information.\n",
    "Typically an axis is associated with one or more variables in a file or\n",
    "dataset, to represent the indexing and/or spatiotemporal coordinate\n",
    "system(s) of the variable(s).\n",
    "\n",
    "Often in climate applications an axis is a one-dimensional variable\n",
    "whose values are floating-point and strictly monotonic. In some cases an\n",
    "axis can be multidimensional (see `Grids`). If an axis is\n",
    "associated with one of the canonical types latitude, longitude, level,\n",
    "or time, then the axis is called temporal .\n",
    "\n",
    "The shape and physical ordering of a variable is represented by the\n",
    "variables domain , an ordered tuple of one-dimensional axes. In the\n",
    "previous example, the domain of the variable u is the tuple (time,\n",
    "latitude, longitude). This indicates the order of the dimensions, with\n",
    "the slowest-varying dimension listed first (time). The domain may be\n",
    "accessed with the ``getAxisList()`` method:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u.getAxisList() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above example, the domain elements are axes that are also\n",
    "spatiotemporal. In general it is not always the case that an element of\n",
    "a domain is spatio-temporal: \n",
    "\n",
    "-  An axis in the domain of a variable need not be spatiotemporal. For\n",
    "   example, it may represent a range of indices, an index coordinate\n",
    "   system.\n",
    "-  The latitude and/or longitude coordinate axes associated with a\n",
    "   variable need not be elements of the domain. In particular this will\n",
    "   be true if the variable is defined on a non-rectangular grid (see Grids <#grids>`__).\n",
    "\n",
    "As previously noted, a spatial and/or temporal coordinate system may be\n",
    "associated with a variable. The methods getLatitude, getLongitude,\n",
    "getLevel, and getTime return the associated coordinate axes. For\n",
    "example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t = u.getTime()\n",
    "print(t[:])\n",
    "print(t.units)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Attributes\n",
    "\n",
    "As mentioned above, variables can have associated attributes ,\n",
    "name-value pairs. In fact, nearly all CDMS objects can have associated\n",
    "attributes, which are accessed using the Python dot notation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u.units='m/s'\n",
    "print(u.units)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Attribute values can be strings, scalars, or 1-D Numpy arrays.\n",
    "\n",
    "When a variable is written to a file, not all the attributes are\n",
    "written. Some attributes, called internal attributes, are used for\n",
    "bookkeeping, and are not intended to be part of the external file\n",
    "representation of the variable. In contrast, external attributes are\n",
    "written to an output file along with the variable. By default, when an\n",
    "attribute is set, it is treated as external. Every variable has a field\n",
    "attributes, a Python dictionary that defines the external attributes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(u.attributes.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Python dir command lists the internal attribute names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dir(u)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general internal attributes should not be modified directly. One\n",
    "exception is the id attribute, the name of the variable. It is used in\n",
    "plotting and I/O, and can be set directly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Masked values\n",
    "\n",
    "Optionally, variables have a mask that represents where data are\n",
    "missing. If present, the mask is an array of ones and zeros having the\n",
    "shape of the data array. A mask value of one indicates that the\n",
    "corresponding data array element is missing or invalid.\n",
    "\n",
    "Arithmetic operations in CDMS take missing data into account. The same\n",
    "is true of the functions defined in the MV2 module. For example:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import MV2\n",
    "a = MV2.array([1,2,3]) # Create array a, with no mask\n",
    "b = MV2.array([4,5,6]) # Same for b\n",
    "a+b # variable_... array([5,7,9,]) \n",
    "\n",
    "a[1]=MV2.masked # Mask the second value of a a.mask()\n",
    "a.mask\n",
    "\n",
    "a+b # The sum is masked also "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When data is read from a file, the result variable is masked if the file\n",
    "variable has a ``fill_value`` attribute. The mask is set to one for\n",
    "those elements equal to the missing value, zero elsewhere. If no such\n",
    "attribute is present in the file, the result variable is not masked.\n",
    "\n",
    "When a variable with masked values is written to a file, data values\n",
    "with a corresponding mask value of one are set to the value of the\n",
    "variables ``missing_value`` attribute. The data and ``missing_value``\n",
    "attribute are then written to the file.\n",
    "\n",
    "[https://www.numpy.org/](https://www.numpy.org/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File Variables\n",
    "\n",
    "A variable can be obtained either from a file, a collection of files, or\n",
    "as the result of computation. Correspondingly there are three types of\n",
    "variables in CDMS:\n",
    "\n",
    "-  *file variable* is a variable associated with a single data file.\n",
    "   Setting or referencing a file variable generates I/O operations.\n",
    "-  A *dataset variable* is a variable associated with a collection of\n",
    "   files. Reference to a dataset variable reads data, possibly from\n",
    "   multiple files. Dataset variables are read-only.\n",
    "-  *transient variable* is an in-memory object not associated with a\n",
    "   file or dataset. Transient variables result from a computation or I/O\n",
    "   operation.\n",
    "\n",
    "Typical use of a file variables is to inquire information about the\n",
    "variable in a file without actually reading the data for the variables.\n",
    "A file variable is obtained by applying the slice operator [] to a file,\n",
    "passing the name of the variable, or by calling the getVariable\n",
    "function. Note that obtaining a file variable does not actually read the\n",
    "data array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u = f.getVariable('u') # or u=f['u']\n",
    "u.shape \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "File variables are also useful for fine-grained I/O. They behave like\n",
    "transient variables, but operations on them also affect the associated\n",
    "file. Specifically:\n",
    "\n",
    "-  slicing a file variable reads data,\n",
    "-  setting a slice writes data,\n",
    "-  referencing an attribute reads the attribute,\n",
    "-  setting an attribute writes the attribute,\n",
    "-  and calling a file variable like a function reads data associated\n",
    "   with the variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget  http://cdat.llnl.gov/cdat/sample_data/clt.nc\n",
    "import os\n",
    "os.system(\"cp clt.nc /tmp\")\n",
    "\n",
    "f = cdms2.open('/tmp/clt.nc','a') # Open read/write\n",
    "uvar = f['u'] # Note square brackets\n",
    "uvar.shape\n",
    "\n",
    "u0 = uvar[0] # Reads data from sample.nc\n",
    "u0.shape\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Writes data to file sample.nc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "uvar[1]=u0 # Writes data to sample.nc\n",
    "uvar.units # Reads the attribute 'm/s'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calling  a variable like a function reads data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u24 = uvar(time=1.0) # Calling  a variable like a function reads data\n",
    "f.close() # Save changes to clt.nc (I/O may be buffered)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For transient variables, the data is printed only if the size of the array is less\n",
    "than the print limit . This value can be set with the function\n",
    "MV2.set_print_limit to force the data to be printed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(MV2.get_print_limit()) # Current limit 1000\n",
    "MV2.set_print_limit(100)\n",
    "print(MV2.get_print_limit())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The datatype of the variable is determined with the typecode function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "u.typecode()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset Variables\n",
    "\n",
    "The third type of variable, a *dataset variable*, is associated with a\n",
    "*dataset*, a collection of files that is treated as a single file. A\n",
    "dataset is created with the ``cdscan`` utility. This generates an XML\n",
    "metafile that describes how the files are organized and what metadata\n",
    "are contained in the files. In a climate simulation application, a\n",
    "dataset typically represents the data generated by one run of a general\n",
    "circulation or coupled ocean-atmosphere model.\n",
    "\n",
    "For example, suppose data for variables u and v are stored in six files:\n",
    "\n",
    "1. u_2000.nc,\n",
    "2. u_2001.nc,\n",
    "3. u_2002.nc,\n",
    "4. v_2000.nc,\n",
    "5. v_2001.nc,\n",
    "6. v_2002.nc.\n",
    "\n",
    "A metafile can be generated with the command:\n",
    "\n",
    "``$ cdscan -x cdsample.xml [uv]*.nc``\n",
    "\n",
    "The metafile **cdsample.xml** is then used like an ordinary data file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget  http://cdat.llnl.gov/cdat/sample_data/u_2000.nc >/dev/null  2>/dev/null\n",
    "!wget  http://cdat.llnl.gov/cdat/sample_data/u_2001.nc >/dev/null  2>/dev/null\n",
    "!wget  http://cdat.llnl.gov/cdat/sample_data/u_2002.nc >/dev/null  2>/dev/null\n",
    "!wget  http://cdat.llnl.gov/cdat/sample_data/v_2000.nc >/dev/null  2>/dev/null\n",
    "!wget  http://cdat.llnl.gov/cdat/sample_data/v_2001.nc >/dev/null  2>/dev/null\n",
    "!wget  http://cdat.llnl.gov/cdat/sample_data/v_2002.nc >/dev/null  2>/dev/null\n",
    "!cdscan -x cdsample.xml [uv]*.nc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls -l cdsample.xml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cdms2\n",
    "fsample=cdms2.open(\"cdsample.xml\")\n",
    "udata=fsample['u']\n",
    "print(\"aggregated u:\",udata.shape)\n",
    "vdata=fsample['u']\n",
    "print(\"aggregated v:\", vdata.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Grids\n",
    "\n",
    "A latitude-longitude grid represents the coordinate information\n",
    "associated with a variable. A grid encapsulates:\n",
    "\n",
    "-  latitude, longitude coordinates\n",
    "-  grid cell boundaries\n",
    "-  area weights\n",
    "\n",
    "CDMS defines a rich set of grid types to represent the variety of\n",
    "coordinate systems used in climate model applications. Grids can be\n",
    "categorized as rectangular or nonrectangular.\n",
    "\n",
    "A rectangular grid has latitude and longitude axes that are\n",
    "one-dimensional, with strictly monotonic values. The grid is essentially\n",
    "the Cartesian product of the axes. If either criterion is not met, the\n",
    "grid is nonrectangular .\n",
    "\n",
    "- CDMS supports two types of nonrectangular grid:\n",
    "\n",
    "  -  A curvilinear grid consists of a latitude and longitude axis, each of\n",
    "     which is a two-dimensional coordinate axis. Curvilinear grids are\n",
    "     often used in ocean model applications.\n",
    "  -  A generic grid consists of a latitude and longitude axis, each of\n",
    "     which is an auxiliary one-dimensional coordinate axis. An auxiliary\n",
    "     axis has values that are not necessarily monotonic. As the name\n",
    "     suggests, generic grids can represent virtually any type of grid.\n",
    "     However, it is more difficult to determine adjacency relationships\n",
    "     between grid points."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: A Curvilinear Grid\n",
    "\n",
    "In this example, variable sample is defined on a 128x192 curvilinear\n",
    "grid. Note that:\n",
    "\n",
    "-  The domain of variable sample is ( y , x ) where y and x are index\n",
    "   coordinate axes.\n",
    "-  The curvilinear grid associated with sample consists of axes ( lat ,\n",
    "   lon ), each a two-dimensional coordinate axis.\n",
    "-  lat and lon each have domain ( y , x )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# uncomment the line below to donwload \"clt.nc\"\n",
    "# wget \"https://cdat.llnl.gov/cdat/sample_data/sampleCurveGrid4.nc\"\n",
    "f = cdms2.open('https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/sampleCurveGrid4.nc')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "``lat`` and ``lon`` are coordinate axes, but are grouped with data variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import MV2\n",
    "print(f.variables.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "y and x are index coordinate axes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f.axes.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read data for variable sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample = f('sample')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The associated grid g is curvilinear"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g = sample.getGrid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The domain of the variable consfigists of index axes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample.getAxisIds()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the coordinate axes associated with the grid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat = g.getLatitude() # or sample.getLatitude()\n",
    "lon = g.getLongitude() # or sample.getLongitude()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lat and lon have the same domain, a subset of the domain of 'sample'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat.getAxisIds()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lat and lon are variables ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(lat.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(lat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lat_in_radians = lat*MV2.pi/180.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(lat_in_radians)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example: A Generic Grid\n",
    "\n",
    "In this example variable zs is defined on a generic grid. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f.variables.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "['lat', 'sample', 'bounds_lon', 'lon', 'bounds_lat']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f.axes.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "['nvert', 'x', 'y']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<TransientCurveGrid, id: ..., shape: (32, 48)>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "zs = f(\"sample\")\n",
    "g = zs.getGrid()\n",
    "g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "variable zs is defined in terms of a single index coordinate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat = g.getLatitude()\n",
    "lon = g.getLongitude()\n",
    "print(lat.shape)\n",
    "print(lon.shape) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cell axis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zs.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zs.getAxisIds()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lat and lon are also defined in terms of the cell axis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat.getAxisIds()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lat and lon are one-dimensional, 'auxiliary' coordinate\n",
    "axes: values are not monotonic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lat.__class__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Generic grids can be used to represent any of the grid types. The method\n",
    "toGenericGrid can be applied to any grid to convert it to a generic\n",
    "representation. Similarly, a rectangular grid can be represented as\n",
    "curvilinear. The method toCurveGrid is used to convert a non-generic\n",
    "grid to curvilinear representation:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = cdms2.open('https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/clt.nc')\n",
    "clt = f('clt')\n",
    "rectgrid = clt.getGrid()\n",
    "print(\"rectgrid:\", rectgrid.shape)\n",
    "\n",
    "curvegrid = rectgrid.toCurveGrid()\n",
    "print(\"curvegrid:\", curvegrid)\n",
    "\n",
    "genericgrid = curvegrid.toGenericGrid()\n",
    "print(\"genericgrid: \", genericgrid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regridding\n",
    "\n",
    "Regridding is the process of mapping variables from one grid to another.\n",
    "CDMS supports two forms of regridding. Which one you use depends on the\n",
    "class of grids being transformed:\n",
    "\n",
    "-  To interpolate from one rectangular grid to another, use the built-in\n",
    "   CDMS regridder. CDMS also has built-in regridders to interpolate from\n",
    "   one set of pressure levels to another, or from one vertical\n",
    "   cross-section to another.\n",
    "-  To interpolate from any lat-lon grid, rectangular or non-rectangular,\n",
    "   use the ``SCRIP`` regridder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CDMS Regridder\n",
    "\n",
    "The built-in CDMS regridder is used to transform data from one\n",
    "rectangular grid to another. For example, to regrid variable ``u`` (from\n",
    "a rectangular grid) to a 96x192 rectangular Gaussian grid:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = cdms2.open('https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/clt.nc')\n",
    "u = f('u')\n",
    "print(\"u.shape:\", u.shape)\n",
    "\n",
    "t63_grid = cdms2.createGaussianGrid(96)\n",
    "u63 = u.regrid(t63_grid)\n",
    "print(\"U63.shape\",u63.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To regrid a variable ``uold`` to the same grid as variable ``vnew``:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = cdms2.open('https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/clt.nc')\n",
    "f2 = cdms2.open('https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/geos5-sample.nc')\n",
    "uold = f('u')\n",
    "unew = f2('uwnd')\n",
    "print(\"uold.shape\", uold.shape)\n",
    "unew.shape\n",
    "print(\"unew.shape\", unew.shape)\n",
    "t63_grid = unew.getGrid() # Obtain the grid for vnew\n",
    "u63 = uold.regrid(t63_grid)\n",
    "u63.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SCRIP Regridder\n",
    "\n",
    "To interpolate between any lat-lon grid types, the SCRIP regridder may\n",
    "be used. The SCRIP package was developed at [Los Alamos National\n",
    "Laboratory] (https://github.com/SCRIP-Project/SCRIP).\n",
    "SCRIP is written in Fortran 90, and must be built and installed\n",
    "separately from the CDAT/CDMS installation.\n",
    "\n",
    "The steps to regrid a variable are:\n",
    "\n",
    "```(external to CDMS)```\n",
    "\n",
    "1. Obtain or generate the grids, in SCRIP netCDF format.\n",
    "2. Run SCRIP to generate a *remapping* file.\n",
    "\n",
    "```(in CDMS)```\n",
    "\n",
    "1. Read the regridder from the SCRIP remapping file.\n",
    "2. Call the regridder with the source data, returning data on the target\n",
    "   grid.\n",
    "\n",
    "Steps 1 and 2 need only be done once. The regridder can be used as often\n",
    "as necessary.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import regrid package for regridder functions\n",
    "import regrid2, cdms2\n",
    "\n",
    "# Get the source variable\n",
    "f = cdms2.open('https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/xieArkin-T42.nc')\n",
    "dat = f('prc')\n",
    "f.close()\n",
    "\n",
    "# Read the regridder from the remapper file\n",
    "remapf = cdms2.open('https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/rmp_T42_to_POP43_conserv.nc')\n",
    "regridf = regrid2.readRegridder(remapf)\n",
    "remapf.close()\n",
    "\n",
    "# Regrid the source variable\n",
    "popdat = regridf(dat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Regridding is discussed in [Chapter 4](https://cdms.readthedocs.io/en/latest/manual/cdms_4.html#regridding-data)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Time types\n",
    "\n",
    "CDMS provides extensive support for time values in the cdtime module.\n",
    "cdtime also defines a set of calendars , specifying the number of days\n",
    "in a given month.\n",
    "\n",
    "Two time types are available: relative time and component time .\n",
    "Relative time is time relative to a fixed base time. It consists of:\n",
    "\n",
    "-  a ``units`` string, of the form ``\"units since basetime\"`` , and\n",
    "-  a floating-point ``value``\n",
    "\n",
    "For example, the time \"28.0 days since 1996-1-1\" has value= 28.0 , and\n",
    "units=\" days since 1996-1-1\". To create a relative time type:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cdtime\n",
    "rt = cdtime.reltime(28.0, \"days since 1996-1-1\")\n",
    "print(rt)\n",
    "print(rt.value)\n",
    "print(rt.units)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A component time consists of the integer fields year, month, day, hour,\n",
    "minute , and the floating-point field second . For example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ct = cdtime.comptime(1996,2,28,12,10,30)\n",
    "print(ct)\n",
    "print(ct.year)\n",
    "print(ct.month)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The conversion functions tocomp and torel convert between the two\n",
    "representations. For instance, suppose that the time axis of a variable\n",
    "is represented in units \" days since 1979\" . To find the coordinate\n",
    "value corresponding to January 1, 1990:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ct = cdtime.comptime(1990,1)\n",
    "rt = ct.torel(\"days since 1979\")\n",
    "print(rt.value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time values can be used to specify intervals of time to read. The syntax\n",
    "time=(c1,c2) specifies that data should be read for times t such that\n",
    "c1<=t<=c2:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cdat_info\n",
    "fh = cdms2.open(\"https://aims3.llnl.gov/thredds/dodsC/user_pub_work/CDAT-sample/v1/tas_6h.nc\")\n",
    "c1 = cdtime.comptime(1980,1)\n",
    "c2 = cdtime.comptime(1980,2)\n",
    "tas = fh['tas']\n",
    "print(tas.shape)\n",
    "x = tas.subRegion(time=(c1,c2))\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or string representations can be used:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tas = fh['tas']\n",
    "x = tas.subRegion(time=('1980-1','1980-2'))\n",
    "print(x.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time types are described in [Chapter 3](https://cdms.readthedocs.io/en/latest/manual/cdms_3.html#module-cdtime)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}