From b3ceddc805c75b1213b94aada1a3b1c4fe53063c Mon Sep 17 00:00:00 2001 From: JonathanGregory Date: Thu, 16 Nov 2023 11:16:10 +0000 Subject: [PATCH 01/10] units_metadata --- appc.adoc | 6 ++- appe.adoc | 10 +++-- ch03.adoc | 105 ++++++++++++++++++++++++++++++++++++++++------- ch04.adoc | 8 +--- conformance.adoc | 4 ++ 5 files changed, 108 insertions(+), 25 deletions(-) diff --git a/appc.adoc b/appc.adoc index 841290b7..4258a23a 100644 --- a/appc.adoc +++ b/appc.adoc @@ -20,7 +20,7 @@ In the __Units__ column, __u__ indicates units dimensionally equivalent to those The use of this modifier is deprecated and the standard_name number_of_observations is preferred to describe this type of metadata variable. | `standard_error` | __u__ -| The uncertainty of the data value. +| ^*^The uncertainty of the data value. The standard error includes both systematic and statistical uncertainty. By default it is assumed that the values supplied are for one standard error. If the values supplied are for some multiple of the standard error, the `standard_error` ancillary variable should have an attribute **`standard_error_multiplier`** stating the multiplication factor. @@ -30,3 +30,7 @@ If the values supplied are for some multiple of the standard error, the `standar The variable should have **`flag_values`** or **`flag_masks`** (or both) and **`flag_meanings`** attributes to show how it should be interpreted (<>). The use of this modifier is deprecated and the standard_name status_flag is preferred to describe this type of metadata variable. |=============== + +^*^ The definition of this modifier implies that if _u_ is a either unit of temperature, or a unit of temperature multiplied by some other unit, the temperature in _u_ must be interpreted as a temperature difference. +Therefore the **`units_metadata`** attribute, if present, must have the value `temperature: difference`, even if the corresponding data variable without the modifier would have `units_metadata="temperature: on_scale"`. +See <> for explanation. diff --git a/appe.adoc b/appe.adoc index 84118694..95206e46 100644 --- a/appe.adoc +++ b/appe.adoc @@ -41,15 +41,17 @@ This is the default method for a quantity that is extensive with respect to the | `mode` | __u__ | Mode (most common value) -| `range` | __u__ | Absolute difference between maximum and minimum +| `range` | __u__ | ^*^Absolute difference between maximum and minimum | `root_mean_square` | __u__ | Root mean square (RMS) -| `standard_deviation` | __u__ | Standard deviation +| `standard_deviation` | __u__ | ^*^Standard deviation | `sum_of_squares` | __u^2^__ | Sum of squares -| `variance` | __u^2^__ | Variance +| `variance` | __u^2^__ | ^*^Variance |=============== - +^*^ The definition of this method implies that if _u_ is a either a unit of temperature, or a unit of temperature multiplied by some other unit, the temperature in _u_ must be interpreted as a temperature difference. +Therefore the **`units_metadata`** attribute, if present, must have the value `temperature: difference`. +See <> for explanation. diff --git a/ch03.adoc b/ch03.adoc index 6f49b50c..373945c6 100644 --- a/ch03.adoc +++ b/ch03.adoc @@ -14,20 +14,29 @@ But since it is an optional attribute, applications that implement these standar === Units The **`units`** attribute is required for all variables that represent dimensional quantities (except for boundary variables defined in <> and climatology variables defined in <>). -The value of the **`units`** attribute is a string that can be recognized by the UDUNITS package <>, with a few exceptions that are given below. +The **`units`** attribute is permitted but not required for dimensionless quantities (see <>). + +The value of the **`units`** attribute is a string that can be recognized by the UDUNITS package <>, with the exceptions that are given in <> and <>. Note that case is significant in the **`units`** strings. +Note also that CF depends on UDUNITS only for the definition of legal **`units`** strings. +CF does not assume or require that the UDUNITS software will be used for **`units`** conversion. +In most **`units`** conversions, the sole operation on the data is multiplication by a scale factor. +Special treatment is required in converting the **`units`** of variables that involve temperature (<>) and the **`units`** of time coordinate variables (<>). The COARDS convention prohibits the unit `degrees` altogether, but this unit is not forbidden by the CF convention because it may in fact be appropriate for a variable containing, say, solar zenith angle. The unit `degrees` is also allowed on coordinate variables such as the latitude and longitude coordinates of a transformed grid. In this case the coordinate values are not true latitudes and longitudes which must always be identified using the more specific forms of `degrees` as described in <> and <>. -Units are not required for dimensionless quantities. + +[[dimensionless-units, Section 3.1.1, "Dimensionless units"]] +=== Dimensionless units + A variable with no **`units`** attribute is assumed to be dimensionless. However, a **`units`** attribute specifying a dimensionless unit may optionally be included. The canonical unit (see also <>) for dimensionless quantities that represent fractions, or parts of a whole, is `1`. -When a dimensionless quantity is a ratio of dimensional quantities, CF suggests that it may be informative to users of data if the **`units`** are given as ratio of dimensional units, for instance `mg kg-1` for a mass ratio of 1e-6, or `microlitre litre-1` for a volume ratio of 1e-6. - The UDUNITS package defines a few dimensionless units, such as `percent`, `ppm` (parts per million, 1e-6), and `ppb` (parts per billion, 1e-9). +When a dimensionless quantity is a ratio of dimensional quantities, CF suggests that it may be informative to users of data if the **`units`** are given as ratio of dimensional units, for instance `mg kg-1` for a mass ratio of 1e-6, or `microlitre litre-1` for a volume ratio of 1e-6. + The CF convention supports dimensionless units that are UDUNITS compatible, with one exception, concerning the dimensionless units defined by UDUNITS for volume ratios, such as `ppmv` and `ppbv`. These units are allowed in the **`units`** attribute by CF only if the data variable has no **`standard_name`**. These units are prohibited by CF if there is a **`standard_name`**, because the **`standard_name`** defines whether the quantity is a volume ratio, so the **`units`** are needed only to indicate a dimensionless number. @@ -41,9 +50,76 @@ The UDUNITS syntax that allows scale factors and offsets to be applied to a unit The application of any scale factors or offsets to data should be indicated by the **`scale_factor`** and **`add_offset`** attributes. Use of these attributes for data packing, which is their most important application, is discussed in detail in <>. -UDUNITS recognizes the following prefixes and their abbreviations. + +[[temperature-units, Section 3.1.2, "Temperature units"]] +=== Temperature units + +The **`units`** of temperature imply an origin (i.e. zero point) for the associated measurement scale. +When the temperature value is the degree of warmth with respect to the origin of the measurement scale, we call it an _on-scale temperature_. +When **`units`** of on-scale temperature are converted, the data may require the addition of an offset as well as multiplication by a scale factor, because the physical meaning of a numerical value of zero for an on-scale temperature depends on the unit of measurement. +On-scale temperature is _unique_ among quantities in this respect; for all other quantities, zero means the same whatever the unit of measurement. For example (using **bold** to indicate a numerical data value), **0** `kilogram` is the same mass as **0** `pound`, but **0** `degC` is not the same temperature as **0** `degF` (= **-17.8** `degC`), because these two temperature **`units`** implicitly refer to measurement scales which have different origins. + +On the other hand, when the temperature value is a _temperature difference_, which compares two on-scale temperatures with the same origin, the value of that origin is irrelevant as it cancels out when taking the difference. +Therefore to change the **`units`** of a temperature difference requires only multiplication by a scale factor, without the addition of an offset. + +The **`units`** attribute does not distinguish between on-scale temperatures and temperature differences. +This ambiguity also affects units of temperature raised to some power e.g. `K^2` or multiplied by other units e.g. `W m-2 K-1`, `degF/foot` or `degC m s-1`. +A **`standard_name`** (<>) or **`standard_name`** modifier (<>) may clarify the intention, but they are optional. +Some statistical operations described by the **`cell_methods`** attribute (<>; <>) imply that temperature must be interpreted as temperature difference, but this attribute is optional too. + +In order to change the **`units`** correctly, it is essential to know whether a temperature is on-scale or a difference. +Therefore this standard strongly recommends that any variable whose **`units`** involve a temperature unit should also have a **`units_metadata`** attribute to make the distinction. +This attribute must have one of the following three values: `temperature: on_scale`, `temperature: difference`, `temperature: unknown`. +The **`units_metadata`** attribute, **`standard_name`** modifier (<>) and **`cell_methods`** attribute (<>) must be consistent if present. + +[[use-of-units-metadata-ex]] +[caption="Example 3.1. "] +.Use of **`units_metadata`** to distinguish temperature quantities +==== + +---- +variables: + float Tonscale; + Tonscale:long_name="global-mean surface temperature"; + Tonscale:standard_name="surface_temperature"; + Tonscale:units="degC"; + Tonscale:units_metadata="temperature: on_scale"; + Tonscale:cell_methods="area: mean"; + float Tdifference; + Tdifference:long_name="change in global-mean surface temperature relative to pre-industrial"; + Tdifference:standard_name="surface_temperature"; + Tdifference:units="degC"; + Tdifference:units_metadata="temperature: difference"; + Tdifference:cell_methods="area: mean"; +---- +==== + +With `temperature: unknown`, correct conversion of the **`units`** cannot be guaranteed. +This value of **`units_metadata`** indicates that the data-writer does not know whether the temperature is on-scale or a difference. +If the **`units_metadata`** attribute is not present, the data-reader should assume `temperature: unknown`. +The **`units_metadata`** attribute was introduced in CF 1.11. +In data written according to versions before 1.11, `temperature: unknown` should be assumed for all **`units`** involving temperature, if it cannot be deduced from other metadata. +We note (for guidance only for `temperature: unknown`, not as a CF convention) that the UDUNITS software assumes `temperature: on_scale` for **`units`** strings containing only a unit of temperature, and `temperature: difference` for **`units`** strings in which a unit of temperature is raised to any power other than unity, or multiplied or divided by any other unit. + +With `temperature: on_scale`, correct conversion can be guaranteed only for pure temperature **`units`**. +If the unit is an on-scale temperature multiplied by some other quantity, it is generally not possible to convert the data correctly from the **`units`** given, whether the canonical or some other, to any other **`units`**, because the **`units`** does not give sufficient information. +For example, the **`standard_name`** of `integral_wrt_depth_of_product_of_conservative_temperature_and_sea_water_density` specifies a canonical unit of `kg degree_C m-2`. +A numerical value of **1** with `units="kg degree_C m-2"` could mean **1** `degree_C` multiplied by **1** `kg m-2`, or **10** `degree_C` multiplied by **0.1** `kg m-2`. +If `temperature: on_scale`, these convert to **274.15** `kg K m-2` and **28.315** `kg K m-2`, respectively, and there are infinitely many other possibilities. + + +[[units-multiples, Section 3.1.3, "Scale factors and offsets"]] +=== Scale factors and offsets + +UDUNITS recognises the SI prefixes shown in <> for decimal multiples and submultiples of units, and allows them to be applied to non-SI units as well. +UDUNITS offers a syntax for indicating arbitrary scale factors and offsets to be applied to a unit. +(Note that this is different from the scale factors and offsets used for converting between **`units`**, as discussed for temperature in <>.) +This UDUNITS syntax for arbitrary transformation of **`units`** is not supported by **the CF** standard, except for the case of specifying reference time (<>). +The application of any scale factors or offsets to data should be indicated by the **`scale_factor`** and **`add_offset`** attributes. +Use of these attributes for data packing, which is their most important application, is discussed in detail in <>. + [[table-supported-units]] -.Supported Units +.Prefixes for decimal multiples and submultiples of units [options="header",caption="Table 3.1. "] |=============== | Factor | Prefix | Abbreviation | | Factor | Prefix | Abbreviation @@ -59,6 +135,7 @@ UDUNITS recognizes the following prefixes and their abbreviations. | 1e24 | yotta | Y | | 1e-24 | yocto | y |=============== + [[long-name, Section 3.2, "Long Name"]] === Long Name @@ -93,7 +170,7 @@ Unless it is dimensionless, a variable with a **`standard_name`** attribute must description:: The description is meant to clarify the qualifiers of the fundamental quantities such as which surface a quantity is defined on or what the flux sign conventions are. We don't attempt to provide precise definitions of fundumental physical quantities (e.g., temperature) which may be found in the literature. -The description may define rules on the variable type, attributes and coordinates which must be complied with by any variable carrying that standard name (such as in example 3.4). +The description may define rules on the variable type, attributes and coordinates which must be complied with by any variable carrying that standard name (such as in <>). When appropriate, the table entry also contains the corresponding GRIB parameter code(s) (from ECMWF and NCEP) and AMIP identifiers. @@ -116,7 +193,7 @@ Other types of quantity modifiers are expressed using the optional modifier part The permissible values of these modifiers are given in <>. [[use-of-standard-name-ex]] -[caption="Example 3.1. "] +[caption="Example 3.2. "] .Use of **`standard_name`** ==== @@ -146,7 +223,7 @@ The nature of the relationship between variables associated via **`ancillary_var The variables listed by the **`ancillary_variables`** attribute will often have the standard name of the variable which points to them including a modifier (<>) to indicate the relationship. [[instrument-data-ex]] -[caption="Example 3.2. "] +[caption="Example 3.3. "] .Ancillary instrument data ==== @@ -182,7 +259,7 @@ Several examples are listed below: The following example illustrates the use of three of these flags to represent two independent quality control tests and an aggregate flag that combines the results of the two tests. [[quality-flag-ex]] -[caption="Example 3.3. "] +[caption="Example 3.4. "] .Ancillary quality flag data ==== @@ -224,7 +301,7 @@ If multi-word phrases are used to describe the flag values, then the words withi The following example illustrates the use of flag values to express a speed quality with an enumerated status code. [[flag-variable-flag-values-ex]] -[caption="Example 3.4. "] +[caption="Example 3.5. "] .A flag variable, using **`flag_values`** ==== @@ -253,7 +330,7 @@ The following example illustrates the use of flag_masks to express six sensor st [[flag-variable-flag-masks-ex]] -[caption="Example 3.5. "] +[caption="Example 3.6. "] .A flag variable, using **`flag_masks`** ==== @@ -277,7 +354,7 @@ The following example illustrates this using integer flag values for a variable [[region-variable-flag-values-ex]] -[caption="Example 3.6. "] +[caption="Example 3.7. "] .A region variable, using **`flag_values`** ==== @@ -301,7 +378,7 @@ Each **`flag_values`** and **`flag_masks`** value must coincide with a **`flag_m The following example illustrates the use of **`flag_masks`** and **`flag_values`** to express two sensor status conditions and one enumerated status code. [[flag-variable-flag-masks-flag-values-ex]] -[caption="Example 3.7. "] +[caption="Example 3.8. "] .A flag variable, using **`flag_masks`** and **`flag_values`** ==== diff --git a/ch04.adoc b/ch04.adoc index cf8114af..8aec3859 100644 --- a/ch04.adoc +++ b/ch04.adoc @@ -97,9 +97,7 @@ Optionally, the longitude type may be indicated additionally by providing the ** Coordinates of longitude with respect to a rotated pole should be given units of **`degrees`**, not **`degrees_east`** or equivalents, because applications which use the units to identify axes would have no means of distinguishing such an axis from real longitude, and might draw incorrect coastlines, for instance. - - -[[vertical-coordinate]] +[[vertical-coordinate, Section 4.3, "Vertical Coordinate"]] === Vertical (Height or Depth) Coordinate Variables representing dimensional height or depth axes must always explicitly include the **`units`** attribute; there is no default value. @@ -214,9 +212,7 @@ The `computed_standard_name` attribute indicates that the values in variable `p` would have a `standard_name` of `air_pressure`. - - -[[time-coordinate]] +[[time-coordinate, Section 4.4, "Time Coordinate"]] === Time Coordinate Variables representing reference time must always explicitly include the **`units`** attribute; there is no default value. diff --git a/conformance.adoc b/conformance.adoc index d158a911..923b22e6 100644 --- a/conformance.adoc +++ b/conformance.adoc @@ -147,11 +147,15 @@ Exceptions are boundary and climatology variables. * The type of the **`units`** attribute is a string that must be recognizable by the UDUNITS package. Exceptions are the units **`level`**, **`layer`**, and **`sigma_level`**. * Dimensionless units for volume fractions defined by UDUNITS (**`ppv`**, **`ppmv`**, **`ppbv`**, **`pptv`**, **`ppqv`**) are not allowed in the **`units`** attribute of any variable which also has a **`standard_name`** attribute. +* If present, the **`units_metadata`** attribute must have one of these values: `temperature: on_scale`, `temperature: difference`, `temperature: unknown`. * The **`units`** of a variable that specifies a **`standard_name`** must be physically equivalent to the canonical units given in the standard name table, as modified by the **`standard_name`** modifier, if there is one, according to Appendix C, and then modified by all the methods listed in order by the **`cell_methods`** attribute, if one is present, according to Appendix E. +* If the **`standard_name`** attribute includes the `standard_error` modifier, the **`units_metadata`** attribute, if present, must have the value `temperature: difference`. +* If the **`cell_methods`** attribute includes any entry with any of the methods `range`, `standard_deviation` or `variance`, the **`units_metadata`** attribute, if present, must have the value `temperature: difference`. *Recommendations:* * The units **`level`**, **`layer`**, and **`sigma_level`** are deprecated. +* Any variable whose **`units`** involve a temperature unit should also have a **`units_metadata`** attribute. [[section-8]] From e7482a2df6c9c07c497962fa5b1dfb5be83bd135 Mon Sep 17 00:00:00 2001 From: JonathanGregory Date: Thu, 16 Nov 2023 13:54:16 +0000 Subject: [PATCH 02/10] units_metadata --- appc.adoc | 4 ++-- appe.adoc | 8 ++++---- ch03.adoc | 8 ++++---- toc-extra.adoc | 15 ++++++++------- 4 files changed, 18 insertions(+), 17 deletions(-) diff --git a/appc.adoc b/appc.adoc index 4258a23a..55689133 100644 --- a/appc.adoc +++ b/appc.adoc @@ -20,7 +20,7 @@ In the __Units__ column, __u__ indicates units dimensionally equivalent to those The use of this modifier is deprecated and the standard_name number_of_observations is preferred to describe this type of metadata variable. | `standard_error` | __u__ -| ^*^The uncertainty of the data value. +| *The uncertainty of the data value. The standard error includes both systematic and statistical uncertainty. By default it is assumed that the values supplied are for one standard error. If the values supplied are for some multiple of the standard error, the `standard_error` ancillary variable should have an attribute **`standard_error_multiplier`** stating the multiplication factor. @@ -31,6 +31,6 @@ The variable should have **`flag_values`** or **`flag_masks`** (or both) and **` The use of this modifier is deprecated and the standard_name status_flag is preferred to describe this type of metadata variable. |=============== -^*^ The definition of this modifier implies that if _u_ is a either unit of temperature, or a unit of temperature multiplied by some other unit, the temperature in _u_ must be interpreted as a temperature difference. +*The definition of this modifier implies that if _u_ is a either unit of temperature, or a unit of temperature multiplied by some other unit, the temperature in _u_ must be interpreted as a temperature difference. Therefore the **`units_metadata`** attribute, if present, must have the value `temperature: difference`, even if the corresponding data variable without the modifier would have `units_metadata="temperature: on_scale"`. See <> for explanation. diff --git a/appe.adoc b/appe.adoc index 95206e46..19b7d76e 100644 --- a/appe.adoc +++ b/appe.adoc @@ -41,17 +41,17 @@ This is the default method for a quantity that is extensive with respect to the | `mode` | __u__ | Mode (most common value) -| `range` | __u__ | ^*^Absolute difference between maximum and minimum +| `range` | __u__ | *Absolute difference between maximum and minimum | `root_mean_square` | __u__ | Root mean square (RMS) -| `standard_deviation` | __u__ | ^*^Standard deviation +| `standard_deviation` | __u__ | *Standard deviation | `sum_of_squares` | __u^2^__ | Sum of squares -| `variance` | __u^2^__ | ^*^Variance +| `variance` | __u^2^__ | *Variance |=============== -^*^ The definition of this method implies that if _u_ is a either a unit of temperature, or a unit of temperature multiplied by some other unit, the temperature in _u_ must be interpreted as a temperature difference. +*The definition of this method implies that if _u_ is a either a unit of temperature, or a unit of temperature multiplied by some other unit, the temperature in _u_ must be interpreted as a temperature difference. Therefore the **`units_metadata`** attribute, if present, must have the value `temperature: difference`. See <> for explanation. diff --git a/ch03.adoc b/ch03.adoc index 373945c6..83027f79 100644 --- a/ch03.adoc +++ b/ch03.adoc @@ -29,7 +29,7 @@ In this case the coordinate values are not true latitudes and longitudes which m [[dimensionless-units, Section 3.1.1, "Dimensionless units"]] -=== Dimensionless units +==== Dimensionless units A variable with no **`units`** attribute is assumed to be dimensionless. However, a **`units`** attribute specifying a dimensionless unit may optionally be included. @@ -52,7 +52,7 @@ Use of these attributes for data packing, which is their most important applicat [[temperature-units, Section 3.1.2, "Temperature units"]] -=== Temperature units +==== Temperature units The **`units`** of temperature imply an origin (i.e. zero point) for the associated measurement scale. When the temperature value is the degree of warmth with respect to the origin of the measurement scale, we call it an _on-scale temperature_. @@ -109,7 +109,7 @@ If `temperature: on_scale`, these convert to **274.15** `kg K m-2` and **28.315* [[units-multiples, Section 3.1.3, "Scale factors and offsets"]] -=== Scale factors and offsets +==== Scale factors and offsets UDUNITS recognises the SI prefixes shown in <> for decimal multiples and submultiples of units, and allows them to be applied to non-SI units as well. UDUNITS offers a syntax for indicating arbitrary scale factors and offsets to be applied to a unit. @@ -170,7 +170,7 @@ Unless it is dimensionless, a variable with a **`standard_name`** attribute must description:: The description is meant to clarify the qualifiers of the fundamental quantities such as which surface a quantity is defined on or what the flux sign conventions are. We don't attempt to provide precise definitions of fundumental physical quantities (e.g., temperature) which may be found in the literature. -The description may define rules on the variable type, attributes and coordinates which must be complied with by any variable carrying that standard name (such as in <>). +The description may define rules on the variable type, attributes and coordinates which must be complied with by any variable carrying that standard name (such as in Example 3.5). When appropriate, the table entry also contains the corresponding GRIB parameter code(s) (from ECMWF and NCEP) and AMIP identifiers. diff --git a/toc-extra.adoc b/toc-extra.adoc index 43b4e659..eac9bb14 100644 --- a/toc-extra.adoc +++ b/toc-extra.adoc @@ -37,13 +37,14 @@ J.5. <> [%hardbreaks] 2.1. <> -3.1. <> -3.2. <> -3.3. <> -3.4. <> -3.5. <> -3.6. <> -3.7. <> +3.1. <> +3.2. <> +3.3. <> +3.4. <> +3.5. <> +3.6. <> +3.7. <> +3.8. <> 4.1. <> 4.2. <> 4.3. <> From 8bc34cc4ecd01a0c98f5ac849479ee6861d3df09 Mon Sep 17 00:00:00 2001 From: Jonathan Gregory Date: Thu, 16 Nov 2023 16:26:33 +0000 Subject: [PATCH 03/10] units_metadata --- cf-conventions.adoc | 2 +- history.adoc | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 0ab503e7..73728ce3 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -1,6 +1,6 @@ include::version.adoc[] = NetCDF Climate and Forecast (CF) Metadata Conventions -Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; Jon{nbsp}Blower; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud; Randy{nbsp}Horne; Timothy{nbsp}Whiteaker; David{nbsp}Blodgett; Charlie{nbsp}Zender; Daniel{nbsp}Lee; David{nbsp}Hassell; Alan{nbsp}D.{nbsp}Snow; Tobias{nbsp}Kölling; Dave{nbsp}Allured; Aleksandar{nbsp}Jelenak; Anders{nbsp}Meier{nbsp}Soerensen; Lucile{nbsp}Gaultier; Sylvain{nbsp}Herlédan; Fernando{nbsp}Manzano; Lars{nbsp}Bärring +Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; Jon{nbsp}Blower; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud; Randy{nbsp}Horne; Timothy{nbsp}Whiteaker; David{nbsp}Blodgett; Charlie{nbsp}Zender; Daniel{nbsp}Lee; David{nbsp}Hassell; Alan{nbsp}D.{nbsp}Snow; Tobias{nbsp}Kölling; Dave{nbsp}Allured; Aleksandar{nbsp}Jelenak; Anders{nbsp}Meier{nbsp}Soerensen; Lucile{nbsp}Gaultier; Sylvain{nbsp}Herlédan; Fernando{nbsp}Manzano; Lars{nbsp}Bärring; Chris{nbsp}Barker Version {current-version}, 31 August, 2022: See https://cfconventions.org for further information :doctype: book :pdf-folio-placement: physical diff --git a/history.adoc b/history.adoc index fe8f1467..d546791d 100644 --- a/history.adoc +++ b/history.adoc @@ -7,6 +7,7 @@ === Working version (most recent first) +* {issues}481[Issue #481]: Introduce **`units_metadata`** attribute and clarify some other aspects of **`units`** * {issues}383[Issue #383]: Link to the CF website and deleted the Preface section * {issues}472[Issue #472]: Fix incorrect formating for some \<= symbols * {issues}458[Issue #458]: Fix broken link to Unidata documentation. From 269f51085a1cd53208f50c2fe7f2a17beba0d801 Mon Sep 17 00:00:00 2001 From: Jonathan Gregory Date: Thu, 16 Nov 2023 16:28:29 +0000 Subject: [PATCH 04/10] units_metadata --- cf-conventions.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index 73728ce3..0ab503e7 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -1,6 +1,6 @@ include::version.adoc[] = NetCDF Climate and Forecast (CF) Metadata Conventions -Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; Jon{nbsp}Blower; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud; Randy{nbsp}Horne; Timothy{nbsp}Whiteaker; David{nbsp}Blodgett; Charlie{nbsp}Zender; Daniel{nbsp}Lee; David{nbsp}Hassell; Alan{nbsp}D.{nbsp}Snow; Tobias{nbsp}Kölling; Dave{nbsp}Allured; Aleksandar{nbsp}Jelenak; Anders{nbsp}Meier{nbsp}Soerensen; Lucile{nbsp}Gaultier; Sylvain{nbsp}Herlédan; Fernando{nbsp}Manzano; Lars{nbsp}Bärring; Chris{nbsp}Barker +Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; Jon{nbsp}Blower; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud; Randy{nbsp}Horne; Timothy{nbsp}Whiteaker; David{nbsp}Blodgett; Charlie{nbsp}Zender; Daniel{nbsp}Lee; David{nbsp}Hassell; Alan{nbsp}D.{nbsp}Snow; Tobias{nbsp}Kölling; Dave{nbsp}Allured; Aleksandar{nbsp}Jelenak; Anders{nbsp}Meier{nbsp}Soerensen; Lucile{nbsp}Gaultier; Sylvain{nbsp}Herlédan; Fernando{nbsp}Manzano; Lars{nbsp}Bärring Version {current-version}, 31 August, 2022: See https://cfconventions.org for further information :doctype: book :pdf-folio-placement: physical From dc11b8bc6f83d87ccf1a72af295af48d834d9466 Mon Sep 17 00:00:00 2001 From: JonathanGregory Date: Fri, 17 Nov 2023 13:41:55 +0000 Subject: [PATCH 05/10] units_metadata not allowed if no temperature unit --- ch03.adoc | 1 + conformance.adoc | 1 + 2 files changed, 2 insertions(+) diff --git a/ch03.adoc b/ch03.adoc index 881ec340..fca2185b 100644 --- a/ch03.adoc +++ b/ch03.adoc @@ -71,6 +71,7 @@ In order to change the **`units`** correctly, it is essential to know whether a Therefore this standard strongly recommends that any variable whose **`units`** involve a temperature unit should also have a **`units_metadata`** attribute to make the distinction. This attribute must have one of the following three values: `temperature: on_scale`, `temperature: difference`, `temperature: unknown`. The **`units_metadata`** attribute, **`standard_name`** modifier (<>) and **`cell_methods`** attribute (<>) must be consistent if present. +A variable must not have a **`units_metadata`** attribute if it has no **`units`** attribute or if its **`units`** do not involve a temperature unit. [[use-of-units-metadata-ex]] [caption="Example 3.1. "] diff --git a/conformance.adoc b/conformance.adoc index 43e99467..ffe4cac3 100644 --- a/conformance.adoc +++ b/conformance.adoc @@ -151,6 +151,7 @@ Exceptions are the units **`level`**, **`layer`**, and **`sigma_level`**. * The **`units`** of a variable that specifies a **`standard_name`** must be physically equivalent to the canonical units given in the standard name table, as modified by the **`standard_name`** modifier, if there is one, according to Appendix C, and then modified by all the methods listed in order by the **`cell_methods`** attribute, if one is present, according to Appendix E. * If the **`standard_name`** attribute includes the `standard_error` modifier, the **`units_metadata`** attribute, if present, must have the value `temperature: difference`. * If the **`cell_methods`** attribute includes any entry with any of the methods `range`, `standard_deviation` or `variance`, the **`units_metadata`** attribute, if present, must have the value `temperature: difference`. +* A variable must not have a **`units_metadata`** attribute if it has no **`units`** attribute or if its **`units`** do not involve a temperature unit. *Recommendations:* From fa30e4f0b791a47cdd2c993dca413ef8b25afe8e Mon Sep 17 00:00:00 2001 From: David Hassell Date: Tue, 31 Oct 2023 14:55:44 +0000 Subject: [PATCH 06/10] Clarify the use of compressed dimensions in related variables --- ch03.adoc | 2 ++ ch05.adoc | 2 +- ch07.adoc | 3 ++- conformance.adoc | 5 +++-- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/ch03.adoc b/ch03.adoc index 25f39ba7..ea2abf17 100644 --- a/ch03.adoc +++ b/ch03.adoc @@ -222,6 +222,8 @@ The attribute **`ancillary_variables`** is used to express these types of relati It is a string attribute whose value is a blank separated list of variable names. The nature of the relationship between variables associated via **`ancillary_variables`** must be determined by other attributes. The variables listed by the **`ancillary_variables`** attribute will often have the standard name of the variable which points to them including a modifier (<>) to indicate the relationship. +The dimensions of an ancilliary variable must be the same as or a subset of the dimensions of the variable to which it is related, but their order is not restricted, and with one exception: +If an ancillary variable of a data variable that has been compressed by gathering (<>) does not span the compressed dimension, then its dimensions may be any subset of the data variable's uncompressed dimensions, i.e. any of the dimensions of the data variable except the compressed dimension, and any of the dimensions listed by the **`compress`** attribute of the compressed coordinate variable. [[instrument-data-ex]] [caption="Example 3.3. "] diff --git a/ch05.adoc b/ch05.adoc index 13a9d216..88234a64 100644 --- a/ch05.adoc +++ b/ch05.adoc @@ -15,7 +15,7 @@ The value of the **`coordinates`** attribute is __a blank separated list of the There is no restriction on the order in which the auxiliary coordinate variables appear in the **`coordinates`** attribute string. The dimensions of an auxiliary coordinate variable must be a subset of the dimensions of the variable with which the coordinate is associated, with three exceptions. First, string-valued coordinates (<>) will have a dimension for maximum string length if the coordinate variable has a type of **`char`** rather than a type of **`string`**. -Second, if the data variable has a list dimension resulting from lossless compression by gathering (<>), its auxiliary coordinate variables may have any of the dimensions named by the compress attribute of the list variable. +Second, if an auxiliary coordinate variable of a data variable that has been compressed by gathering (<>) does not span the compressed dimension, then its dimensions may be any subset of the data variable's uncompressed dimensions, i.e. any of the dimensions of the data variable except the compressed dimension, and any of the dimensions listed by the **`compress`** attribute of the compressed coordinate variable. Third, in the ragged array representations of data (<>), special methods are needed to connect the data and coordinates. We recommend that the name of a multidimensional coordinate variable should not match the name of any of its dimensions because that precludes supplying a coordinate variable for the dimension. diff --git a/ch07.adoc b/ch07.adoc index 9cc3a5f5..aadfa032 100644 --- a/ch07.adoc +++ b/ch07.adoc @@ -195,7 +195,8 @@ To indicate extra information about the spatial properties of a variable's grid This is a string attribute comprising a list of blank-separated pairs of words of the form "**`measure: name`**". For the moment, "**`area`**" and "**`volume`**" are the only defined measures, but others may be supported in future. The "name" is the name of the variable containing the measure values, which we refer to as a "measure variable". -The dimensions of the measure variable should be the same as or a subset of the dimensions of the variable to which they are related, but their order is not restricted. +The dimensions of a measure variable must be the same as or a subset of the dimensions of the variable to which it is related, but their order is not restricted, and with one exception: +If a cell measure variable of a data variable that has been compressed by gathering (<>) does not span the compressed dimension, then its dimensions may be any subset of the data variable's uncompressed dimensions, i.e. any of the dimensions of the data variable except the compressed dimension, and any of the dimensions listed by the **`compress`** attribute of the compressed coordinate variable. In the case of area, for example, the field itself might be a function of longitude, latitude, and time, but the variable containing the area values would only include longitude and latitude dimensions (and the dimension order could be reversed, although this is not recommended). The variable must have a **`units`** attribute and may have other attributes such as a **`standard_name`**. diff --git a/conformance.adoc b/conformance.adoc index ffe4cac3..e313eeae 100644 --- a/conformance.adoc +++ b/conformance.adoc @@ -288,7 +288,7 @@ If the **`calendar`** attribute is given a non-standard value, then the attribut All specified variable names must exist in the file. * The dimensions of each auxiliary coordinate must be a subset of the dimensions of the variable they are attached to, with three exceptions. First, a label variable of type **`char`** will have a trailing dimension for the maximum string length. -Second, if the data variable to which the auxiliary coordinate variable is attached has a dimension whose coordinate variable has a **`compress`** attribute, the auxiliary coordinate variable may have any of the dimensions named by that attribute. +Second, if an auxiliary coordinate variable of a data variable that has been compressed by gathering (<>) does not span the compressed dimension, then its dimensions may be any subset of the data variable's uncompressed dimensions, i.e. any of the dimensions of the data variable except the compressed dimension, and any of the dimensions listed by the **`compress`** attribute of the compressed coordinate variable. Third, a ragged array (Chapter 9, Discrete sampling geometries and Appendix H) uses special, more indirect, methods to connect the data and coordinates. *Recommendations:* @@ -398,7 +398,8 @@ If a named variable in the **`formula_terms`** attribute of the vertical coordin * The type of the **`cell_measures`** attribute is a string whose value is list of blank separated word pairs in the form **`measure: var`**. The valid values for **`measure`** are **`area`** or **`volume`**. The **`var`** token specifies a variable that must either exist in the file or be named by the **`external_variables`** attribute. -The dimensions of the variable specified by **`var`** must be the same as, or be a subset of, the dimensions of the variable to which they are related. +The dimensions of the variable specified by **`var`** must be the same as, or be a subset of, the dimensions of the variable to which they are related, with one exception: +If a cell measure variable of a data variable that has been compressed by gathering (<>) does not span the compressed dimension, then its dimensions may be any subset of the data variable's uncompressed dimensions, i.e. any of the dimensions of the data variable except the compressed dimension, and any of the dimensions listed by the **`compress`** attribute of the compressed coordinate variable. * A measure variable must have units that are consistent with the measure type, i.e., square meters for area measures and cubic meters for volume measures. [[section-20]] From 7919d02238dda021153a058f1c2d6ee8d1194a67 Mon Sep 17 00:00:00 2001 From: David Hassell Date: Tue, 31 Oct 2023 14:57:22 +0000 Subject: [PATCH 07/10] Clarify the use of compressed dimensions in related variables --- history.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/history.adoc b/history.adoc index af2bbd8c..af898f25 100644 --- a/history.adoc +++ b/history.adoc @@ -8,11 +8,13 @@ === Working version (most recent first) * {issues}481[Issue #481]: Introduce **`units_metadata`** attribute and clarify some other aspects of **`units`** +* {issues}458[Issue #147]: Clarify the use of compressed dimensions in related variables * {issues}483[Issue #483]: Add a missing author * {issues}463[Issue #463]: Convert URLs with HTTP protocol to HTTPS if available, fixed a few dead links * {issues}383[Issue #383]: Link to the CF website and deleted the Preface section * {issues}472[Issue #472]: Fix incorrect formating for some \<= symbols * {issues}458[Issue #458]: Fix broken link to Unidata documentation. +* {issues}458[Issue #458]: Fix broken link to Unidata documentation * {issues}423[Issue #423]: Always use "strictly monotonic" when describing coordinate variables * {issues}420[Issue #420]: Add List of Figures * {issues}210[Issue #210]: Correct errors in examples H9-H11 From 4b69f48f785e1f5e69c5f650d0fa04dce568b16c Mon Sep 17 00:00:00 2001 From: David Hassell Date: Thu, 2 Nov 2023 08:43:13 +0000 Subject: [PATCH 08/10] fix typos --- ch03.adoc | 2 +- history.adoc | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/ch03.adoc b/ch03.adoc index ea2abf17..bef2dcca 100644 --- a/ch03.adoc +++ b/ch03.adoc @@ -222,7 +222,7 @@ The attribute **`ancillary_variables`** is used to express these types of relati It is a string attribute whose value is a blank separated list of variable names. The nature of the relationship between variables associated via **`ancillary_variables`** must be determined by other attributes. The variables listed by the **`ancillary_variables`** attribute will often have the standard name of the variable which points to them including a modifier (<>) to indicate the relationship. -The dimensions of an ancilliary variable must be the same as or a subset of the dimensions of the variable to which it is related, but their order is not restricted, and with one exception: +The dimensions of an ancillary variable must be the same as or a subset of the dimensions of the variable to which it is related, but their order is not restricted, and with one exception: If an ancillary variable of a data variable that has been compressed by gathering (<>) does not span the compressed dimension, then its dimensions may be any subset of the data variable's uncompressed dimensions, i.e. any of the dimensions of the data variable except the compressed dimension, and any of the dimensions listed by the **`compress`** attribute of the compressed coordinate variable. [[instrument-data-ex]] diff --git a/history.adoc b/history.adoc index af898f25..168bc0b1 100644 --- a/history.adoc +++ b/history.adoc @@ -14,7 +14,6 @@ * {issues}383[Issue #383]: Link to the CF website and deleted the Preface section * {issues}472[Issue #472]: Fix incorrect formating for some \<= symbols * {issues}458[Issue #458]: Fix broken link to Unidata documentation. -* {issues}458[Issue #458]: Fix broken link to Unidata documentation * {issues}423[Issue #423]: Always use "strictly monotonic" when describing coordinate variables * {issues}420[Issue #420]: Add List of Figures * {issues}210[Issue #210]: Correct errors in examples H9-H11 From 956a47a0b87270e5c65bca532f5541bb9b3df02f Mon Sep 17 00:00:00 2001 From: Dave Allured Date: Tue, 21 Nov 2023 08:14:27 -0700 Subject: [PATCH 09/10] Fix affiliation for Dave Allured --- cf-conventions.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cf-conventions.adoc b/cf-conventions.adoc index d78364a3..f8a8b527 100644 --- a/cf-conventions.adoc +++ b/cf-conventions.adoc @@ -40,7 +40,7 @@ include::toc-extra.adoc[] * David Hassell, NCAS and University of Reading * Alan D. Snow, Corteva Agriscience * Tobias Kölling, MPIM -* Dave Allured, NOAA +* Dave Allured, CIRES/University of Colorado/NOAA/PSL * Aleksandar Jelenak, HDF Group * Anders Meier Soerensen, EUMETSAT * Lucile Gaultier, OceanDataLab From 1751213dc70f1f02efd924bce06c07b5e3c98196 Mon Sep 17 00:00:00 2001 From: JonathanGregory Date: Tue, 21 Nov 2023 20:04:21 +0000 Subject: [PATCH 10/10] ch03.adoc --- ch03.adoc | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/ch03.adoc b/ch03.adoc index bef2dcca..6e549fc6 100644 --- a/ch03.adoc +++ b/ch03.adoc @@ -35,7 +35,7 @@ A variable with no **`units`** attribute is assumed to be dimensionless. However, a **`units`** attribute specifying a dimensionless unit may optionally be included. The canonical unit (see also <>) for dimensionless quantities that represent fractions, or parts of a whole, is `1`. The UDUNITS package defines a few dimensionless units, such as `percent`, `ppm` (parts per million, 1e-6), and `ppb` (parts per billion, 1e-9). -When a dimensionless quantity is a ratio of dimensional quantities, CF suggests that it may be informative to users of data if the **`units`** are given as ratio of dimensional units, for instance `mg kg-1` for a mass ratio of 1e-6, or `microlitre litre-1` for a volume ratio of 1e-6. +As an alternative to the canonical **`units`** of `1` or some other unitless number, the **`units`** for a dimensionless quantity may be given as a ratio of dimensional units, for instance `mg kg-1` for a mass ratio of 1e-6, or `microlitre litre-1` for a volume ratio of 1e-6. Data-producers are invited to consider whether this alternative would be more helpful to the users of their data. The CF convention supports dimensionless units that are UDUNITS compatible, with one exception, concerning the dimensionless units defined by UDUNITS for volume ratios, such as `ppmv` and `ppbv`. These units are allowed in the **`units`** attribute by CF only if the data variable has no **`standard_name`**. @@ -57,7 +57,10 @@ Use of these attributes for data packing, which is their most important applicat The **`units`** of temperature imply an origin (i.e. zero point) for the associated measurement scale. When the temperature value is the degree of warmth with respect to the origin of the measurement scale, we call it an _on-scale temperature_. When **`units`** of on-scale temperature are converted, the data may require the addition of an offset as well as multiplication by a scale factor, because the physical meaning of a numerical value of zero for an on-scale temperature depends on the unit of measurement. -On-scale temperature is _unique_ among quantities in this respect; for all other quantities, zero means the same whatever the unit of measurement. For example (using **bold** to indicate a numerical data value), **0** `kilogram` is the same mass as **0** `pound`, but **0** `degC` is not the same temperature as **0** `degF` (= **-17.8** `degC`), because these two temperature **`units`** implicitly refer to measurement scales which have different origins. +On-scale temperature is _unique_ among quantities in the respect that the origin and the unit of measurement are both defined by the **`units`** and therefore cannot be chosen independently. +For all other quantities, the origin and the unit of measurement are independent. +Converting the unit of measurement alone, without changing the origin, does not change the meaning of zero. +For example (using **bold** to indicate a numerical data value), **0** `kilogram` is the same mass as **0** `pound`, and **0** `seconds since 1970-1-1` means the same as **0** `days since 1970-1-1`, but **0** `degC` is not the same temperature as **0** `degF` (= **-17.8** `degC`), because these two temperature **`units`** implicitly refer to measurement scales which have different origins. On the other hand, when the temperature value is a _temperature difference_, which compares two on-scale temperatures with the same origin, the value of that origin is irrelevant as it cancels out when taking the difference. Therefore to convert the **`units`** of a temperature difference requires only multiplication by a scale factor, without the addition of an offset. @@ -103,10 +106,8 @@ In data written according to versions before 1.11, `temperature: unknown` should We note (for guidance only for `temperature: unknown`, not as a CF convention) that the UDUNITS software assumes `temperature: on_scale` for **`units`** strings containing only a unit of temperature, and `temperature: difference` for **`units`** strings in which a unit of temperature is raised to any power other than unity, or multiplied or divided by any other unit. With `temperature: on_scale`, correct conversion can be guaranteed only for pure temperature **`units`**. -If the unit is an on-scale temperature multiplied by some other quantity, it is generally not possible to convert the data correctly from the **`units`** given, whether the canonical or some other, to any other **`units`**, because the **`units`** does not give sufficient information. -For example, the **`standard_name`** of `integral_wrt_depth_of_product_of_conservative_temperature_and_sea_water_density` specifies a canonical unit of `kg degree_C m-2`. -A numerical value of **1** with `units="kg degree_C m-2"` could mean **1** `degree_C` multiplied by **1** `kg m-2`, or **10** `degree_C` multiplied by **0.1** `kg m-2`. -If `temperature: on_scale`, these convert to **274.15** `kg K m-2` and **28.315** `kg K m-2`, respectively, and there are infinitely many other possibilities. +If the quantity is an on-scale temperature multiplied by some other quantity, it is not possible to convert the data from the **`units`** given to any other **`units`** that involve a temperature with a different origin, given only the **`units`**. +For instance, when temperature is on-scale, a value in `kg degree_C m-2` can be converted to a value in `kg K m-2` only if we know separately the values in `degree_C` and `kg m-2` of which it is the product. [[units-multiples, Section 3.1.3, "Scale factors and offsets"]]