Not possible to modify packing of fortran diagnostics in a prognostic run

[spencerkclark]

When validating aspects of the fortran model it can be useful to output diagnostics in full double precision (what we have configured the model to use internally) instead of single precision. fv3config supports modifying the packing, but the prepare-config infrastructure in fv3net does not.

While I've often run the fortran executable in isolation in doing these validation tasks—which gets around this since it involves directly using fv3config's write_run_directory—it can be convenient to use the fv3net infrastructure to do test runs as well.

Example

Take the following config as an example:

base_version: SHiELD/v0.1
wrapper: shield.wrapper
field_table: gs://vcm-fv3config/config/field_table/TKE-EDMF/v1.1/field_table
initial_conditions:
  base_url: gs://vcm-ml-raw-flexible-retention/2023-02-27-C384-coarsening-sensitivity-runs/pressure-level/C384-to-C48-restart-files
  timestep: "20160801.030000"
fortran_diagnostics:
- name: sfc_dt_atmos.zarr
  chunks:
    time: 96
  times:
    frequency: 900
    kind: interval
  variables:
  - {module_name: dynamics, field_name: grid_lont, output_name: lon, packing: 1}

If we provide this to prepare-config we get the following error:

root@934c3b24235b:/fv3net/workflows/prognostic_c48_run# prepare-config example.yaml
2023-12-15 21:28:03.473768: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-12-15 21:28:03.473831: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING:root:Config not found...using defaults.
Traceback (most recent call last):
  File "/usr/local/bin/prepare-config", line 11, in <module>
    load_entry_point('prognostic-run', 'console_scripts', 'prepare-config')()
  File "/fv3net/workflows/prognostic_c48_run/runtime/segmented_run/prepare_config.py", line 262, in main
    prepare_config(args)
  File "/fv3net/workflows/prognostic_c48_run/runtime/segmented_run/prepare_config.py", line 254, in prepare_config
    final = to_fv3config(dict_)
  File "/fv3net/workflows/prognostic_c48_run/runtime/segmented_run/prepare_config.py", line 238, in to_fv3config
    user_config = HighLevelConfig.from_dict(dict_)
  File "/fv3net/workflows/prognostic_c48_run/runtime/segmented_run/prepare_config.py", line 125, in from_dict
    return dacite.from_dict(
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 63, in from_dict
    value = _build_value(type_=field.type, data=transformed_value, config=config)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 90, in _build_value
    return _build_value_for_collection(collection=type_, data=data, config=config)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 140, in _build_value_for_collection
    return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 140, in <genexpr>
    return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 92, in _build_value
    return from_dict(data_class=type_, data=data, config=config)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 63, in from_dict
    value = _build_value(type_=field.type, data=transformed_value, config=config)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 90, in _build_value
    return _build_value_for_collection(collection=type_, data=data, config=config)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 140, in _build_value_for_collection
    return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 140, in <genexpr>
    return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 92, in _build_value
    return from_dict(data_class=type_, data=data, config=config)
  File "/usr/local/lib/python3.8/dist-packages/dacite/core.py", line 53, in from_dict
    raise UnexpectedDataError(keys=extra_fields)
dacite.exceptions.UnexpectedDataError: can not match "packing" to any data class field

I think this is in part because packing is not an attribute defined for the FortranVariableNameSpec class, which later cannot be passed when constructing fv3config.DiagFieldConfig objects:

fv3net/workflows/prognostic_c48_run/runtime/diagnostics/fortran.py

Lines 17 to 32 in f852766

	@dataclasses.dataclass
	class FortranVariableNameSpec:
	"""Names required to specify Fortran diagnostic variable.
	Attributes:
	module_name: Fortran module of diagnostic
	field_name: name of variable in Fortran code
	output_name: name to use for variable in output diagnostic file
	"""
	module_name: str
	field_name: str
	output_name: str
	def in_physics_module(self) -> bool:
	return self.module_name in FORTRAN_PHYSICS_MODULES