v0.8.1 - 2024-12-3

Primarily adds new synthesized outputs, improves performance by adding output
holding to 14 variables and works on improving the User Guide. Also removes
testing data from Git LFS.

Added

• Implements holding for AVLN, RM, CMTNUM, MOSSDZ, MOSSDEATHC, MOSSDEATHN, LAI,
  LWCLAYER, RG, SOMA SOMCR, SOMPR, SOMRAWC, INGPP.
• Synthetic outputs RECO (RHSOM, RHDWD, RM, RG) and SOC (SOMRAWC, SOMA, SOMPR,
  SOMCR). These outputs are simply the sums of their respective component
  outputs, combined to make output smaller and simpler.
• Doctests for MadsTEMDriver.
• Content to User Guide, ported from the JOSS paper draft.
• Additional Python tools to Docker image (xarray, pypdf).
• Git SHA to restart and run_status files.
• New outputs for frozen and unfrozen C pools in soil.

Changed

• Ice and liquid redistribution when splitting layers.
• Spin-up outputs set to OFF by default.
• References list in User Guide to be auto-generated from .bib file using
  Sphinx plugin.
• Reorganize various files in scripts/ directory.
• Bump fiona from 1.8.18 to 1.10.0
• Bump certifi from 2023.7.22 to 2024.7.4
• Bump tornado from 6.4.1 to 6.4.2
• Bump tornado from 6.3.3 to 6.4.1
• Bump jinja2 from 3.1.3 to 3.1.4
• Bump pillow from 10.2.0 to 10.3.0

Fixed

• Bug with the default (template) outspec csv file and extra commas.
• Bug with MadsTEMDriver function not working properly for all combinations of
  PFT, PFT-compartment, and layer soil targets.
• Bug with plotting, axes labels in MadsTEMDriver.

Removed

• Use of Git LFS for tracking testing data. Contact maintainers for access
  to testing data.
• Duplicate entry in the default (template) config/config.js.

v0.8.0 - 2024-08-14

The primary features of this release are a bug fix for soil fronts and the
addition of a new suite of tools and workflows for performing Sensitivity
Analysis and Semi-automated Calibration. These tools are implemented using a
number of new Python classes and several wrapper scripts that work with the
classes. Example code and configuration files are provided as well.

Various additional improvements were made to code organization, documentation,
variables names, logging organization, and Docker containers.

Improved

• Log level organization - categories are now debug, info, warn, monitor, fatal.
• Release instructions to allow publishing docs with proper version number.
• Documentation pages on debugging, gitk, and working with documentation.
• Code style in various Layer classes for heat capacity calculation.
• Documentation and the README.

Added

• Sensitivity Analysis and Semi-automated Calibration workflows and supporting
  tools.
• Logging for nc file access.
• Config file settings for baseline climate averaging window.
• Snow thickness and temperature outputs by layer.
• Initial test framework for comparing Pull Request to master branch.
• CLI flag to dvmdostem binary for printing SHA used for build.
• New flags for bulk enabling outputs using outspec.py command line tool.
• Basic config file for debugging in Docker with VSCode.
• Draft code for testing, both the "pull request comparison" workflow and a
  standard suite of plots to be used in the comparison.

Changed

• Documentation Example Experiment 0 paths to inputs and outputs to
  facilitated easier building of the documentation.
• Burn related output variable names to be more intuitive.
• Default year count for pre-run stage (from 10 to 100).
• Moves per-stage module settings to the config.js file.
• Reorganize scripts/ directory to put various tools into util/
  directory.
• Burned V2A C output to a summation of the monthly stored values.
• Remove N mineralization when soil temperatures are sub zero.
• Bump jinja2 version in Docker images.
• Bump pillow version in Docker images.
• Bump GDAL version in mapping support Docker image.
• Bump pygments version in Docker images.
• Bump certifi version in Docker images.
• Bump tornado version in Docker images.

Fixed

• Bug with soil fronts being inverted at stage changes and with
  Talik formation.
• Off-by-one error in fire severity index.
• Dependency problem when building mapping support Docker image.
• Deprecated matplotlib function in plot_output_var.py.
• Formatting in C++ logging call.
• Possible issue with randomly initialized output arrays.
• Problem with UID and GID in Docker containers on Ubuntu systems. Now
  develop user inside the containers will have the same UID and GID as the
  user on the host system allowing volume mounts to work properly.

v0.7.0 - 2023-06-14

This release brings a significant fix to the restart capability of the model.
Previously there were a number of bugs, mostly with the soil structure, that
prevented restart simulations from resuming with the state of the previous
stage. This has been fixed and now a continuous simulation matches a restart
simulation.

Added

• Fill value and units for run status file.
• Max output option for command line interface for outspec_utils.py

Fixed

• Restart capability of model so that stages can be run using the saved state
  from prior stages.
• Comments, metadata for various CMTs in parameter files.
• Confusing aspect of command line interface for outspec_utils.py.
• Bug with ParamUtilSpeedHelper class list feature.

Changed

• Various parameters for CMT 05 (Tussock Tundra).
• CMT 10 renamed to CMT 21 (EML Tussock Tundra).
• Assorted target values for CMTs 05 and 21 (renamed from CMT 10).
• Release instructions.

v0.6.2 - 2023-04-11

This is a catch-all release covering a bunch of relativley small fixes as well
as three more noteworthy updates:

• Multi-PFT implementation of the Sensitivity Analysis tool.
• Fix bug that was preventing winter respiration in soil.
• Major updates to the documentation that were done in preparation for the
  Feburary 2023 UAF / Woodwell Climate Research Center modeling workshop.

Added

• Multi-PFT capability to Sensitivity.py. Each parameter can now be set to a
  custom list of PFTs for which it will be assessed.
• Basic test file for qcal.py.
• Adds FillValue for restart files.
• input_util.py time/space summary plot: adds fill_bewtwen for +/- 1 standard
  deviation.
• Adds logging to utility functions for ingesting parameters (to ease
  debugging).
• Ability to control Sensitivity Analysis run years for the stages from command
  line.
• Adds various functionality to io_view.py Python/Bokeh tool for plotting
  inputs data sets on a map: tooltips, table printouts, shows cursor
  coordinates, functioning input boxes for user to change settings.

Fixed

• Bug with winter respiration: allow heterotrophic respiration during winter by
  lifting the limitation related to low soil moisture when tsoil < 0.
• Units for intial fibric and humic thickness parameters.
• Units for DEADC output variable.
• Parameters: formatting for CMT31 (Boreal Bog).
• Adjust cpart parameters for CMT 12 (Lowland Boreal Wet Shrubland), and 21
  (EML Tussock Tundra), so that values match the initvegc values.
• Normalize line endings in some straggling C++ files to LF (unix).
• Bug in qcal.py: issue with KeyError when reading newer calibration_targets.py
  files that have metadata structure (introduced v0.6.1).
• Whitespace in test files.
• Updated paths for compilation on Chinook.
• Updated SConstruct file for building with MPI on Chinook.
• input_util.py time/space summary plot: bug with not averaging over spatial
  dimensions.
• input_util.py time/space summary plot: bug with plotting zeros and
  autoscaling.
• input_util.py time/space summart plot: improve title, comments, command line
  interface.
• Documentation:
  • heading levels in software dev info document,
  • allow sphinx auto-doc to process docstrings in python files,
  • re-organize to put general software development info into dedicated section
    so as to streamline the dvmdostem specific documentation,
  • add examples site run with embedded plots and code,
  • re-organize to detach certain material from specific workshop years,
  • added Bokeh plotting example,
  • enabled python API documentation,
  • improve "Example Experiment 0",
  • added/updated references page,
  • updated text for calibration documentation,
  • improve local preview section for building documentation, and
  • add pip example for keeping up to date.
• Several typos used as git workflow exercises for Feb 2023 workshop.
• Issue whereby outspec_utils.py could produce misleading output when handling
  layer variables.
• Requirements file for mapping support Docker container to fix build issues
  stemming from GDAL dependancy.

Changed

• Modify batch merging command.
• Default dataset for Sensitivity.py now uses demo_data, which makes testing
  more platform independent.

Updated

• Calibration targets for CMTS 0, 1, and 2 (Black Spruce, White Spruce, and
  Deciduous Forest).
• Bump python dependency for several libaries (certifi, pillow, ipython).
• Documentation: references page.

v0.6.1 - 2022-10-11

This release primarily brings the addition of using the Python Sphinx tool for
building and working with documentation. The intent is that the Sphinx-built
documentation will be publishable on the web. The other large change is the
improvement of metadata storage and handling in the parameter files and
utilities for converting the parameter files back and forth between fixed width
text and command separated values. Additionally there were updates to a few
outputs, and the addition of a new variable (ecosystem respiration) to the
calibration tools.

Added

• Parameter utility functions for converting between Fixed Width Text (fwt)
  and Comma Separated Value (csv) parameter files.

• Notation for recording metadata in fixed width text parameter files.

• Documentation system using Sphinx with outline and some of the content
  written.

• Respiration_ecosystem (or RE or ER, depending on context) added to the
  calibration json files and calibration plotter.

• NEE (Net Ecosystem Exchange) added to the calibration json.

• Internal holding for LAYERTYPE, LAYERDEPTH, and LAYERDZ (efficiency gain
  as this reduces the overall number of file writes).

• Add unit specification for the following: LAYERDEPTH, LAYERDZ, MOSSDEATHC,
  MOSSDEATHN

Fixed

• Outputs that only exist by layer default to always be output by layer:
  PERCOLATION, ROOTWATERUPTAKE, HKLAYER, IWCLAYER, LAYERDEPTH, LAYERDZ,
  LAYERTYPE,LWCLAYER, QDRAINLAYER, TCLAYER, TLAYER, VWCLAYER

• Port numbers and specification for docker containers, specifically -dev
  container. This allows things such as jupyter notebooks to be accessed from
  outside the container.

• Missing Graphviz dependency.

Changed

• Organization of Doxygen project to better work with Sphinx.

• Standardize unit specification for unitless variables to use empty string
  the following: FRONTSTYPE

• Example notebooks and scripts to refer to the newer layout and naming of
  Docker images/containers.

v0.6.0 - 2022-08-26

The largest features added here are the Docker reorganization and the basic
testing framework using Python's doctest module. The shift in our use of
Docker will better facilitate the integration with PEcAn and should generally
streamline the developer experience. The testing, while far from complete, gives
a basic framework for writing both narrative testing docs and regression tests
for the various Python scripts in the project.

Added

• Vim as installed package in Docker development images.

• Script for running and "input modification experiment".

• New parameters for tcsolid, porosity, bulk density, hksat, and nfactor.
  Previously these were hard-coded in the C++ files. Now they are available in
  the parameters/ files so they can be modified without re-compiling the
  model.

• First steps toward testing paradigm using the Python doctest module. The
  majority of the testing added comes in the form of stand alone markdown
  files that can be read (and executed) by the Python doctest module.

• Custom averaging range for baseline climate calculation. Previously the
  baseline climate was fixed to be calculated from the first 30 years of the
  input data set (generally 1901-1930). With this change, the averaging range
  can now be set to a custom range by the user. This setting is not surfaced
  in the config file (yet) so a user would need to manually change the file
  and re-compile.

• diagnostics.py: Sort tabular output by time.

• diagnostics.py: Save capability for timeseries plots.

• calibration_targets.py: Helper function for selecting CMT
  by number.

• Tagging pattern for Docker images, based on git tags.

Fixed

• Standardization fixes for various parameter files to allow new tests in
  param_utils.py to pass.

• Algorithm for computing soil organic carbon in mineral layers.

  • Typo in Ground.cpp that set the minimum values of deep organic layer
    thickness.
  • Update algorithm for computing soil organic carbon stock to match H.
    Genet's re-analysis of pedon database publishied by Kris Johnson, 2009.

• diagnostics.py: Casting error in format string.

• calibration_targets.py: Duplicate name in calibration_targets data
  structure that was causing bugs in various utility scripts.

• Typo in Snow_Env.cpp.

Changed

• Re-organization of the Docker images for the project. Now the project has
  5 images that can be produced:

  • cpp-dev: basic development tools for C++
  • dvmdostem-dev: includes dvmdostem dependencies and Python
  • dvmdostem-build: environment for compiling dvmdostem
  • dvmdostem-run: "lean" production environment for dvmdostem
  • dvmdostem-mapping-support: a GDAL enabled environment
    The different images allow for isolating the needs of "production" images
    (which need to be smaller and don't need all the developer tooling) vs
    development images (which may be larger and need many software tools
    installed).

• Version bump dependencies: numpy, ipython, pillow.

• Boilerplate code surrounding command line entry points in various scripts to
  ease testing using Python's doctest module.

v0.5.6 - 2022-01-13

This is a trivial update that should have
been included in the last release.

Fixed

• Missing a new config key in setup script.

v0.5.5 - 2021-12-01

The two major additions with this release are the Sensitivity.py
code that lays out the framework for a sensitivity analysis
workflow, and the hold output feature.

The sensitivity workflow allows the user to include any of the
parameters in our parameter files in the analysis and has provisions
for eventually including any of our model outputs. The workflow
helps with organizing the model runs, setting up the model runs,
modifying the parameters according to different statistical schemes,
and executing the model runs. There are basic stubs provided for
the actual sensitivity analysis.

The hold output feature allows a user to have control about when
data is written out to NetCDF files, which can be a significant
time savings in some circumstances. The basic idea is to hold monthly
or daily output data internally and only write it to a file at the end
of a year for example.

Several other general updates are included and listed below.

Added

• Rough draft of a sensitivity analysis workflow:
  Implements workflow, including run management, parameter
  adjustment and basic output analysis with Python module and
  class.
• Updates to scripts for running on Chinook.
• Variables: unnormalleaf, vegC2N, vegcov, ICEsoil to RestartData.
• Experiments with Github Actions.
• Python API method to param_utils.py for listing available CMTs.
• Python ParamUtilSpeedHelper class that helps mapping parameters
  to the files they occur in.
• Various small features to param_utils.py for improving parameter
  search and modification abilities.
• Jupyter Notebooks for demonstrating Sensitivity.py
• Hold output capability for 29 commonly used output variables.

Fixed

• Release instructions.
• Bug with setting fire severity in input preparation.
• Bug with output_utils division operator.

Changed

• Dependency update for Python Pillow library.
• Fire input prep, change units to km-2 for area of burn.

v0.5.4 - 2021-07-15

v0.5.4 - 2021-07-15

Assorted updates including a few bug fixes, fixing/adding the
explicit fire input data prep, adding Docker capability, and
adding some sample Jupyter/IPython Notebooks.

Added

• Full custom config option for create_region_input.py.
• Docker files with instructions for building development images for dvmdostem and an ancillary mapping support image.
• docker-compose.yml that allows for bringing up a coordinated stack of docker containers.
• Basic scripts and workflow for doing an ensemble of runs.
• Jupyter Notebooks (IPython) showing very basic sensitivity analysis and how to work with Docker from a Jupyter Notebook.
• Logic for creating explicit fire inputs from a variety of sources: various ALFRESCO modeled scenarios and observed fire history from a shapefile downloaded from the BLM.

Fixed

• Pointer bug with soil stack, when stack is thawed.
• Overlooked python 3 upgrade bug in output_utils.py where we need explicit integer division.
• Logic in C++ code for loading explicit fire input data.

Changed

• Version bumps for pillow, pygments, jinga, pyyaml to address security vulnerabilites identified by github dependabot.

v0.5.3 - 2021-02-19

v0.5.3 - 2021-02-19

Assorted updates to plotting, docs, parameters, and various minor bugs.

Added

• Settings for spatial dimension in slurm wrappers (create_region_input.py).
• Notes for mirroring input catalog.
• Plotting snippet for gpp temperature function.
• Plots for regional outputs (heatmap, soil variables).
• Boreal calibraiton parameters (CMT01)

Fixed

• Whitespace confusion in output_spec files.
• MPI error when writing outputs.
• Restricts ROOTWATERUPTAKE to the implemented options.
• Comparison operator bug in param_util.py code.

Changed

• Upgraded create_region_input.py to Python 3.
• Update README, assorted other docs.
• Update to support API of newer Pandas versions (input_util.py)