v0.1.0 GMD release

This is the initial release of the full Pace ecosystem at time of submission to Geoscientific Model Development (GMD).

This is alpha research software not meant for production use.

What's Changed

• Add submodules for existing repositories by @mcgibbon in #1
• does this do a thing? by @twicki in #3
• Docker image for top level directory by @elynnwu in #4
• Add physics savepoint test by @elynnwu in #5
• Add plotting tools by @elynnwu in #6
• Update sumodules to latest by @elynnwu in #7
• Add dependencies graph for project to README by @mcgibbon in #8
• Add script to check if package or its dependencies have changed for tests by @mcgibbon in #9
• Feature/plotting tools by @FlorianDeconinck in #13
• Merge submodules as subtrees by @mcgibbon in #15
• change cache plan location by @twicki in #18
• Allow objects with storage attribute as call arguments to FrozenStencil by @mcgibbon in #19
• Update cache location by @elynnwu in #20
• Add jenkins at top level by @elynnwu in #24
• fix grid tests for 54 ranks by @rheacangeo in #17
• Jenkins physics tests by @elynnwu in #21
• override failing tests by @rheacangeo in #26
• fv3core uses the MetricTerms by @rheacangeo in #16
• Merge buildenv and daint_venv as files in pace by @mcgibbon in #30
• Re-enable CircleCI for linting and for fv3gfs-util by @mcgibbon in #22
• Jenkins env name clarification by @elynnwu in #28
• Make jenkins green by @rheacangeo in #32
• Enable tests for fv3core infrastructure code on CircleCI by @mcgibbon in #23
• baroclinic initialization by @rheacangeo in #31
• Clean up old files in physics by @elynnwu in #34
• Add file caching infrastructure and test for fv3gfs-util using it by @mcgibbon in #27
• Point daint_venv GT4PY_VERSION.txt reference to pace git repo by @elynnwu in #37
• Feature/add shared by @elynnwu in #25
• Restore buildenv as a submodule instead of hard files by @mcgibbon in #38
• Make jenkins green... again by @rheacangeo in #35
• Enable flake8 tests on fv3gfs-util by @mcgibbon in #42
• run_on_daint fixed for more layouts by @rheacangeo in #41
• Rename fv3gfs-util to pace-util by @mcgibbon in #43
• Add required backend arg to make_storage methods by @mcgibbon in #40
• Physics use GridData by @elynnwu in #46
• fix the baroclinic test case option by @rheacangeo in #45
• Enable linting in physics by @elynnwu in #48
• Fix/enable physics linting by @elynnwu in #49
• Re-enable KCacheDetection for FVSubgridZ.init stencil by @eddie-c-davis in #52
• Add unit tests of DynamicalCore.call by @mcgibbon in #47
• Test all packages when non-subproject files change by @mcgibbon in #55
• stop re-downloading of data on gsutil fail by @rheacangeo in #58
• update_dwind_phys 54 ranks fix by @rheacangeo in #57
• Feature/move gt4py utils to pace dsl by @elynnwu in #50
• physics and dycore share constants by @rheacangeo in #51
• Gt4py is used as a submodule by @mcgibbon in #56
• Revert "Gt4py is used as a submodule" by @mcgibbon in #59
• Remove env.sh and references to it, use buildenv submodule instead by @mcgibbon in #53
• fix a typo in the overrides file by @rheacangeo in #63
• parallel and sequential tests cache compiled stencil to the same folder by @rheacangeo in #65
• Add print statements when fetching gt_cache by @elynnwu in #62
• init baroclinic gpu by @rheacangeo in #67
• Consolidate make_storage_from_shape by @elynnwu in #68
• update buildenv directory in run_on_daint.sh by @mcgibbon in #69
• Allow relative path in install virtualenv by @elynnwu in #70
• ensure that artifacts are stored if in the main branch by @twicki in #71
• create template again by @twicki in #72
• Consolidate daint logic into buildenv (env vars) and daint_venv (python packages) by @mcgibbon in #54
• remove G2G arg from run_command, consolidate gpu logic in buildenv by @mcgibbon in #76
• Remove global access in fv3core and physics tests by @elynnwu in #60
• Add timesteps arg to run_standalone.sh by @mcgibbon in #77
• Bump buildenv from fd940f2 to 88465c1 by @dependabot in #78
• Remove references to s1053, use installdir instead by @mcgibbon in #80
• add pace driver, update dycore and physics state by @rheacangeo in #79
• Bump buildenv from fc27296 to e72a0c7 by @dependabot in #85
• driver final review adjustments by @rheacangeo in #86
• Update PR checklist and readme by @elynnwu in #87
• pace-util release v0.7.0 by @mcgibbon in #88
• fix pace-util make release by adding dependent targets by @mcgibbon in #89
• Bump external/fv3gfs-wrapper from 9d62bbf to 5e9dc78 by @dependabot in #74
• Bump external/fv3gfs-fortran from 9d76b0c to 5db2c24 by @dependabot in #73
• fix script argument bug in buildenv by @mcgibbon in #90
• stabilize fv3core performance by @rheacangeo in #91
• DycoreState uses Quantity attributes by @mcgibbon in #83
• Cleanup pace stencils testing by @elynnwu in #84
• refactor DycoreState initialization routines to use only GridSizer when possible by @mcgibbon in #92
• Pace util mostly does not depend on dsl by @elynnwu in #93
• Gt4py is used as a submodule by @mcgibbon in #64
• Remove unused run_cached_fv_dynamics.sh by @mcgibbon in #95
• driver can run from serialized init data again and is tested by @rheacangeo in #96
• Renaming DummyComm to LocalComm, Adding Tags by @oelbert in #97
• Add Sphinx documentation by @mcgibbon in #94
• Namelist class in pace.util by @elynnwu in #98
• Update top level documentation by @elynnwu in #100
• Docker runs gtc cpu backend by @elynnwu in #101
• Driver demo for AMS by @mcgibbon in #99
• remove fv3gfs-wrapper, fv3gfs-fortran, and wrapper docker infrastructure by @mcgibbon in #102
• Remove unused with current namelist sum_te in remapping by @FlorianDeconinck in #104
• Add edge_interior_ratio argument to TilePartitioner by @yniederm in #44
• Feature/driver init from serialbox by @elynnwu in #103
• Passing down q_min through in tracer remapping by @FlorianDeconinck in #105
• Add Yannick to contributors by @mcgibbon in #108
• Dockerfile for plotting maps in pace by @elynnwu in #106
• Report stencil argument intents, update fv3core docstrings by @mcgibbon in #107
• fix daint compilation by @rheacangeo in #114
• Adding In/Out to Remapping by @oelbert in #110
• Update requirements to Python 3.8 by @elynnwu in #113
• Remove pytest dependency from non-testing files by @jdahm in #111
• Feature/physics cache plan by @elynnwu in #82
• convert u_contra_dxc and v_contra_dyc to temporaries, remove ptc input form D_SW by @mcgibbon in #115
• testing new buildenv by @rheacangeo in #121
• switch to https submodule by @elynnwu in #127
• Daint compilation by @rheacangeo in #128
• Driver outputs performance timing by @elynnwu in #109
• Clarify tmp_wk and tmp_vort usages in d_sw.py by @mcgibbon in #118
• fix mirror_grid for even layouts by @rheacangeo in #132
• Driver diagnostics write out lat/lon info by @elynnwu in #125
• merge main_uc and main_vc stencils in fxadv by @mcgibbon in #129
• single node compilation by @rheacangeo in #133
• Fix translate init tests by @elynnwu in #137
• Clean up concrete externals for fv3core, even on failure by @mcgibbon in #140
• Transfer dace changes for fvtp2d/d_sw, remove CopiedCorners by @mcgibbon in #139
• Update grid thresholds for higher resolution by @oelbert in #144
• add Driver regression test, change data organization by @rheacangeo in #138
• Pin click version when running black in pre-commit by @mcgibbon in #148
• Use https URL for buildenv by @jdahm in #150
• Add compare_to_numpy=True to StencilConfig by @mcgibbon in #152
• Add CachingCommReader and CachingCommWriter by @mcgibbon in #153
• Feature/jenkins baroclinic init plan by @elynnwu in #135
• Format source when using numpy backend by @jdahm in #149
• Update to latest gt4py by @jdahm in #143
• Remove numpy -> gtc:numpy hack by @jdahm in #160
• Fix/jenkins cache setup by @elynnwu in #161
• Add pace.driver.run config for CachingCommReader, CachingCommWriter by @mcgibbon in #157
• Delete deprecated files by @mcgibbon in #163
• Add gt4py cache settings to env.daint.sh by @mcgibbon in #164
• Split pace.driver.run module into multiple files by @mcgibbon in #165
• Add a configuration option to driver to disable step_physics call by @mcgibbon in #169
• util/v0.8.0 + zarr monitor: allow transposed diagnostics to be written by @brianhenn in #170
• merge two stencils in d_sw, clarify kinetic energy variable by @mcgibbon in #146
• disable driver diagnostics if no path given by @mcgibbon in #172
• Fix/driver translate test on Jenkins by @elynnwu in #173
• Remove SerialboxConfig initialization configuration from pace.driver by @mcgibbon in #168
• Consolidate driver registration code for init and comm configuration by @mcgibbon in #167
• Add Jupyter notebook example running driver with write-read by @mcgibbon in #174
• pace-util: zarr monitor quantity attribute handling by @brianhenn in #175
• pace-util release v0.9.0 by @brianhenn in #176
• consolidate imports of mpi4py into pace.util, remove fv3core null_comm by @mcgibbon in #179
• Fix Overrides by @rheacangeo in #178
• Refactor optional dependencies, make zarr optional in pace.driver by @mcgibbon in #180
• Add restart capability to pace.driver by @elynnwu in #185
• Fix/remove RESTART instead of make clean by @elynnwu in #187
• Fix jenkins driver test by @elynnwu in #189
• Move to data version 8.0.5 by @rheacangeo in #155
• Add NullCheckpointer called for C_SW and D_SW by @mcgibbon in #190
• Update gt4py by @jdahm in #183
• Use netcdf data for savepoint regressions in fv3core by @mcgibbon in #193
• Revert serialbox init by @elynnwu in #195
• hotfix grid for caching by @twicki in #197
• Add comment about fortran restart format by @mcgibbon in #202
• Remove temporary backend string replace calls by @jdahm in #207
• Fix save_netcdf for sequential savepoints by @mcgibbon in #201
• Refactor dycore translate tests to use Namelist class by @mcgibbon in #205
• Add SnapshotCheckpointer by @mcgibbon in #200
• Zero heat_source temporary between dycore calls by @mcgibbon in #196
• Fix/restart gpu backend by @elynnwu in #212
• Use netCDF data loading in physics tests by @mcgibbon in #209
• add diagnostics frequency by @twicki in #194
• Add plotting script by @elynnwu in #199
• DaCe orchestration by @FlorianDeconinck in #124
• Write intermediate restart files when specified by @elynnwu in #215
• Set gt_cache directory at module import time and remove future stencil by @jdahm in #188
• Update gt4py by @jdahm in #204
• Updated readme for pace by @ajdas1 in #221
• Add support for halo updates in TileCommunicator, use in dycore by @mcgibbon in #216
• Revert "Add support for halo updates in TileCommunicator, use in dycore" by @mcgibbon in #226
• Documentation/pace doc update by @ajdas1 in #230
• Collect build and execution timings by @jdahm in #220
• Fix cache building by @jdahm in #229
• Match timings by @jdahm in #224
• Restore dynamical core statefulness test by @mcgibbon in #234
• Add test of physics statefulness to CircleCI by @mcgibbon in #235
• Automatically calculate translate thresholds using perturbations by @mcgibbon in #222
• Add support for halo updates in TileCommunicator, use in dycore by @mcgibbon in #227
• Fix caching plan by @elynnwu in #239
• Add top-level test directory for "main" tests by @mcgibbon in #236
• Fixed an unallocated temporary causing NaNs in qvapor by @oelbert in #242
• Diagnostics include column integrated tracer by @elynnwu in #219
• fix race condition by @twicki in #244
• Ajda pointed out a deprecated link by @twicki in #247
• Reorchestrate all modules by @FlorianDeconinck in #228
• Bad timestep calculation by @FlorianDeconinck in #249
• Efficient building of caches by @twicki in #248
• Use much simpler dockerfile for make test targets by @mcgibbon in #241
• Add "pair_debug" configuration option for driver by @mcgibbon in #252
• Run make savepoint_tests_mpi on CircleCI by @mcgibbon in #218
• Driver: increment time and delete unused function by @elynnwu in #258
• fix fv3core daint_venv path in performance plan by @mcgibbon in #259
• Ingest code for GT4py by @twicki in #254
• Fix/potential race condition by @elynnwu in #260
• Fix normalize vector by @ajdas1 in #263
• Enable fetching basic test data from public FTP server. by @ofuhrer in #264
• Revise top-level README.md documentation by @ofuhrer in #265
• Add LICENSE.md by @ofuhrer in #267
• Feature/refactor grid data by @ajdas1 in #271
• Notebook stencil by @ajdas1 in #246
• Ensure kslice_from_inputs makes a slice of ints by @jdahm in #273
• Run init tests on savepoint_tests_mpi by @jdahm in #275
• Update README.md by @ofuhrer in #277
• Fix fetching test data from FTP server by @ofuhrer in #282
• clean up checking by @twicki in #284
• Fix the model inconsistencies between 6 and 54 rank runs by @twicki in #283
• do not build physics if disabled by @twicki in #290
• Clean up test decomposition by @elynnwu in #292
• Add ValidationCheckpointer and ThresholdCalibrationCheckpointer by @mcgibbon in #287
• Stable orchestration by @FlorianDeconinck in #250
• Update gt4py on August 19, 2022 by @jdahm in #285
• CI visualize validation against fortran by @elynnwu in #280
• Speed up initialization by @twicki in #294
• Update gt4py by @jdahm in #296
• Replace dace_ with dace: in backend name by @jdahm in #298
• Consolidate makefile logic for data download into top level by @mcgibbon in #293
• DaCe VRAM pooling by @FlorianDeconinck in #295
• read version from top level get data Makefile by @elynnwu in #305
• Rename packages and package directories to be consistent by @mcgibbon in #304
• Feature/example notebooks in circle ci by @ajdas1 in #303
• Add 54rank savepoint tests to CircleCI by @mcgibbon in #255
• Make DycoreState a statically-defined type by @mcgibbon in #308
• Fix caching of gt4py caches in 54rank test by @mcgibbon in #310
• Extend CircleCI to include sequential tests and cpu backend by @mcgibbon in #311
• Fix run_standalone execution by @mcgibbon in #312
• Update the fv3core readme to include bare-metal installation instructions by @oelbert in #266
• Introduce kernel_theoritical_timing from a SDFG by @FlorianDeconinck in #306
• Microphysics savepoint test passes on standard case by @elynnwu in #313
• Feature/transform grid stretch by @ajdas1 in #316
• Add test that init is decomposition invariant by @mcgibbon in #317
• Update gt4py on 2022.09.08 by @jdahm in #315
• Hack in -Wno-set-but-unused-variable warning for fv3core by @jdahm in #321
• Consolidate reqs into top level requirements_dev.txt by @mcgibbon in #307
• DaCe Orchestration: fix parsing, de-inline grid data scalars & memory leak by @FlorianDeconinck in #319
• Add version strings to packages by @mcgibbon in #325

New Contributors

• @mcgibbon made their first contribution in #1
• @rheacangeo made their first contribution in #17
• @eddie-c-davis made their first contribution in #52
• @dependabot made their first contribution in #78
• @yniederm made their first contribution in #44
• @jdahm made their first contribution in #111
• @brianhenn made their first contribution in #170
• @ajdas1 made their first contribution in #221
• @ofuhrer made their first contribution in #264

Full Changelog: https://github.com/ai2cm/pace/commits/v0.1.0