Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update and add new support for configurations with dynamic Greenland ice sheet component #6226

Conversation

stephenprice
Copy link
Contributor

@stephenprice stephenprice commented Feb 8, 2024

This PR updates and/or adds new support that allows for simulations, through
existing IG- and BG- compsets, with a dynamic MALI component for the
Greenland ice sheet. These configurations are still under development and are
not yet scientifically supported.

Changes include:

  • added support for Greenland 20km res., trigrid configuration
  • updated Greenland 1-to-10km res. grid and input files
  • add support for configs. using new 1-to-10km grid and init. cond.
  • updated defaults for MALI history (output) files
  • updated default Albany solver parameter settings
  • added SMS, ERS, and ERP tests for 20km res. IG configuration
  • updated machine file support for Albany & Trilinos on pm-cpu & Chrysalis
  • add default layouts for 20km & 1-to-10km configs. on pm-cpu & Chrysalis
  • allow MALI global and regional analysis AMs to operate during e3sm runs

Add configuration and mapping file support for MALI, 20 km res
Greenland ice sheet, tri-grid configuration.
Add explicit path to Albany lib to support perlmutter builds.
Add and correct support for GIS 20km and 1-to-10km configs with r05,
ne30pg2, and EC30to60E2r2 grids.
Add CIME support for new GIS 1-to-10 (r02). Confirmed that BG compset
calling for related grids builds successfully.
Updated root paths for kokkos, albany, and trilinos in pm-cpu machine
file. Removed no longer needed info from gnu_pum-cpu cmake file. Added
support for e3sm + mali (Greenland) configurations for IG and BG cases,
including: updated default output freq for mali hist files; add default
PE layout for Greenland 20km IG case; add support for Greenland 20km IG
test case to various testing infrastructure files.
Explicitly specify which glcmask to use for Greenland 20km test case,
which allows elm namelist changes for test to be removed.
…e sheet.

Archive settings switched back to mpas component defaults;
change days run for ERP test.
Fix default layout to work on pm-cpu, alvarez, and chrysalis.
Increase default value for h2osno_max in many-layer snowpack model.

Add missing code from MALI driver so that global and regional analysis
members are called in simulations with dynamic ice sheets.
jonbob added a commit that referenced this pull request Apr 25, 2024
…6226)

Update and add new support for configurations with dynamic Greenland ice
sheet component

This PR updates and/or adds new support that allows for simulations,
through existing IG- and BG- compsets, with a dynamic MALI component for
the Greenland ice sheet. These configurations are still under
development and are not yet scientifically supported.

Changes include:
* added support for Greenland 20km res., trigrid configuration
* updated Greenland 1-to-10km res. grid and input files
* add support for configs. using new 1-to-10km grid and init. cond.
* updated defaults for MALI history (output) files
* updated default Albany solver parameter settings
* added SMS, ERS, and ERP tests for 20km res. IG configuration
* updated machine file support for Albany & Trilinos on pm-cpu &
  Chrysalis
* add default layouts for 20km & 1-to-10km configs on pm-cpu & Chrysalis
* allow MALI global and regional analysis AMs to operate during e3sm
  runs

[non-BFB] only for tests with active MALI
@jonbob
Copy link
Contributor

jonbob commented Apr 25, 2024

Passes:

  • ERS.f09_g16_g.MALISIA.chrysalis_intel
  • SMS.T62_oQU120_ais20.MPAS_LISIO_TEST.chrysalis_intel
  • SMS_D_Ld1.ne30pg2_r05_IcoswISC30E3r5.CRYO1850-DISMF.chrysalis_intel
  • ERS.ne4pg2_oQU480.WCYCL1850NS.chrysalis_intel
  • new e3sm_landice_developer suite on chrysalis with gnu compiler

verified all new files are in the correct format and available on the inputdata repo

merged to next

@rljacob
Copy link
Member

rljacob commented Apr 26, 2024

Re: changes to config_pesall.xml. looks like you explicitly added SGLC to all the fully coupled cases except for the one where you actually run MALI. I think thats a good way to make that distinction.

@stephenprice
Copy link
Contributor Author

Re: changes to config_pesall.xml. looks like you explicitly added SGLC to all the fully coupled cases except for the one where you actually run MALI. I think thats a good way to make that distinction.

Thanks. In fact, we could not figure out how to get the GLC-specific configs. (which are similar to some other configs in many ways) to be recognized without making this change. This was Jon W's suggested solution.

@jonbob
Copy link
Contributor

jonbob commented Apr 26, 2024

requested running the new e3sm_landice_developer test suite on chrysalis and pm-cpu using the gnu compiler, SES-3111

@ndkeen
Copy link
Contributor

ndkeen commented Apr 26, 2024

This PR causes a build error with intel pm-cpu. I think we should revert and fix. Also, maybe ping the POC before making changes to machine files.

@@ -263,8 +263,9 @@
<env name="PERL5LIB">/global/cfs/cdirs/e3sm/perl/lib/perl5-only-switch</env>
<env name="FI_CXI_RX_MATCH_MODE">software</env>
<env name="MPICH_COLL_SYNC">MPI_Bcast</env>
<env name="Albany_ROOT">$SHELL{if [ -z "$Albany_ROOT" ]; then echo /global/common/software/e3sm/mali_tpls/albany-e3sm-serial-release-gcc; else echo "$Albany_ROOT"; fi}</env>
<env name="Albany_ROOT">$SHELL{if [ -z "$Albany_ROOT" ]; then echo /global/common/software/e3sm/mali_tpls/albany-e3sm-serial-release-gcc-cmake-fix; else echo "$Albany_ROOT"; fi}</env>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we put these changes in the gnu environment_variables compiler like done in Chrysalis?

<environment_variables compiler="gnu">
   <env name="Albany_ROOT">$SHELL{if [ -z "$Albany_ROOT" ]; then echo /global/common/software/e3sm/mali_tpls/albany-e3sm-serial-release-gcc-cmake-fix; else echo "$Albany_ROOT"; fi}</env>
    <env name="Trilinos_ROOT">$SHELL{if [ -z "$Trilinos_ROOT" ]; then echo /global/common/software/e3sm/mali_tpls/trilinos-e3sm-serial-release-gcc; else echo "$Trilinos_ROOT"; fi}</env>
    <env name="Kokkos_ROOT">$SHELL{if [ -z "$Kokkos_ROOT" ]; then echo /global/common/software/e3sm/mali_tpls/trilinos-e3sm-serial-release-gcc; else echo "$Kokkos_ROOT"; fi}</env>
</environment_variables>

@mperego
Copy link
Contributor

mperego commented Apr 26, 2024

This PR causes a build error with intel pm-cpu. I think we should revert and fix. Also, maybe ping the POC before making changes to machine files.

Sorry for the inconvenience. I guess we should have put the pm-cpu changes to the machine file under the gnu environment, like we did with Chrysalis.

@jonbob
Copy link
Contributor

jonbob commented Apr 29, 2024

@mperego -- can you make that fix and either add it to this PR or make a separate PR? We don't want to mess up pm-cpu testing any longer than necessary

@mperego
Copy link
Contributor

mperego commented Apr 29, 2024

@jonbob yes, I'll add the fix to this PR.

@stephenprice
Copy link
Contributor Author

@ndkeen -- we're working to test both Mauro's recent fix for this and will also re-test one of your failing intel tests as well. Will get back to you asap.

@stephenprice
Copy link
Contributor Author

@ndkeen -- with Mauro's fix above, ERP_Ld3.ne4pg2_oQU480.F2010.pm-cpu_intel passes.

@ndkeen
Copy link
Contributor

ndkeen commented Apr 30, 2024

@stephenprice next still fails for me with SMS_D_Ld1.ne30pg2_r05_IcoswISC30E3r5.CRYO1850.pm-cpu_intel

@jonbob
Copy link
Contributor

jonbob commented Apr 30, 2024

@ndkeen -- I'm just re-merging to next right now. I'll let you know once it's there

jonbob added a commit that referenced this pull request Apr 30, 2024
@jonbob
Copy link
Contributor

jonbob commented Apr 30, 2024

Re-merged to next. @ndkeen -- we'd very much appreciate any testing you can do. With apologies

@ndkeen
Copy link
Contributor

ndkeen commented Apr 30, 2024

I verified these two tests now build/run with next.
I have not looked more, but we can see what happens with testing tonight.

ERP_Ld3.ne4pg2_oQU480.F2010.pm-cpu_intel
SMS_D_Ld1.ne30pg2_r05_IcoswISC30E3r5.CRYO1850.pm-cpu_intel

@jonbob jonbob merged commit 7de41c9 into E3SM-Project:master May 1, 2024
2 checks passed
@jonbob
Copy link
Contributor

jonbob commented May 1, 2024

merged to master and new tests blessed

@stephenprice stephenprice deleted the stephenprice/glc/add-gis-support-020724 branch May 9, 2024 19:10
jonbob added a commit that referenced this pull request Jun 20, 2024
Move extrasnowlayers test to e3sm_landice_developer suite

Since PR #6226 redefined the IGELM compset, the
SMS.ne30_oECv3_gis.IGELM_MLI.elm-extrasnowlayers test has been failing.
This moves that test from the e3sm_ocnice_extra_coverage suite to the
e3sm_landice_developer suite, which is only run on platform/compiler
combinations that support active MALI.

[BFB] for all current E3SM tests
jonbob added a commit that referenced this pull request Jun 21, 2024
Move extrasnowlayers test to e3sm_landice_developer suite

Since PR #6226 redefined the IGELM compset, the
SMS.ne30_oECv3_gis.IGELM_MLI.elm-extrasnowlayers test has been failing.
This moves that test from the e3sm_ocnice_extra_coverage suite to the
e3sm_landice_developer suite, which is only run on platform/compiler
combinations that support active MALI.

[BFB] for all current E3SM tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Chrysalis compset enhancement Land mpas-albany-landice pm-cpu Perlmutter at NERSC (CPU-only nodes) Testing Anything related to unit/system tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants