Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade e3sm-imgs to 0.0.9 and add singularity2 machine #6519

Merged
merged 1 commit into from
Jul 26, 2024
Merged

upgrade e3sm-imgs to 0.0.9 and add singularity2 machine #6519

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .github/workflows/e3sm-gh-ci-cime-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,16 @@ jobs:
fail-fast: false
matrix:
test:
- SMS_D_P4.ne4pg2_oQU480.F2010.singularity_gnu
- SMS_P4.ne4pg2_oQU480.F2010.singularity_gnu
- REP_P4.ne4pg2_oQU480.F2010.singularity_gnu
- ERS_P4.ne4pg2_oQU480.F2010.singularity_gnu
- ERS_P4.ne4pg2_oQU480.F2010.singularity_gnu.eam-wcprod_F2010
- ERP_P4.ne4pg2_oQU480.F2010.singularity_gnu
- PET_P4.ne4pg2_oQU480.F2010.singularity_gnu
- PEM_P4.ne4pg2_oQU480.F2010.singularity_gnu
- SMS_D_P4.ne4pg2_oQU480.F2010.singularity2_gnu
- SMS_P4.ne4pg2_oQU480.F2010.singularity2_gnu
- REP_P4.ne4pg2_oQU480.F2010.singularity2_gnu
- ERS_P4.ne4pg2_oQU480.F2010.singularity2_gnu
- ERS_P4.ne4pg2_oQU480.F2010.singularity2_gnu.eam-wcprod_F2010
- ERP_P4.ne4pg2_oQU480.F2010.singularity2_gnu
- PET_P4.ne4pg2_oQU480.F2010.singularity2_gnu
- PEM_P4.ne4pg2_oQU480.F2010.singularity2_gnu
container:
image: ghcr.io/mahf708/e3sm-imgs:v0.0.6
image: ghcr.io/mahf708/e3sm-imgs:v0.0.9
Copy link
Member

@rljacob rljacob Jul 23, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move these images to an E3SM account?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

I was planning of moving the entire "e3sm-imgs" repo (or we can rename it) to E3SM-Project once we cross the 0.1.0 version threshold. There's only substantial one item on my todo list before reaching the 0.1.0: making an intel container (cpu). There's also one logistical point I would like to address (saving less of the intermediate "test"-tagged containers, which I could do in one PR soon).

Do you prefer I address these two items before integrating this PR or in a subsequent PR?

Alternatively, I can also upload these images directly to the E3SM Docker account from my repo, though I think we should have the repo be under E3SM-Project sooner or later.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can wait.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other thing you will notice is that I opted to introduce "singularity2" temporarily because I didn't want to mess up the circleci workflow (I will reunite the two singularities soon, but I am trying to focus on getting the intel container as well as some basic runtime/production container going...)


steps:
-
Expand Down
18 changes: 9 additions & 9 deletions .github/workflows/e3sm-gh-ci-w-cime-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,16 @@ jobs:
fail-fast: false
matrix:
test:
- SMS_D_P8.ne4pg2_oQU480.WCYCL2010NS.singularity_gnu
- SMS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity_gnu
- REP_P8.ne4pg2_oQU480.WCYCL2010NS.singularity_gnu
- ERS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity_gnu
- ERS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity_gnu.allactive-wcprod_1850
- ERP_P8.ne4pg2_oQU480.WCYCL2010NS.singularity_gnu
- PET_P8.ne4pg2_oQU480.WCYCL2010NS.singularity_gnu
- PEM_P8.ne4pg2_oQU480.WCYCL2010NS.singularity_gnu
- SMS_D_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
- SMS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
- REP_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
- ERS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
- ERS_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu.allactive-wcprod_1850
- ERP_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
- PET_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
- PEM_P8.ne4pg2_oQU480.WCYCL2010NS.singularity2_gnu
container:
image: ghcr.io/mahf708/e3sm-imgs:v0.0.6
image: ghcr.io/mahf708/e3sm-imgs:v0.0.9

steps:
-
Expand Down
45 changes: 45 additions & 0 deletions cime_config/machines/config_machines.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1714,6 +1714,51 @@
</environment_variables>
</machine>

<machine MACH="singularity2">
<DESC>Singularity container 2.0</DESC>
<NODENAME_REGEX>singularity2</NODENAME_REGEX>
<OS>LINUX</OS>
<COMPILERS>gnu</COMPILERS>
<MPILIBS>mpich</MPILIBS>
<CIME_OUTPUT_ROOT>$ENV{HOME}/projects/e3sm/scratch</CIME_OUTPUT_ROOT>
<DIN_LOC_ROOT>$ENV{HOME}/projects/e3sm/cesm-inputdata</DIN_LOC_ROOT>
<DIN_LOC_ROOT_CLMFORC>$ENV{HOME}/projects/e3sm/ptclm-data</DIN_LOC_ROOT_CLMFORC>
<DOUT_S_ROOT>$ENV{HOME}/projects/e3sm/scratch/archive/$CASE</DOUT_S_ROOT>
<BASELINE_ROOT>$ENV{HOME}/projects/e3sm/baselines/$COMPILER</BASELINE_ROOT>
<CCSM_CPRNC>/usr/local/packages/bin/cprnc</CCSM_CPRNC>
<GMAKE>make</GMAKE>
<GMAKE_J>4</GMAKE_J>
<TESTS>e3sm_developer</TESTS>
<BATCH_SYSTEM>none</BATCH_SYSTEM>
<SUPPORTED_BY>e3sm-team</SUPPORTED_BY>
<MAX_TASKS_PER_NODE>16</MAX_TASKS_PER_NODE>
<MAX_MPITASKS_PER_NODE>16</MAX_MPITASKS_PER_NODE>
<mpirun mpilib="default">
<executable>mpirun</executable>
<arguments>
<arg name="num_tasks"> -launcher fork -hosts localhost -np {{ total_tasks }}</arg>
</arguments>
</mpirun>
<module_system type="none"/>
<RUNDIR>$ENV{HOME}/projects/e3sm/scratch/$CASE/run</RUNDIR>
<EXEROOT>$ENV{HOME}/projects/e3sm/scratch/$CASE/bld</EXEROOT>
<environment_variables>
<env name="E3SM_SRCROOT">$SRCROOT</env>
</environment_variables>
<environment_variables mpilib="mpi-serial">
<env name="NETCDF_PATH">/usr/local/packages</env>
<env name="PATH">/usr/local/packages/bin:$ENV{PATH}</env>
<env name="LD_LIBRARY_PATH">/usr/local/packages/lib</env>
</environment_variables>
<environment_variables mpilib="!mpi-serial">
<env name="NETCDF_PATH">/usr/local/packages</env>
<env name="PNETCDF_PATH">/usr/local/packages</env>
<env name="HDF5_ROOT">/usr/local/packages</env>
<env name="PATH">/usr/local/packages/bin:$ENV{PATH}</env>
<env name="LD_LIBRARY_PATH">/usr/local/packages/lib</env>
</environment_variables>
</machine>

<machine MACH="melvin">
<DESC>Linux workstation for Jenkins testing</DESC>
<NODENAME_REGEX>(melvin|watson|s999964|climate|penn|sems)</NODENAME_REGEX>
Expand Down