Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Interface and Validation Testing for MAM4xx Online Emissions #3083

Merged
merged 41 commits into from
Nov 19, 2024

Conversation

mjs271
Copy link
Contributor

@mjs271 mjs271 commented Oct 31, 2024

An existing EAMxx interface (eamxx_mam_srf_and_online_emissions_process_interface.cpp) is modified to include the invocation of the "online" emissions of dust and sea salt. The new additions are for:

  1. Reading soil erodibility file for dust emissions
  2. Reading the marine organic input file for marine organic aerosols
  3. Read sst and dust fluxes from the land and ocean model via surface coupler
  4. Invoke dust and sea salt emissions codes from MAM4xx

Since the CMake option, SCREAM_ENABLE_MAM, is always ON, these codes are always compiled with the EAMxx codebase, but this interface can be turned off/on using the namelist flags.

Output variables

The output variable names are "constituent_fluxes" and interstitial aerosols whose emissions are computed:
nacl_a1, mom_a1, num_a1, nacl_a2, mom_a2, num_a2, dust_a3, nacl_a3, mom_a3,num_a3, mom_a4, num_a4

Note that the "constituent_fluxes" indices are exactly the same as EAM's state%q.

Testing

We have existing tests for this interface, so we didn't add any new tests. We have a suite of validation tests in the MAM4xx repo to validate the online emission processes.

The srf_online_emiss process that invokes online emissions codes has to be invoked with the constituent_fluxes process to modify interstitial aerosols. Otherwise, only the constituent_fluxes array will be modified. These processes are added as an EAMxx process with the ability to turn it on/off using the labels mam4_srf_online_emiss and mam4xx-constituent_fluxes in the namelist_scream.xml file. The atmchange command to invoke these processes in a CIME simulation is:

./atmchange physics::atm_procs_list="mam4_constituent_fluxes,mac_aero_mic,rrtmgp,mam4_srf_online_emiss"

To run a CIME simulation, we have created a test modifier mam4xx-srf_online_emiss_constituent_fluxes:

cd cime/scripts
./create_test SMS_D_P32x1.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.<machine>_<compiler>.scream-mam4xx-srf_online_emiss_constituent_fluxes -t <test id> -p <project id>

Where,
<machine> is the machine name
<compiler> is the compiler to use on the machine
<test id> is a unique string test identifier
<project id> is the allocation project to charge

New input data

We have added 6 files for each of the three grid resolutions:
ne2np4, ne4pg2, ne30pg2

for soil erodibility (3 files) and marine organic aerosols (3 files)

For finer resolutions, the code will interpolate the ne30pg2 grid. All the files are on the E3SM input data server.

Figures from model analysis

Aerosol burdens from surface emissions only simulation at ne30pg2 resolution. We have added prescribed emissions plots to show that they remain unchanged (plots generated by Meng):
image

NOTE: There is an initial unphysical spike in dust burden when we use ne4 resolution. @kaizhangpnl found that that the spike may be due to the huge jump in mean u10 velocity between 1st and 2nd time step (see his comments below). Here are plots from ne4 simulation (generated by Chandru)
image

Standalone Test Timings (Compy):

We will upload the timings once the PR is reviewed and approved

Copy link

github-actions bot commented Oct 31, 2024

PR Preview Action v1.4.8
🚀 Deployed preview to https://E3SM-Project.github.io/scream/pr-preview/pr-3083/
on branch gh-pages at 2024-11-16 00:10 UTC

@mjs271 mjs271 marked this pull request as draft October 31, 2024 16:04
@singhbalwinder
Copy link
Contributor

singhbalwinder commented Oct 31, 2024

Thanks, @mjs271 !

Reviewers: I will let you know when it is ready for review. We need to run some more tests, which might bring in more changes.

@singhbalwinder
Copy link
Contributor

Tagging @susburrows.

config_.amicphys.nucleation.pbl_nuc_wang2008_user_choice = 1;
config_.amicphys.nucleation.adjust_factor_pbl_ratenucl = 1.0;
config_.amicphys.nucleation.accom_coef_h2so4 = 1.0;
//config_.amicphys.nucleation.dens_so4a_host = 1770.0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These mods are due to the mismatch in the mam4xx submodule. Once microphysics PR is merged, we will rebase to remove these mods.

aircon, num_gas_ids, qgas_cur, qgas_avg, qgas_netprod_otrproc,
qaer_cur, qnum_cur, dgn_awet, alnsg_aer, uptk_rate_factor, uptkaer,
uptkrate_h2so4, niter_out, g0_soa_out);
// gasaerexch::mam_gasaerexch_1subarea(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These mods are due to the mismatch in the mam4xx submodule. Once microphysics PR is merged, we will rebase to remove these mods.

@singhbalwinder singhbalwinder added EAGLES Work related to the EAGLES aerosol project AT: RETEST MAM and removed AT: WIP labels Oct 31, 2024
@singhbalwinder
Copy link
Contributor

Removed WIP to get it tested once. I will WIP it again after the testing.

@singhbalwinder singhbalwinder marked this pull request as ready for review October 31, 2024 18:58
@E3SM-Bot
Copy link
Collaborator

Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing.

@E3SM-Bot
Copy link
Collaborator

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

  • Build Num: 6268
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
PR_LABELS AT: RETEST;AT: PRE-TEST INSPECTED;EAGLES;MAM
PULLREQUESTNUM 3083
SCREAM_SOURCE_REPO https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA 74b92ec
SCREAM_TARGET_BRANCH master
SCREAM_TARGET_REPO https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA e140df9
TEST_REPO_ALIAS SCREAM

Using Repos:

Repo: SCREAM (E3SM-Project/scream)
  • Branch: mjs/eamxx/aero_emis_interface
  • SHA: 74b92ec
  • Mode: TEST_REPO

Pull Request Author: mjs271

@E3SM-Bot
Copy link
Collaborator

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

  • Build Num: 6268
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
PR_LABELS AT: RETEST;AT: PRE-TEST INSPECTED;EAGLES;MAM
PULLREQUESTNUM 3083
SCREAM_SOURCE_REPO https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA 74b92ec
SCREAM_TARGET_BRANCH master
SCREAM_TARGET_REPO https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA e140df9
TEST_REPO_ALIAS SCREAM
SCREAM_PullRequest_Autotester_Weaver # 6268 FAILED (click to see last 100 lines of console output)

--   PLEV = 72
--   QSIZE_D = 10
--   ENERGY = FALSE
-- ** Building YAKL for a CUDA backend **
-- ** YAKL is using the following compiler flags: -DYAKL_ARCH_CUDA --expt-extended-lambda --expt-relaxed-constexpr -Wno-deprecated-gpu-targets -std=c++17 -DYAKL_ARCH_CUDA -extended-lambda -Wext-lambda-captures-this -arch=sm_70 --expt-relaxed-constexpr -ccbin /projects/ppc64le-pwr9-rhel8/tpls/openmpi/4.1.6/gcc/11.3.0/base/g5mp6iz/bin/mpicxx  **
Using CUDAToolkit macros
-- Found MPI: TRUE (found version "3.1") found components: C 
-- Disabling all warnings for target rrtmgp
CMake Warning (dev) at /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/externals/haero/CMakeLists.txt:10 (option):
  Policy CMP0077 is not set: option() honors normal variables.  Run "cmake
  --help-policy CMP0077" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

For compatibility with older versions of CMake, option is clearing the
normal variable 'HAERO_ENABLE_GPU'.
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) at /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/externals/haero/CMakeLists.txt:11 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.

For compatibility with older versions of CMake, option is clearing the
normal variable 'HAERO_ENABLE_MPI'.
This warning is for project developers. Use -Wno-dev to suppress it.

-- Configuring with build type: Release
CMake Warning (dev) at /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/externals/haero/CMakeLists.txt:12 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.

For compatibility with older versions of CMake, option is clearing the
normal variable 'HAERO_ENABLE_TESTS'.
This warning is for project developers. Use -Wno-dev to suppress it.

-- Building for CPU
-- Building Haero within another project.
-- Generating project files in build directory: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/components/eamxx/ctest-build/release/externals/haero
-- C++ compiler is /projects/ppc64le-pwr9-rhel8/tpls/openmpi/4.1.6/gcc/11.3.0/base/g5mp6iz/bin/mpicxx (GNU 11.3.0)
-- C compiler is /projects/ppc64le-pwr9-rhel8/tpls/openmpi/4.1.6/gcc/11.3.0/base/g5mp6iz/bin/mpicc (GNU 11.3.0)
-- Configuring haero v0.1.0
-- Using double precision floating point numbers
-- Configuring on Linux.
-- Installation prefix is /usr/local
-- MPI is enabled
-- Configuring with build type: Release
-- Generating project files in build directory: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/components/eamxx/ctest-build/release/externals/mam4xx
-- C++ compiler is /projects/ppc64le-pwr9-rhel8/tpls/openmpi/4.1.6/gcc/11.3.0/base/g5mp6iz/bin/mpicxx (GNU 11.3.0)
-- C compiler is /projects/ppc64le-pwr9-rhel8/tpls/openmpi/4.1.6/gcc/11.3.0/base/g5mp6iz/bin/mpicc (GNU 11.3.0)
-- Configuring MAM4xx v0.1.0
-- Using double precision floating point numbers
-- Installation prefix is /usr/local
CMake Error at cmake/ScreamUtils.cmake:212 (message):
Could not download scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc,
output was: WARNING:root:wget failed with output: and errput File
/home/projects/e3sm/scream/data/atm/scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc
already there; not retrieving.

Checking for file
atm/scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc within
/home/projects/e3sm/scream/data

Input file /home/projects/e3sm/scream/data/atm/scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc needs to be downloaded.
Attempting to download atm/scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc from https://web.lcrc.anl.gov/public/e3sm/inputdata/ with protocol wget
FAILED

ERROR: Unsupported inputdata protocol: svn

Call Stack (most recent call first):
tests/single-process/mam/emissions/CMakeLists.txt:43 (GetInputFile)

-- Configuring incomplete, errors occurred!
See also "/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/components/eamxx/ctest-build/release/CMakeFiles/CMakeOutput.log".
See also "/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/components/eamxx/ctest-build/release/CMakeFiles/CMakeError.log".

Error(s) occurred during test phase
OVERALL STATUS: FAIL
Starting analysis on weaver with cmd: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && export HDF5_USE_FILE_LOCKING=FALSE && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -m weaver
RUN: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && export HDF5_USE_FILE_LOCKING=FALSE && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -m weaver
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6268/scream/components/eamxx
weaver failed
######################################################
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash -le

cd $WORKSPACE/${BUILD_ID}/

./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
[SCREAM_PullRequest_Autotester_Weaver] $ /bin/bash -le /tmp/jenkins7766924187033877448.sh
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Sending e-mails to: [email protected]
Finished: FAILURE

@singhbalwinder
Copy link
Contributor

Looks like the model failed to download a data file:

Input file /home/projects/e3sm/scream/data/atm/scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc needs to be downloaded.
Attempting to download atm/scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc from https://web.lcrc.anl.gov/public/e3sm/inputdata/

This file is present on the server at:https://web.lcrc.anl.gov/public/e3sm/inputdata/atm/scream/mam4xx/emissions/ne2np4/

I am not sure why the model can't download it.

@mjs271
Copy link
Contributor Author

mjs271 commented Oct 31, 2024

Looks like the model failed to download a data file:

Input file /home/projects/e3sm/scream/data/atm/scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc needs to be downloaded.
Attempting to download atm/scream/mam4xx/emissions/ne2np4/dst_ne2np4_c20241028.nc from https://web.lcrc.anl.gov/public/e3sm/inputdata/

This file is present on the server at:https://web.lcrc.anl.gov/public/e3sm/inputdata/atm/scream/mam4xx/emissions/ne2np4/

I am not sure why the model can't download it.

@singhbalwinder

Stranger yet, the file appears to be on weaver already

jschm@weaver11:/home/projects/e3sm/scream/data/atm/scream/mam4xx/emissions/ne2np4$ pwd
/home/projects/e3sm/scream/data/atm/scream/mam4xx/emissions/ne2np4
mjschm@weaver11:/home/projects/e3sm/scream/data/atm/scream/mam4xx/emissions/ne2np4$ ls
cmip6_mam4_bc_a4_surf_ne2np4_2010_clim_c20240726.nc
cmip6_mam4_num_a1_surf_ne2np4_2010_clim_c20240726.nc
cmip6_mam4_num_a2_surf_ne2np4_2010_clim_c20240726.nc
cmip6_mam4_num_a4_surf_ne2np4_2010_clim_c20240726.nc
cmip6_mam4_pom_a4_surf_ne2np4_2010_clim_c20240726.nc
cmip6_mam4_so4_a1_surf_ne2np4_2010_clim_c20240726.nc
cmip6_mam4_so4_a2_surf_ne2np4_2010_clim_c20240726.nc
DMSflux.2010.ne2np4_conserv.POPmonthlyClimFromACES4BGC_c20240726.nc
dst_ne2np4_c20241028.nc

@mahf708
Copy link
Contributor

mahf708 commented Oct 31, 2024

ERROR: Unsupported inputdata protocol: svn

There was a PR that went in recently: #3080 (luca may have something to say)

@E3SM-Bot
Copy link
Collaborator

E3SM-Bot commented Nov 1, 2024

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

  • Build Num: 6270
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
PR_LABELS AT: PRE-TEST INSPECTED;EAGLES;MAM
PULLREQUESTNUM 3083
SCREAM_SOURCE_REPO https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA ce44955
SCREAM_TARGET_BRANCH master
SCREAM_TARGET_REPO https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA e140df9
TEST_REPO_ALIAS SCREAM

Using Repos:

Repo: SCREAM (E3SM-Project/scream)
  • Branch: mjs/eamxx/aero_emis_interface
  • SHA: ce44955
  • Mode: TEST_REPO

Pull Request Author: mjs271

@E3SM-Bot
Copy link
Collaborator

E3SM-Bot commented Nov 1, 2024

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

  • Build Num: 6270
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
PR_LABELS AT: PRE-TEST INSPECTED;EAGLES;MAM
PULLREQUESTNUM 3083
SCREAM_SOURCE_REPO https://github.com/E3SM-Project/scream
SCREAM_SOURCE_SHA ce44955
SCREAM_TARGET_BRANCH master
SCREAM_TARGET_REPO https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA e140df9
TEST_REPO_ALIAS SCREAM
SCREAM_PullRequest_Autotester_Weaver # 6270 FAILED (click to see last 100 lines of console output)

/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Cuda/Kokkos_Cuda_Parallel_Team.hpp:533:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  533 |   ParallelFor(const FunctorType& arg_functor, const Policy& arg_policy)
      | ^ ~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp: In function 'void Kokkos::parallel_for(const string&, const ExecPolicy&, const FunctorType&) [with ExecPolicy = Kokkos::MDRangePolicy, Kokkos::IndexType >; FunctorType = __nv_hdl_wrapper_t >, int, int), &Homme::BoundaryExchange::register_field, 1>, void(int, int), Kokkos::View >**, Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >, int, Kokkos::View >, int>; Enable = void]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp:133:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  133 | inline void parallel_for(const std::string& str, const ExecPolicy& policy,
      | ^~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Cuda/Kokkos_Cuda_Parallel_Team.hpp: In constructor 'Kokkos::Impl::ParallelFor, Kokkos::Cuda>::ParallelFor(const FunctorType&, const Policy&) [with FunctorType = __nv_hdl_wrapper_t, void(const Kokkos::Impl::CudaTeamMember&), const Kokkos::View***, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >, const Kokkos::View**, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >, const double, Kokkos::View*, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> > >; Properties = {Kokkos::Cuda}]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Cuda/Kokkos_Cuda_Parallel_Team.hpp:533:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  533 |   ParallelFor(const FunctorType& arg_functor, const Policy& arg_policy)
      | ^ ~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp: In function 'void Kokkos::parallel_for(const string&, const ExecPolicy&, const FunctorType&) [with ExecPolicy = Kokkos::MDRangePolicy, Kokkos::IndexType >; FunctorType = __nv_hdl_wrapper_t >, int, int), &Homme::BoundaryExchange::register_field, 1>, void(int, int), Kokkos::View >**, Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >, int, Kokkos::View >, int>; Enable = void]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp:133:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  133 | inline void parallel_for(const std::string& str, const ExecPolicy& policy,
      | ^~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp: In function 'void Kokkos::parallel_for(const string&, const ExecPolicy&, const FunctorType&) [with ExecPolicy = Kokkos::MDRangePolicy, Kokkos::IndexType >; FunctorType = __nv_hdl_wrapper_t >, int, int), &Homme::BoundaryExchange::register_field, 1>, void(int, int), Kokkos::View >**, Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >, int, Kokkos::View >, int>; Enable = void]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp:133:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  133 | inline void parallel_for(const std::string& str, const ExecPolicy& policy,
      | ^~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/share/field/field_alloc_prop.hpp: In copy constructor 'scream::FieldAllocProp::FieldAllocProp(const scream::FieldAllocProp&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/share/field/field_alloc_prop.hpp:106:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  106 |   FieldAllocProp (const FieldAllocProp&) = default;
      | ^ ~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::apply_iop_forcing(scream::Real)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {const Kokkos::Impl::CudaTeamMember&}; F1 = const int; F2 = ekat::WorkspaceManager, Kokkos::Device >; F3 = Kokkos::View, Kokkos::MemoryTraits<0> >; F4 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F5 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F6 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F7 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F8 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F9 = const int; F10 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F11 = const double; F12 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F13 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F14 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F15 = const int; F16 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp:412:1060: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  412 |   Kokkos::parallel_for("compute_qdp_and_vtheta_dp", policy_iop, KOKKOS_LAMBDA (const KT::MemberType& team) {
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::apply_iop_forcing(scream::Real)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {const Kokkos::Impl::CudaTeamMember&}; F1 = const int; F2 = ekat::WorkspaceManager, Kokkos::Device >; F3 = Kokkos::View, Kokkos::MemoryTraits<0> >; F4 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F5 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F6 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F7 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F8 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F9 = const int; F10 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F11 = const double; F12 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F13 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F14 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F15 = const int; F16 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp:412:1060: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  412 |   Kokkos::parallel_for("compute_qdp_and_vtheta_dp", policy_iop, KOKKOS_LAMBDA (const KT::MemberType& team) {
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::apply_iop_forcing(scream::Real)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {const Kokkos::Impl::CudaTeamMember&}; F1 = const int; F2 = ekat::WorkspaceManager, Kokkos::Device >; F3 = Kokkos::View, Kokkos::MemoryTraits<0> >; F4 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F5 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F6 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F7 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F8 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F9 = const int; F10 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F11 = const double; F12 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F13 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F14 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F15 = const int; F16 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp:412:1060: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  412 |   Kokkos::parallel_for("compute_qdp_and_vtheta_dp", policy_iop, KOKKOS_LAMBDA (const KT::MemberType& team) {
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/share/field/field_alloc_prop.hpp: In copy constructor 'scream::FieldAllocProp::FieldAllocProp(const scream::FieldAllocProp&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/share/field/field_alloc_prop.hpp:106:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  106 |   FieldAllocProp (const FieldAllocProp&) = default;
      | ^ ~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/share/field/field_alloc_prop.hpp: In copy constructor 'scream::FieldAllocProp::FieldAllocProp(const scream::FieldAllocProp&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/share/field/field_alloc_prop.hpp:106:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  106 |   FieldAllocProp (const FieldAllocProp&) = default;
      | ^ ~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/KokkosExp_MDRangePolicy.hpp: In copy constructor 'Kokkos::MDRangePolicy, Kokkos::IndexType >::MDRangePolicy(const Kokkos::MDRangePolicy, Kokkos::IndexType >&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/KokkosExp_MDRangePolicy.hpp:153:8: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  153 | struct MDRangePolicy : public Kokkos::Impl::PolicyTraits {
      |        ^~~~~~~~~~~~~
nvcc_internal_extended_lambda_implementation: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::Field::update_impl(const scream::Field&, int, int, int)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {int}; F1 = Kokkos::View, Kokkos::MemoryTraits<0> >; F2 = Kokkos::View, Kokkos::MemoryTraits<0> >; F3 = const int; F4 = const int; F5 = const int]':
nvcc_internal_extended_lambda_implementation:463:19: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/KokkosExp_MDRangePolicy.hpp: In copy constructor 'Kokkos::MDRangePolicy, Kokkos::IndexType >::MDRangePolicy(const Kokkos::MDRangePolicy, Kokkos::IndexType >&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/externals/ekat/extern/kokkos/core/src/KokkosExp_MDRangePolicy.hpp:153:8: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  153 | struct MDRangePolicy : public Kokkos::Impl::PolicyTraits {
      |        ^~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::initialize_homme_state()::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {int}; F1 = const int; F2 = const int; F3 = const int; F4 = const Kokkos::View, 1> >* [2][10][4][4][128], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F5 = const int; F6 = const Kokkos::View, 1> >** [4][4][128], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F7 = const Kokkos::View, 1> >* [3][4][4][128], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F8 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp:1211:653: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 1211 |   Kokkos::parallel_for(Kokkos::RangePolicy<>(0,nelem*qsize*NGP*NGP*npacks_mid),
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ^
[ 68%] Linking CXX static library libscream_rrtmgp.a
[ 68%] Built target scream_rrtmgp
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::initialize_homme_state()::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {int}; F1 = const int; F2 = const int; F3 = const int; F4 = const Kokkos::View, 1> >* [2][41][4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F5 = const int; F6 = const Kokkos::View, 1> >** [4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F7 = const Kokkos::View, 1> >* [3][4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F8 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp:1211:653: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 1211 |   Kokkos::parallel_for(Kokkos::RangePolicy<>(0,nelem*qsize*NGP*NGP*npacks_mid),
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::initialize_homme_state()::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {int}; F1 = const int; F2 = const int; F3 = const int; F4 = const Kokkos::View, 1> >* [2][10][4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F5 = const int; F6 = const Kokkos::View, 1> >** [4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F7 = const Kokkos::View, 1> >* [3][4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F8 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp:1211:653: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 1211 |   Kokkos::parallel_for(Kokkos::RangePolicy<>(0,nelem*qsize*NGP*NGP*npacks_mid),
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ^
[ 68%] Linking CXX static library libscream_test_support.a
[ 68%] Built target scream_test_support
[ 68%] Linking CXX static library libscream_theta-l_kokkos_4_128_10.a
[ 68%] Built target scream_theta-l_kokkos_4_128_10
[ 68%] Linking CXX static library libscream_theta-l_kokkos_4_72_41.a
[ 68%] Built target scream_theta-l_kokkos_4_72_41
[ 68%] Linking CXX static library libscream_control.a
[ 68%] Built target scream_control
[ 68%] Linking CXX static library libscream_theta-l_kokkos_4_72_10.a
[ 68%] Built target scream_theta-l_kokkos_4_72_10
gmake: *** [Makefile:166: all] Error 2

Error(s) occurred during test phase
OVERALL STATUS: FAIL
Starting analysis on weaver with cmd: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && export HDF5_USE_FILE_LOCKING=FALSE && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -m weaver
RUN: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && export HDF5_USE_FILE_LOCKING=FALSE && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -m weaver
FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/6270/scream/components/eamxx
weaver failed
######################################################
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash -le

cd $WORKSPACE/${BUILD_ID}/

./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
[SCREAM_PullRequest_Autotester_Weaver] $ /bin/bash -le /tmp/jenkins5933295293729296883.sh
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Sending e-mails to: [email protected]
Finished: FAILURE

@singhbalwinder singhbalwinder force-pushed the mjs/eamxx/aero_emis_interface branch from d7cadd7 to c8c4d65 Compare November 16, 2024 00:08
mahf708
mahf708 previously approved these changes Nov 16, 2024
mahf708
mahf708 previously approved these changes Nov 16, 2024
@singhbalwinder
Copy link
Contributor

@mahf708 : Thanks again for the review. We have addressed all the comments except for the file reading (as we will revisit it later). I will create an issue for the other comment about using the scaling function. @mjs271 have tested on PM-GPUs and these codes are not generating any new warnings.

This should be ready to merge now.

Copy link
Contributor

@mahf708 mahf708 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@odiazib odiazib self-requested a review November 17, 2024 18:15
@bartgol bartgol added CI: automerge WARNING: Still in an experimental phase CI: skip cuda Skip cuda testing for this PR labels Nov 19, 2024
@bartgol bartgol merged commit 0af3b66 into master Nov 19, 2024
12 of 21 checks passed
@bartgol bartgol deleted the mjs/eamxx/aero_emis_interface branch November 19, 2024 00:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI: automerge WARNING: Still in an experimental phase CI: skip cuda Skip cuda testing for this PR EAGLES Work related to the EAGLES aerosol project MAM
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants