Point MAM4xx submodule to its main branch to get recent changes #2911

singhbalwinder · 2024-07-19T14:17:58Z

The recent MAM4xx submodule has all the changes for new and upcoming EAMxx-MAM4xx interfaces. @tcclevenger changed all TeamThreadRange to TeamVectorRange for all Kokkos parallel_for loops as they should improve performance and help with race conditions.

singhbalwinder · 2024-07-19T14:18:44Z

Tagging @susburrows

E3SM-Bot · 2024-07-19T15:41:58Z

Status Flag 'Pre-Test Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO INSPECTION HAS BEEN PERFORMED ON THIS PULL REQUEST! - This PR must be inspected by setting label 'AT: PRE-TEST INSPECTED'.

E3SM-Bot · 2024-07-19T16:11:56Z

Status Flag 'Pre-Test Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED by label AT: PRE-TEST INSPECTED! Autotester is Removing Label; this inspection will remain valid until a new commit to source branch is performed.

E3SM-Bot · 2024-07-19T16:15:31Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Build Num: 5668
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;EAGLES
PULLREQUESTNUM	2911
SCREAM_SOURCE_REPO	https://github.com/eagles-project/scream
SCREAM_SOURCE_SHA	`77fcad2`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`17dda18`
TEST_REPO_ALIAS	SCREAM

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Build Num: 5909
Status: STARTED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;EAGLES
PULLREQUESTNUM	2911
SCREAM_SOURCE_REPO	https://github.com/eagles-project/scream
SCREAM_SOURCE_SHA	`77fcad2`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`17dda18`
TEST_REPO_ALIAS	SCREAM

Using Repos:

Repo: SCREAM (eagles-project/scream)

Pull Request Author: singhbalwinder

E3SM-Bot · 2024-07-19T17:02:46Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: SCREAM_PullRequest_Autotester_Mappy

Build Num: 5668
Status: PASSED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;EAGLES
PULLREQUESTNUM	2911
SCREAM_SOURCE_REPO	https://github.com/eagles-project/scream
SCREAM_SOURCE_SHA	`77fcad2`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`17dda18`
TEST_REPO_ALIAS	SCREAM

Build Information

Test Name: SCREAM_PullRequest_Autotester_Weaver

Build Num: 5909
Status: FAILED

Jenkins Parameters

Parameter Name	Value
PR_LABELS	AT: AUTOMERGE;EAGLES
PULLREQUESTNUM	2911
SCREAM_SOURCE_REPO	https://github.com/eagles-project/scream
SCREAM_SOURCE_SHA	`77fcad2`
SCREAM_TARGET_BRANCH	master
SCREAM_TARGET_REPO	https://github.com/E3SM-Project/scream
SCREAM_TARGET_SHA	`17dda18`
TEST_REPO_ALIAS	SCREAM

SCREAM_PullRequest_Autotester_Mappy # 5668 PASSED (click to see last 100 lines of console output)


Starting SHAREDLIB_BUILD for test SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 with 1 procs
Finished SETUP for test ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 in 2.530240 seconds (PASS)
Starting SHAREDLIB_BUILD for test ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 with 1 procs
Finished SETUP for test ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 in 2.099335 seconds (PASS)
Finished SETUP for test ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 in 2.582264 seconds (PASS)
Finished SETUP for test PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 in 2.597650 seconds (PASS)
Starting SHAREDLIB_BUILD for test ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 with 1 procs
Starting SHAREDLIB_BUILD for test ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 with 1 procs
Starting SHAREDLIB_BUILD for test PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 with 1 procs
Finished SETUP for test ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 in 2.545563 seconds (PASS)
Starting SHAREDLIB_BUILD for test ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 with 1 procs
Finished SHAREDLIB_BUILD for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-aci in 55.808491 seconds (PASS)
Finished SHAREDLIB_BUILD for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics in 56.640106 seconds (PASS)
Starting MODEL_BUILD for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-aci with 16 procs
Starting MODEL_BUILD for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics with 16 procs
Finished SHAREDLIB_BUILD for test SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 in 55.503356 seconds (PASS)
Starting MODEL_BUILD for test SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 with 16 procs
Finished SHAREDLIB_BUILD for test ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 in 55.506415 seconds (PASS)
Starting MODEL_BUILD for test ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 with 16 procs
Finished SHAREDLIB_BUILD for test ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu in 74.853057 seconds (PASS)
Finished SHAREDLIB_BUILD for test ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 in 72.842032 seconds (PASS)
Finished SHAREDLIB_BUILD for test PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 in 94.990278 seconds (PASS)
Finished SHAREDLIB_BUILD for test ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 in 185.947985 seconds (PASS)
Finished SHAREDLIB_BUILD for test ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 in 205.333348 seconds (PASS)
Starting MODEL_BUILD for test ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 with 16 procs
Finished MODEL_BUILD for test SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 in 197.056169 seconds (PASS)
Starting RUN for test SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 with 1 proc on interactive node and 64 procs on compute nodes
Finished RUN for test SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 in 1.368460 seconds (PEND). [COMPLETED 1 of 9]
Starting MODEL_BUILD for test ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu with 16 procs
Finished MODEL_BUILD for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-aci in 199.783443 seconds (PASS)
Starting RUN for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-aci with 1 proc on interactive node and 64 procs on compute nodes
Finished RUN for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-aci in 0.998598 seconds (PEND). [COMPLETED 2 of 9]
Starting MODEL_BUILD for test ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 with 16 procs
Finished MODEL_BUILD for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics in 201.877301 seconds (PASS)
Starting RUN for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics with 1 proc on interactive node and 64 procs on compute nodes
Finished RUN for test SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics in 0.990930 seconds (PEND). [COMPLETED 3 of 9]
Starting MODEL_BUILD for test ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 with 16 procs
Finished MODEL_BUILD for test ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 in 211.394275 seconds (PASS)
Starting RUN for test ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 with 1 proc on interactive node and 64 procs on compute nodes
Finished RUN for test ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 in 4.993036 seconds (PEND). [COMPLETED 4 of 9]
Starting MODEL_BUILD for test PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 with 16 procs
Finished MODEL_BUILD for test ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu in 420.022279 seconds (PASS)
Starting RUN for test ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu with 1 proc on interactive node and 16 procs on compute nodes
Finished RUN for test ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu in 0.968153 seconds (PEND). [COMPLETED 5 of 9]
Finished MODEL_BUILD for test ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 in 423.851179 seconds (PASS)
Starting RUN for test ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 with 1 proc on interactive node and 64 procs on compute nodes
Finished RUN for test ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 in 1.032048 seconds (PEND). [COMPLETED 6 of 9]
Finished MODEL_BUILD for test PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 in 440.435452 seconds (PASS)
Starting RUN for test PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 with 1 proc on interactive node and 64 procs on compute nodes
Finished RUN for test PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 in 0.900415 seconds (PEND). [COMPLETED 7 of 9]
Finished MODEL_BUILD for test ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 in 553.043374 seconds (PASS)
Starting RUN for test ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 with 1 proc on interactive node and 64 procs on compute nodes
Finished RUN for test ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 in 0.878793 seconds (PEND). [COMPLETED 8 of 9]
Finished MODEL_BUILD for test ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 in 671.596757 seconds (PASS)
Starting RUN for test ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 with 1 proc on interactive node and 64 procs on compute nodes
Finished RUN for test ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 in 1.452641 seconds (PEND). [COMPLETED 9 of 9]
Waiting for tests to finish
PASS ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_D_Lh4.ne4_ne4.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240719_104454_rbkqus
PASS ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4 RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERP_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-4.C.20240719_104454_rbkqus
PASS ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5 RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_D_Ln22.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-rad_frequency_2--scream-output-preset-5.C.20240719_104454_rbkqus
PASS ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2 RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_Ln9.ne4_ne4.F2000-SCREAMv1-AQP1.mappy_gnu.scream-output-preset-2.C.20240719_104454_rbkqus
PASS ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/ERS_P16_Ln22.ne30_ne30.F2010-SCREAMv1-DP-DYCOMSrf01.mappy_gnu.C.20240719_104454_rbkqus
PASS PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1 RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/PET_Ln9_P32x2.ne4pg2_ne4pg2.F2010-SCREAMv1.mappy_gnu.scream-output-preset-1.C.20240719_104454_rbkqus
PASS SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-aci RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-aci.C.20240719_104454_rbkqus
PASS SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.mappy_gnu.scream-mam4xx-optics.C.20240719_104454_rbkqus
PASS SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3 RUN
    Case dir: /ascldap/users/e3sm-jenkins/acme/scratch/SMS_D_Ln9.ne4_ne4.F2010-SCREAMv1-noAero.mappy_gnu.scream-output-preset-3.C.20240719_104454_rbkqus
test-scheduler took 970.7360515594482 seconds'
+ [[ 0 != 0 ]]
+ set +x
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 16909 killed;
[ssh-agent] Stopped.
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script  : #!/bin/bash -le
cd $WORKSPACE/${BUILD_ID}/
./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh
We're having issues with some test-launcher job hanging forever. So let's make sure we clean all penting test-launcher jobs
squeue -o"%.7i %u %40j" | grep e3sm-jenkins | grep test-launcher | awk '{ print $1 }' | xargs -r scancel
[SCREAM_PullRequest_Autotester_Mappy] $ /bin/bash -le /tmp/jenkins13021536926471398896.sh

POST BUILD TASK : SUCCESS

END OF POST BUILD TASK : 0

Sending e-mails to: [email protected]

Finished: SUCCESS

SCREAM_PullRequest_Autotester_Weaver # 5909 FAILED (click to see last 100 lines of console output)


      | ^~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/Cuda/Kokkos_Cuda_Parallel_Team.hpp: In constructor 'Kokkos::Impl::ParallelFor, Kokkos::Cuda>::ParallelFor(const FunctorType&, const Policy&) [with FunctorType = __nv_hdl_wrapper_t, void(const Kokkos::Impl::CudaTeamMember&), const Kokkos::View***, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >, const Kokkos::View**, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >, const double, Kokkos::View*, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> > >; Properties = {Kokkos::Cuda}]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/Cuda/Kokkos_Cuda_Parallel_Team.hpp:533:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  533 |   ParallelFor(const FunctorType& arg_functor, const Policy& arg_policy)
      | ^ ~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp: In function 'void Kokkos::parallel_for(const string&, const ExecPolicy&, const FunctorType&) [with ExecPolicy = Kokkos::MDRangePolicy, Kokkos::IndexType >; FunctorType = __nv_hdl_wrapper_t >, int, int), &Homme::BoundaryExchange::register_field, 1>, void(int, int), Kokkos::View >**, Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >, int, Kokkos::View >, int>; Enable = void]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp:133:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  133 | inline void parallel_for(const std::string& str, const ExecPolicy& policy,
      | ^~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp: In function 'void Kokkos::parallel_for(const string&, const ExecPolicy&, const FunctorType&) [with ExecPolicy = Kokkos::MDRangePolicy, Kokkos::IndexType >; FunctorType = __nv_hdl_wrapper_t >, int, int), &Homme::BoundaryExchange::register_field, 1>, void(int, int), Kokkos::View >**, Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >, int, Kokkos::View >, int>; Enable = void]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/Kokkos_Parallel.hpp:133:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  133 | inline void parallel_for(const std::string& str, const ExecPolicy& policy,
      | ^~~~~~~~~~~~
nvcc_internal_extended_lambda_implementation: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::RRTMGPRadiation::run_impl(double)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {const Kokkos::Impl::CudaTeamMember&}; F1 = const int; F2 = const int; F3 = yakl::Array; F4 = Kokkos::View, Kokkos::MemoryTraits<0> >]':
nvcc_internal_extended_lambda_implementation:690:19: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/Cuda/Kokkos_Cuda_Parallel_Team.hpp: In constructor 'Kokkos::Impl::ParallelFor, Kokkos::Cuda>::ParallelFor(const FunctorType&, const Policy&) [with FunctorType = __nv_hdl_wrapper_t, void(const Kokkos::Impl::CudaTeamMember&), const Kokkos::View***, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >, const Kokkos::View**, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >, const double, Kokkos::View*, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> > >; Properties = {Kokkos::Cuda}]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/Cuda/Kokkos_Cuda_Parallel_Team.hpp:533:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  533 |   ParallelFor(const FunctorType& arg_functor, const Policy& arg_policy)
      | ^ ~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::apply_iop_forcing(scream::Real)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {const Kokkos::Impl::CudaTeamMember&}; F1 = const int; F2 = ekat::WorkspaceManager, Kokkos::Device >; F3 = Kokkos::View, Kokkos::MemoryTraits<0> >; F4 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F5 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F6 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F7 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F8 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F9 = const int; F10 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F11 = const double; F12 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F13 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F14 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F15 = const int; F16 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp:417:1060: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  417 |   Kokkos::parallel_for("compute_qdp_and_vtheta_dp", policy_iop, KOKKOS_LAMBDA (const KT::MemberType& team) {
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::apply_iop_forcing(scream::Real)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {const Kokkos::Impl::CudaTeamMember&}; F1 = const int; F2 = ekat::WorkspaceManager, Kokkos::Device >; F3 = Kokkos::View, Kokkos::MemoryTraits<0> >; F4 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F5 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F6 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F7 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F8 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F9 = const int; F10 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F11 = const double; F12 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F13 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F14 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F15 = const int; F16 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp:417:1060: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  417 |   Kokkos::parallel_for("compute_qdp_and_vtheta_dp", policy_iop, KOKKOS_LAMBDA (const KT::MemberType& team) {
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/share/field/field_alloc_prop.hpp: In copy constructor 'scream::FieldAllocProp::FieldAllocProp(const scream::FieldAllocProp&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/share/field/field_alloc_prop.hpp:106:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  106 |   FieldAllocProp (const FieldAllocProp&) = default;
      | ^ ~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/share/field/field_alloc_prop.hpp: In copy constructor 'scream::FieldAllocProp::FieldAllocProp(const scream::FieldAllocProp&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/share/field/field_alloc_prop.hpp:106:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  106 |   FieldAllocProp (const FieldAllocProp&) = default;
      | ^ ~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::apply_iop_forcing(scream::Real)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {const Kokkos::Impl::CudaTeamMember&}; F1 = const int; F2 = ekat::WorkspaceManager, Kokkos::Device >; F3 = Kokkos::View, Kokkos::MemoryTraits<0> >; F4 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F5 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F6 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F7 = Kokkos::View*****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F8 = Kokkos::View****, Kokkos::LayoutRight, Kokkos::Device, Kokkos::MemoryTraits<0> >; F9 = const int; F10 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F11 = const double; F12 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F13 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F14 = const Kokkos::View, Kokkos::MemoryTraits<0> >; F15 = const int; F16 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_iop.cpp:417:1060: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  417 |   Kokkos::parallel_for("compute_qdp_and_vtheta_dp", policy_iop, KOKKOS_LAMBDA (const KT::MemberType& team) {
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/share/field/field_alloc_prop.hpp: In copy constructor 'scream::FieldAllocProp::FieldAllocProp(const scream::FieldAllocProp&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/share/field/field_alloc_prop.hpp:106:1: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  106 |   FieldAllocProp (const FieldAllocProp&) = default;
      | ^ ~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/KokkosExp_MDRangePolicy.hpp: In copy constructor 'Kokkos::MDRangePolicy, Kokkos::IndexType >::MDRangePolicy(const Kokkos::MDRangePolicy, Kokkos::IndexType >&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/KokkosExp_MDRangePolicy.hpp:153:8: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  153 | struct MDRangePolicy : public Kokkos::Impl::PolicyTraits {
      |        ^~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/KokkosExp_MDRangePolicy.hpp: In copy constructor 'Kokkos::MDRangePolicy, Kokkos::IndexType >::MDRangePolicy(const Kokkos::MDRangePolicy, Kokkos::IndexType >&)':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/externals/ekat/extern/kokkos/core/src/KokkosExp_MDRangePolicy.hpp:153:8: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
  153 | struct MDRangePolicy : public Kokkos::Impl::PolicyTraits {
      |        ^~~~~~~~~~~~~
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::initialize_homme_state()::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {int}; F1 = const int; F2 = const int; F3 = const int; F4 = const Kokkos::View, 1> >* [2][10][4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F5 = const int; F6 = const Kokkos::View, 1> >** [4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F7 = const Kokkos::View, 1> >* [3][4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F8 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp:1210:653: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 1210 |   Kokkos::parallel_for(Kokkos::RangePolicy<>(0,nelem*qsize*NGP*NGP*npacks_mid),
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::initialize_homme_state()::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {int}; F1 = const int; F2 = const int; F3 = const int; F4 = const Kokkos::View, 1> >* [2][10][4][4][128], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F5 = const int; F6 = const Kokkos::View, 1> >** [4][4][128], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F7 = const Kokkos::View, 1> >* [3][4][4][128], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F8 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp:1210:653: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 1210 |   Kokkos::parallel_for(Kokkos::RangePolicy<>(0,nelem*qsize*NGP*NGP*npacks_mid),
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ^
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::HommeDynamics::initialize_homme_state()::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {int}; F1 = const int; F2 = const int; F3 = const int; F4 = const Kokkos::View, 1> >* [2][41][4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F5 = const int; F6 = const Kokkos::View, 1> >** [4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F7 = const Kokkos::View, 1> >* [3][4][4][72], Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<8> >; F8 = const int]':
/home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx/src/dynamics/homme/eamxx_homme_process_interface.cpp:1210:653: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
 1210 |   Kokkos::parallel_for(Kokkos::RangePolicy<>(0,nelem*qsize*NGP*NGP*npacks_mid),
      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             ^
nvcc_internal_extended_lambda_implementation: In static member function 'static void* __nv_hdl_wrapper_t::manager::do_copy(void*) [with Lambda = scream::Field::update_impl(const scream::Field&, int, int, int)::; Tag = __nv_dl_tag; OpFuncR = void; OpFuncArgs = {int}; F1 = Kokkos::View, Kokkos::MemoryTraits<0> >; F2 = Kokkos::View, Kokkos::MemoryTraits<0> >; F3 = const int; F4 = const int; F5 = const int]':
nvcc_internal_extended_lambda_implementation:463:19: note: the layout of aggregates containing vectors with 8-byte alignment has changed in GCC 5
[ 68%] Linking CXX static library libscream_test_support.a
[ 68%] Built target scream_test_support
[ 68%] Linking CXX static library libscream_rrtmgp.a
[ 68%] Built target scream_rrtmgp
[ 68%] Linking CXX static library libscream_theta-l_kokkos_4_72_10.a
[ 68%] Built target scream_theta-l_kokkos_4_72_10
[ 68%] Linking CXX static library libscream_theta-l_kokkos_4_128_10.a
[ 68%] Linking CXX static library libscream_theta-l_kokkos_4_72_41.a
[ 68%] Built target scream_theta-l_kokkos_4_128_10
[ 68%] Built target scream_theta-l_kokkos_4_72_41
[ 68%] Linking CXX static library libscream_control.a
[ 68%] Built target scream_control
gmake: *** [Makefile:166: all] Error 2
Error(s) occurred during test phase

OVERALL STATUS: FAIL

Starting analysis on weaver with cmd: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && export HDF5_USE_FILE_LOCKING=FALSE && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -m weaver

RUN: cd /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx && source /etc/profile.d/modules.sh && module purge && module load cmake/3.25.1 git/2.39.1 python/3.10.8 py-netcdf4/1.5.8 gcc/11.3.0 cuda/11.8.0 openmpi netcdf-c netcdf-fortran parallel-netcdf netlib-lapack && export HDF5_USE_FILE_LOCKING=FALSE && true && bsub -I -q rhel8 -n 4 -gpu num=4 ./scripts/test-all-scream --baseline-dir AUTO $compiler -p -c EKAT_DISABLE_TPL_WARNINGS=ON -m weaver

FROM: /home/e3sm-jenkins/weaver/workspace/SCREAM_PullRequest_Autotester_Weaver/5909/scream/components/eamxx

weaver failed

######################################################

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash -le
cd $WORKSPACE/${BUILD_ID}/
./scream/components/eamxx/scripts/jenkins/jenkins_cleanup.sh

[SCREAM_PullRequest_Autotester_Weaver] $ /bin/bash -le /tmp/jenkins6123839940881798185.sh

POST BUILD TASK : SUCCESS

END OF POST BUILD TASK : 0

Sending e-mails to: [email protected]

Finished: FAILURE

E3SM-Bot · 2024-07-20T02:59:30Z