Merge branch 'jgfouca/scream_downstream_2024_11_07' into next (PR #6739)

[BFB]

* jgfouca/scream_downstream_2024_11_07: (182 commits)
  Add ghci-oci
  Workflows: reworked how to skip eamxx testing jobs
  Workflows: added more eamxx workflows
  Workflows: fix logic for eamxx jobs skipping
  Workflows: allow to generate ALL baselines at ones for eamxx
  EAMxx: Use the main branch of mam4xx.
  EAMxx: Add tracer_reader_utils.hpp to microphysics.hpp.
  EAMxx: Remove temporary CPP macro.
  EAMxx: Update description of dry aerosol particle diameters.
  EAMxx: Use a temporary branch for mam4xx to test changes in the photo table code.
  EAMxx: Move read utils files to readfiles folder.
  EAMxx: Refine linoz parameter handling and class member references
  Correcting data types for variables, refining comments, and updating YAML input configurations.
  Addresses review comments: adds doc strings, reverts scorpio file change and cleanup
  Adds emissions file for ne4pg2, removes qaerwat from the input.yaml for single process test
  Using mam4::utils::extract_stateq_from_prognostics to copy data from prog to state_q.
  Refactor: move allocation of views from the tracer data struct interface to helper code. Combine multiple NC file registrations into a single function call: setup_tracer_data.
  Using one data structure to store beginning, end, and output data sets.
  Retrieve dgncur_a, dgncur_awet, wetdens, and qaerwat from FM
  1. Use the Scorpio interface to get altitude from vertical emission files. I was getting zeros or junk values when I employed the AtmInput class. 2. Add altitude_int as a member of the Altitude class.
  ...

.github/actions/show-workflow-trigger/README.md

@@ -0,0 +1,3 @@
+# Composite action to show the trigger of a workflow
+
+If possible, prints also the user that triggered it

.github/actions/show-workflow-trigger/action.yml

@@ -0,0 +1,27 @@
+name: 'Show workflow trigger'
+description: 'Prints what triggered this workflow'
+
+runs:
+  using: "composite"
+  steps:
+    - name: Print trigger info
+      uses: actions/github-script@v7
+      with:
+        script: |
+          const eventName = context.eventName;
+          const actor = context.actor || 'unknown';  // Default to 'unknown' if actor is not defined
+          let eventAction = 'N/A';
+
+          // Determine the event action based on the event type
+          if (eventName === 'pull_request') {
+            eventAction = context.payload.action || 'N/A';
+          } else if (eventName === 'pull_request_review') {
+            eventAction = context.payload.review.state || 'N/A';
+          } else if (eventName === 'workflow_dispatch') {
+            eventAction = 'manual trigger';
+          } else if (eventName === 'schedule') {
+            eventAction = 'scheduled trigger';
+          }
+          console.log(`The job was triggered by a ${eventName} event.`);
+          console.log(`  - Event action: ${eventAction}`);
+          console.log(`  - Triggered by: ${actor}`);

.github/actions/test-all-scream/README.md

@@ -0,0 +1,33 @@
+# Composite action to call test-all-scream inside a workflow
+
+This action is meant to be used inside a workflow. E.g.,
+
+```yaml
+jobs:
+  my-testing:
+    steps:
+      ...
+      - name: run-test-all-scream
+        uses: ./.github/actions/eamxx-test-all-scream
+        with:
+          build_type: <build-type>
+          machine: <machine>
+          run_type: <run-type>
+```
+
+The input run_type is the only input that this action has to explicitly handle.
+As such, this action checks that its value is one of the following.
+
+- nightly: runs tests and then submit to cdash
+- at-run: runs tests without submitting to cdash
+- bless: runs tests and copy output files to baselines folder
+
+As for build_type and machine, we do not prescribe a list of
+valid values, as that will be handled by components/eamxx/scripts/test-all-scream.
+If their values are not supported, it is up to test-all-scram to handle the error.
+As a guideline, however, you may have to ensure that the following exist:
+
+- the file components/eamxx/cmake/machine-files/${machine}.cmake
+- the entry ${machine} in the MACHINE_METADATA dict in components/eamxx/scripts/machine_specs.py
+
+ Questions? Contact Luca Bertagna [[email protected]](mailto:[email protected])

.github/actions/test-all-scream/action.yml

@@ -0,0 +1,104 @@
+name: EAMxx standalone testing
+description: |
+  Run EAMxx standalone testing with required configuration inputs.
+  More precisely, it launches test-all-scream with the proper flags.
+  See components/eamxx/scripts/test-all-scream for more details.
+  The configuration inputs are:
+    - build_type: the type of build to pass to test-all-scream.
+    - machine: the name of the machine to pass to test-all-scream
+    - generate: whether to generate baselines
+    - submit: whether to submit to cdash (unused if generate is 'true')
+
+inputs:
+  build_type:
+    description: 'Build type to run'
+    required: true
+  machine:
+    description: 'Machine name for test-all-scream'
+    required: true
+  generate:
+    description: 'Generate baselines instead of running tests'
+    required: true
+    default: 'false'
+    valid_values:
+      - 'true'
+      - 'false'
+  submit:
+    description: 'Submit results to cdash (unused if generate=true)'
+    required: true
+    default: 'false'
+    valid_values:
+      - 'true'
+      - 'false'
+  cmake-configs:
+    description: 'Semicolon-separated list of key=value pairs for CMake to pass to test-all-scream'
+    required: false
+    default: ''
+
+runs:
+  using: "composite"
+  steps:
+    - name: Set CA certificates env var
+      run: |
+        # Ensure the operating system is Linux
+        if [ "$(uname)" != "Linux" ]; then
+          echo "This action only supports Linux."
+          exit 1
+        fi
+        # Set env var to be used in upload-artifacts phase
+        if [ -f /etc/debian_version ]; then
+          echo "NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt" >> $GITHUB_ENV
+        elif [ -f /etc/redhat-release ] || [ -f /etc/centos-release ] || [ -f /etc/fedora-release ]; then
+          echo "NODE_EXTRA_CA_CERTS=/etc/pki/tls/certs/ca-bundle.crt" >> $GITHUB_ENV
+        else
+          echo "Unsupported Linux distribution"
+          exit 1
+        fi
+      shell: sh
+    - name: Check repo presence
+      run: |
+        if [ ! -d ".git" ]; then
+          echo "Repository is not checked out. Please ensure the repository is checked out before running this action."
+          exit 1
+        fi
+      shell: sh
+    - name: Print build specs
+      run: |
+        echo "Testing EAMxx standalone, for the following configuration:"
+        echo "  build type   : ${{ inputs.build_type }}"
+        echo "  machine      : ${{ inputs.machine }}"
+        echo "  generate     : ${{ inputs.generate }}"
+        echo "  submit       : ${{ inputs.submit }}"
+        echo "  cmake-configs: ${{ inputs.cmake-configs }}"
+      shell: sh
+    - name: Run test-all-scream
+      working-directory: components/eamxx
+      run: |
+        cmd="./scripts/test-all-scream -m ${{ inputs.machine }} -t ${{inputs.build_type}} --baseline-dir AUTO -c EKAT_DISABLE_TPL_WARNINGS=ON"
+        if [ "${{ inputs.generate }}" = "true" ]; then
+          cmd+=" -g"
+        elif [ "${{ inputs.submit }}" = "true" ]; then 
+          cmd+=" -s"
+        fi
+
+        # If cmake-configs is non-empty, add tokens to test-all-scream via "-c key=val"
+        IFS=';' read -ra configs <<< "${{ inputs.cmake-configs }}"
+        for config in "${configs[@]}"; do
+          cmd+=" -c $config"
+        done
+
+        # Print the full command, then run it
+        echo "test-all-scream call: $cmd"
+        $cmd
+      shell: sh
+    - name: Upload ctest logs
+      if: always()
+      uses: actions/upload-artifact@v4
+      with:
+        name: log-files-${{ inputs.build_type }}-${{ inputs.machine }}
+        path: |
+          components/eamxx/ctest-build/*/Testing/Temporary/Last*.log
+          components/eamxx/ctest-build/*/ctest_resource_file.json
+          components/eamxx/ctest-build/*/CMakeCache.txt
+      env:
+        NODE_EXTRA_CA_CERTS: ${{ env.NODE_EXTRA_CA_CERTS }}

.github/workflows/eamxx-gh-pages.yml

@@ -9,15 +9,15 @@ on:
     branches: [ master ]
     # Only if docs-related files are touched
     paths:
-      - components/eamxx/mkdocs.yml
+      - components/eamxx/mkdocs.yaml
       - components/eamxx/docs/**
       - components/eamxx/cime_config/namelist_defaults_scream.xml
   # Runs every time a PR is open against master
   pull_request:
     branches: [ master ]
     # Only if docs-related files are touched
     paths:
-      - components/eamxx/mkdocs.yml
+      - components/eamxx/mkdocs.yaml
       - components/eamxx/docs/**
       - components/eamxx/cime_config/namelist_defaults_scream.xml
 

.github/workflows/eamxx-sa-coverage.yml

@@ -0,0 +1,58 @@
+name: eamxx-sa-coverage
+
+on:
+  workflow_dispatch:
+
+  # Add schedule trigger for nightly runs at midnight MT (Standard Time)
+  schedule:
+    - cron: '0 7 * * *'  # Runs at 7 AM UTC, which is midnight MT during Standard Time
+
+concurrency:
+  # Two runs are in the same group if they are testing the same git ref
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  submit: ${{ github.event_name == 'schedule' && 'true' || 'false' }}  # Submit to cdash only for nightlies
+
+jobs:
+  gcc-openmp:
+    runs-on:  [self-hosted, ghci-snl-cpu, gcc]
+    name: gcc-openmp / cov
+    steps:
+      - name: Check out the repository
+        uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          show-progress: false
+          submodules: recursive
+      - name: Show action trigger
+        uses: ./.github/actions/show-workflow-trigger
+      - name: Run tests
+        uses: ./.github/actions/test-all-scream
+        with:
+          build_type: cov
+          machine: ghci-snl-cpu
+          generate: false
+          submit: ${{ env.submit }}
+          cmake-configs: Kokkos_ENABLE_OPENMP=ON
+  gcc-cuda:
+    runs-on:  [self-hosted, ghci-snl-cuda, cuda, gcc]
+    name: gcc-cuda / cov
+    steps:
+      - name: Check out the repository
+        uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          show-progress: false
+          submodules: recursive
+      - name: Show action trigger
+        uses: ./.github/actions/show-workflow-trigger
+      - name: Run tests
+        uses: ./.github/actions/test-all-scream
+        with:
+          build_type: cov
+          machine: ghci-snl-cuda
+          generate: false
+          submit: ${{ env.submit }}
+          cmake-configs: Kokkos_ARCH_VOLTA70=ON;CMAKE_CUDA_ARCHITECTURES=70

.github/workflows/eamxx-sa-sanitizer.yml

@@ -0,0 +1,62 @@
+name: eamxx-sa-sanitizer
+
+on:
+  workflow_dispatch:
+
+  # Add schedule trigger for nightly runs at midnight MT (Standard Time)
+  schedule:
+    - cron: '0 7 * * *'  # Runs at 7 AM UTC, which is midnight MT during Standard Time
+
+concurrency:
+  # Two runs are in the same group if they are testing the same git ref
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  submit: ${{ github.event_name == 'schedule' && 'true' || 'false' }}  # Submit to cdash only for nightlies
+
+jobs:
+  gcc-openmp:
+    runs-on:  [self-hosted, ghci-snl-cpu, gcc]
+    name: gcc-openmp / cov
+    steps:
+      - name: Check out the repository
+        uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          show-progress: false
+          submodules: recursive
+      - name: Show action trigger
+        uses: ./.github/actions/show-workflow-trigger
+      - name: Run tests
+        uses: ./.github/actions/test-all-scream
+        with:
+          build_type: valg
+          machine: ghci-snl-cpu
+          generate: false
+          submit: ${{ env.submit }}
+          cmake-configs: Kokkos_ENABLE_OPENMP=ON
+  gcc-cuda:
+    runs-on:  [self-hosted, ghci-snl-cuda, cuda, gcc]
+    strategy:
+      fail-fast: false
+      matrix:
+        build_type: [csm, csr, csi, css]
+    name: gcc-cuda / ${{ matrix.build_type }}
+    steps:
+      - name: Check out the repository
+        uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          show-progress: false
+          submodules: recursive
+      - name: Show action trigger
+        uses: ./.github/actions/show-workflow-trigger
+      - name: Run tests
+        uses: ./.github/actions/test-all-scream
+        with:
+          build_type: ${{ matrix.build_type }}
+          machine: ghci-snl-cuda
+          generate: false
+          submit: ${{ env.submit }}
+          cmake-configs: Kokkos_ARCH_VOLTA70=ON;CMAKE_CUDA_ARCHITECTURES=70