Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds MAM4xx: Aerosol Cloud Interactions (ACI) interface #2794

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
142 commits
Select commit Hold shift + click to select a range
f6246e1
Started with MAMOptics and changed to MAMAci
singhbalwinder Nov 5, 2023
3da2c97
Minimal additions for a MAMAci class with a constructor
singhbalwinder Nov 6, 2023
1386145
Model compiles after registering the process, added extra stuff, will…
singhbalwinder Nov 7, 2023
10f5790
Removed un-needed code, minimal code for invoking and printing init,s…
singhbalwinder Nov 8, 2023
4a59aca
Minimal code for adding a test for aci with two outputs
singhbalwinder Nov 14, 2023
6d2d273
Adds some required inputs for the aci process
singhbalwinder Nov 14, 2023
3dad804
Fixes unit for the number mixing ratio
singhbalwinder Nov 14, 2023
1557547
Adds Commented out code to stage next changes
singhbalwinder Nov 15, 2023
f9539b1
Adds aerosol fields as required and also in the input yaml
singhbalwinder Nov 17, 2023
1333331
Adds cloud borne aerosols; fixes a minor bug in mam coupling
singhbalwinder Nov 17, 2023
25d45f0
Adds all input output variables in set grid
singhbalwinder Nov 21, 2023
7f9d5a3
Adds commented out Fortran code with comments, some cleanup
singhbalwinder Nov 25, 2023
6ebbf3a
Cleanup
singhbalwinder Nov 28, 2023
a05b975
Add some scratch views to store density and velocity.
overfelt Nov 30, 2023
82eb4d8
Add subgrid_mean_updraft implementation.
overfelt Dec 5, 2023
512818d
Add call to nucleate_ice_
overfelt Dec 7, 2023
c1a6635
Update eamxx_mam_aci_process_interface to call dropmixnuc.
overfelt Dec 19, 2023
7a5aaec
Just stashing a working version.
overfelt Jan 23, 2024
83ba683
General code clean up by using structs from mam_coupling.hpp
overfelt Jan 25, 2024
1b174de
Fix accidental push of wrong file.
overfelt Jan 25, 2024
7621745
Fix accidental push of wrong file.
overfelt Jan 25, 2024
4880e6c
Fixes for Cuda compiler. Requires latest mam4xx to work.
overfelt Jan 31, 2024
161b8ed
Fixes broken aci test after rebase
singhbalwinder Feb 6, 2024
ada26c8
Finally fix all the problems with running on the GPU.
overfelt Feb 5, 2024
608369a
Fixes compilation errors and update submodules of mam4xx and haero
singhbalwinder Feb 7, 2024
26a1c50
Add wet/dry atm and aerosol states with buffer
singhbalwinder Feb 18, 2024
9da7f70
Replaces pmid and zmid with quanties coming from EAMxx
singhbalwinder Feb 20, 2024
8cf3ad7
clag format+ some minor movement of variables
singhbalwinder Mar 14, 2024
93fb192
Adds some temp comments and comment out state_q and qqcw inputs
singhbalwinder Mar 15, 2024
6f0132d
Adds call to progs and atm for getting state_q for dropmixnuc
singhbalwinder Mar 17, 2024
5f5f2a7
Adds a working state_q extraction
singhbalwinder Mar 18, 2024
96f577b
Adds a working qqcw for dropmixnuc
singhbalwinder Mar 18, 2024
ce90fd2
A working state_q1 as a 2d view
singhbalwinder Mar 18, 2024
5c5f90b
Particle args for dropmixnuc upto state_q works
singhbalwinder Mar 18, 2024
6fd9bec
A working qqcw using a scratch memory with partial arg list
singhbalwinder Mar 18, 2024
812bc3f
Full dropmixnuc call is working now
singhbalwinder Mar 18, 2024
19d31b2
Creates state_q with columns and scratch memory
singhbalwinder Mar 18, 2024
a215ef3
Some cleanup-still need a lot of cleanup
singhbalwinder Mar 19, 2024
fb40cd6
Adds a comment
singhbalwinder Mar 19, 2024
7a16e65
Modified species order to match E3SM with the folling change in mam4xx
singhbalwinder Mar 20, 2024
b841ffc
Reverted back mam_coupling changes for indexing but added a func to g…
singhbalwinder Mar 23, 2024
a1d289b
Passes dt as an arg to make hetro freeze work
singhbalwinder Mar 25, 2024
f958170
Codes works after changes in mam4xx hetfrz.hpp were reverted to original
singhbalwinder Mar 25, 2024
2ab183f
Moves function get_haero_prognostics_index to mam4xx
singhbalwinder Mar 25, 2024
cbe943d
Some minor FIXME comments and cleanup
singhbalwinder Mar 27, 2024
3ffe006
Another round of minor cleanup
singhbalwinder Mar 27, 2024
2d4ecf9
Replaces dtmicro with dt and some cleanup
singhbalwinder Mar 28, 2024
f7b9c11
Replaces ncldwtr with dry_atmospher.nc
singhbalwinder Mar 28, 2024
eb06b3b
changes some variable names and add dummy comments to fix later
singhbalwinder Mar 28, 2024
4952a92
Models compiles fine after the rebase off of optics branch
singhbalwinder Mar 28, 2024
59bf802
Single process test is working with a new init file now
singhbalwinder Mar 30, 2024
1afbf98
Removes some variables from the input file and cleanup
singhbalwinder Apr 2, 2024
94cd1fa
Cleanup of some I/O variables for ice nuc and dropmixnuc processes
singhbalwinder Apr 2, 2024
3c90273
Further cleanup and code lines moved to match I/O for the process
singhbalwinder Apr 2, 2024
08430d5
Adds eddy diff of heat as an output for SHOC
singhbalwinder Apr 2, 2024
bb0e696
Adds tkh-eddy diffusivity of heat in SHOC-2nd attempt
singhbalwinder Apr 3, 2024
7fc6b61
Adds a multi process test
singhbalwinder Apr 3, 2024
4f0e085
Adds two multi-process tests and changes to SHOC for copying tkh_out
singhbalwinder Apr 4, 2024
dc5cec2
Adds shoc and aci multi process test
singhbalwinder Apr 4, 2024
ad8ec23
Adds another multi-process test with optics
singhbalwinder Apr 4, 2024
929da35
Adds w_variance in SHOC
singhbalwinder Apr 8, 2024
0e2b470
Add old and new cloud fraction inputs from SHOC
singhbalwinder Apr 9, 2024
968f5f3
A lot of cleanup and changed Fieldlayour for dgnum
singhbalwinder Apr 9, 2024
5f152af
Clean up round for initialize impl
singhbalwinder Apr 10, 2024
9ce18af
Moves oemga to DryAtmosphere
singhbalwinder Apr 10, 2024
5c1c7a6
Fixes cld frac zero out bug and removes wet_atm_ from run_impl
singhbalwinder Apr 11, 2024
4ba9252
Reaaranges args for store_liquid_cloud_fraction and add const
singhbalwinder Apr 11, 2024
1c3f930
Fixes k loop range, adds wsubmin as input for nuc ice and re-orders i…
singhbalwinder Apr 11, 2024
b4d191d
Fixes in/outs of store_liquid_cloud_fraction
singhbalwinder Apr 11, 2024
449af2c
Removes atm vars from the dropmixnuc arg list
singhbalwinder Apr 11, 2024
e2c1278
Moves some i/o vars in dropmixnuc call, not done yet
singhbalwinder Apr 11, 2024
ab24261
Moves ndrop init functions outside of kokkos loop
singhbalwinder Apr 11, 2024
8c96855
Moves some variables into output and moves some code around in dropmi…
singhbalwinder Apr 12, 2024
5c221ab
Moves some more fields to output
singhbalwinder Apr 12, 2024
9235c84
Adds wsubmin namelist and also a lot of debugging cout stamenents for…
singhbalwinder Apr 13, 2024
f179ffb
Removes subgrid_mean_updraft velocities functions as they are not used
singhbalwinder Apr 13, 2024
dbbd5a5
One grid cell validated code with all debug statements
singhbalwinder Apr 19, 2024
9a08cc6
Fakes interfaces values for kvh and wsec; fixes test and namelist wsu…
singhbalwinder Apr 19, 2024
fa3cc91
Moves all resize views codes in set_grid
singhbalwinder Apr 19, 2024
4d6ec50
Connects ni_activated with P3
singhbalwinder Apr 20, 2024
4ea07a3
nctend or tendnd is not connected with P3
singhbalwinder Apr 20, 2024
39957f9
Adds a branched mam4xx submodule with ndrop validation changes
singhbalwinder Apr 20, 2024
e78203f
Removes ACIRuntime namelist struct and used wsubmin on its own
singhbalwinder Apr 21, 2024
e34a548
Clean up and moves validation intput output in functions
singhbalwinder Apr 21, 2024
831ffbb
further cleanup for the output to be used by other processes
singhbalwinder Apr 21, 2024
e7730c7
Adds some comments
singhbalwinder Apr 21, 2024
7f0cf90
Adds functions to update to qqcw and state
singhbalwinder Apr 22, 2024
19606b8
Removes finanlize from cpp and mam_copy functions
singhbalwinder Apr 22, 2024
9f8b2af
Removes another set of unused mam copy functions
singhbalwinder Apr 22, 2024
931278e
Removes nihf_ from FM
singhbalwinder Apr 22, 2024
5f23711
Removes all ice nuc diags and ccn from FM
singhbalwinder Apr 22, 2024
f4b01a6
Removes some unused variables from FM
singhbalwinder Apr 22, 2024
4d3b83d
Removed all diagnostics from FM
singhbalwinder Apr 22, 2024
bf2e4a5
Replaces a for with parallel for for klev
singhbalwinder Apr 22, 2024
5a9c5ec
Some more cleanup
singhbalwinder Apr 23, 2024
387cbf8
Adds updated submodule
singhbalwinder Apr 23, 2024
9187869
Adds the post processing step
singhbalwinder Apr 23, 2024
dc3e08f
Adds top_lev as a namelist
singhbalwinder Apr 23, 2024
d6f8a94
Repaces parallel_for copy logic with deep_copy
singhbalwinder Apr 23, 2024
c76b426
Add function call to get interface values for wsec and kvh
singhbalwinder Apr 23, 2024
21cba9d
Removes unused field tags and add missing namelist vars in the intput…
singhbalwinder Apr 23, 2024
a4be737
Adds a CIME test
singhbalwinder Apr 23, 2024
cd1f99a
Removes print statements and recomputes dry/wet mmrs in mam_coupling
singhbalwinder Apr 23, 2024
94a80d9
Revives small kernels shoc function, changes shell cmd for CIME test,…
singhbalwinder Apr 23, 2024
888fdb2
Adds int def after reabse
singhbalwinder Apr 23, 2024
7e33cda
Fixes GNU compilation: Make views public and removed an unused variable
singhbalwinder Apr 24, 2024
9f54b0d
PM-GPU: Fixes nested Kokkos loops compilation
singhbalwinder Apr 25, 2024
365325c
Removes a debug change
singhbalwinder Apr 25, 2024
727cc8a
Changes needed for single cell validation
singhbalwinder Apr 30, 2024
8f237ff
Fixes GPU runtime error about illegal memory access
singhbalwinder Apr 30, 2024
6c77406
Clang format
singhbalwinder Apr 30, 2024
c047515
After rebase changes, adds make_layout for aci
singhbalwinder May 1, 2024
f7d3e40
Fixes wrong layouts and removed function to copy views
singhbalwinder May 1, 2024
5e90223
Fixes process order by adding spa, adds dgnum nominal value
singhbalwinder May 3, 2024
99782e4
Explicitly specify do_predict_nc and do_prescribed_ccn flags
singhbalwinder May 3, 2024
2fcc3c3
Temporarily adds cloudborne aerosols in IC section of the namelist
singhbalwinder May 4, 2024
efa2119
Fix allocation of tkh.
overfelt May 6, 2024
893df5e
Seems that tkh is an array of nlev+1, not nlev.
overfelt May 6, 2024
04bd1a6
Use this version if tkh is nlev in length, not nlev+1.
overfelt May 6, 2024
cec5469
Init all computed vars in aci to zero; remove tkh_out from SHOC
singhbalwinder May 10, 2024
c162ee4
Removes some unneeded deep copies
singhbalwinder May 10, 2024
6349d14
Adds ndropmix and nsource as output variables
singhbalwinder May 13, 2024
3ce0a91
Adds a temporary varaible to capture nc before aci
singhbalwinder May 14, 2024
4f295d6
Temporarily adds ccn diagnostics
singhbalwinder May 16, 2024
380ad10
Fixes optics test, this commit has ACI single grid cell printouts
singhbalwinder May 19, 2024
98ae17e
Fixes optics tests
singhbalwinder May 19, 2024
b3ea7ba
Rebases mam4xx submodule
singhbalwinder May 19, 2024
423add5
Moves aci functions to a separate hpp file
singhbalwinder May 24, 2024
570e910
Fixes GPU access inaccessible memory space error with deep_copy
singhbalwinder May 25, 2024
483db99
Moves an include from hpp to cpp
singhbalwinder May 25, 2024
1db5be8
Adds a new function to copy views
singhbalwinder May 28, 2024
b00adf2
Updates CMAKE for the aci tests to mimic P3 CMAKE to enable np1 vs np…
singhbalwinder May 28, 2024
be1417e
Changes after rebase
singhbalwinder May 28, 2024
2e0ebfb
Fixes np1_vs_npX standalone tests and adds mam4xx vars to output
singhbalwinder May 30, 2024
a2afdfc
Adds some TODO comments and minor cleanup
singhbalwinder May 31, 2024
49db86d
Changes TeamThreadRange to TeamVectorRange; [=] to [&]; removes aitke…
singhbalwinder May 31, 2024
f8a1992
GPU fixes: changes array of views to views in Kokkos loop
singhbalwinder Jun 1, 2024
2e4832c
Fixes warnings and changes [=] to [&]
singhbalwinder Jun 1, 2024
2beef5c
Fixes LABLES of the tests to change mam to mam4_aci for time reporting
singhbalwinder Jun 4, 2024
699a486
Fixed a layout and Clang format
singhbalwinder Jun 5, 2024
3f01e52
Adds a cime test for aci
singhbalwinder Jun 5, 2024
ea3a693
Fix uninitialized variable and a few Cuda compiler warnings.
overfelt Jun 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cime_config/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -690,6 +690,7 @@
"time" : "01:00:00",
"tests" : (
"SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.scream-mam4xx-optics",
"SMS_D_Ln5.ne4pg2_oQU480.F2010-SCREAMv1-MPASSI.scream-mam4xx-aci",
)
},

Expand Down
33 changes: 33 additions & 0 deletions components/eamxx/cime_config/namelist_defaults_scream.xml
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,12 @@ be lost if SCREAM_HACK_XML is not enabled.
<Ckm type="real" doc="Eddy diffusivity coefficient for momentum">0.1</Ckm>
</shoc>

<!-- MAM4xx-ACI -->
<mam4_aci inherit="atm_proc_base">
<wsubmin type="real" doc="Minimum diagnostic sub-grid vertical velocity">0.001</wsubmin>
kaizhangpnl marked this conversation as resolved.
Show resolved Hide resolved
<top_level_mam4xx type="integer" doc="Top level for MAM4xx">6</top_level_mam4xx>
singhbalwinder marked this conversation as resolved.
Show resolved Hide resolved
</mam4_aci>

<!-- CLD fraction -->
<cldFraction inherit="atm_proc_base"/>

Expand Down Expand Up @@ -481,6 +487,33 @@ be lost if SCREAM_HACK_XML is not enabled.
<surf_sens_flux >0.0</surf_sens_flux>
<surf_lw_flux_up >0.0</surf_lw_flux_up>
<surf_mom_flux type="array(real)">0.0,0.0</surf_mom_flux>
<!-- Note: MAM4xx-ACI needs dry aerosol diameter (dgnum), currently set to aitken model nominal diameter, remove it once all of MAM4xx is integrated -->
<dgnum type="real" doc="Dry aerosol particles diameter in meters">2.6e-08</dgnum>
<bc_c1 type="real">0.0</bc_c1>
<bc_c3 type="real">0.0</bc_c3>
<bc_c4 type="real">0.0</bc_c4>
<dst_c1 type="real">0.0</dst_c1>
<dst_c3 type="real">0.0</dst_c3>
<so4_c1 type="real">0.0</so4_c1>
<so4_c2 type="real">0.0</so4_c2>
<so4_c3 type="real">0.0</so4_c3>
<pom_c1 type="real">0.0</pom_c1>
<pom_c3 type="real">0.0</pom_c3>
<pom_c4 type="real">0.0</pom_c4>
<soa_c1 type="real">0.0</soa_c1>
<soa_c2 type="real">0.0</soa_c2>
<soa_c3 type="real">0.0</soa_c3>
<nacl_c1 type="real">0.0</nacl_c1>
<nacl_c2 type="real">0.0</nacl_c2>
<nacl_c3 type="real">0.0</nacl_c3>
<mom_c1 type="real">0.0</mom_c1>
<mom_c2 type="real">0.0</mom_c2>
<mom_c3 type="real">0.0</mom_c3>
<mom_c4 type="real">0.0</mom_c4>
<num_c1 type="real">0.0</num_c1>
<num_c2 type="real">0.0</num_c2>
<num_c3 type="real">0.0</num_c3>
<num_c4 type="real">0.0</num_c4>
singhbalwinder marked this conversation as resolved.
Show resolved Hide resolved
<!-- default ne1024 initial condition files do not have these, so init to zero here -->
<!-- TODO: delete this once we can tell the AD that some fields can be inited by procs -->
<qc hgrid="ne256np4|ne1024np4">0.0</qc>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@

#Default scream has 10 tracers, MAM4xx adds another 31 making a total of 41 tracer
#Set total number of tracers to 41. We are using append here as last entry wins while parsing xml options
./xmlchange --append SCREAM_CMAKE_OPTIONS="SCREAM_NUM_TRACERS 41"

#modify initial condition file to get aerosol species ICs
$CIMEROOT/../components/eamxx/scripts/atmchange initial_conditions::Filename='$DIN_LOC_ROOT/atm/scream/init/screami_mam4xx_ne4np4L72_c20240208.nc' -b
kaizhangpnl marked this conversation as resolved.
Show resolved Hide resolved

# Add spa as RRTMG needs spa
$CIMEROOT/../components/eamxx/scripts/atmchange physics::atm_procs_list="mac_aero_mic,spa,rrtmgp" -b

# Replace spa with mam4_aci to invoke mam4 aci scheme
$CIMEROOT/../components/eamxx/scripts/atmchange mac_aero_mic::atm_procs_list="tms,shoc,cldFraction,mam4_aci,p3" -b

#Set precribed ccn to false so that P3 uses input from ACI
$CIMEROOT/../components/eamxx/scripts/atmchange p3::do_prescribed_ccn=false -b

#Set predicted ccn to true so that P3 uses input from ACI
$CIMEROOT/../components/eamxx/scripts/atmchange p3::do_predict_nc=true -b




2 changes: 1 addition & 1 deletion components/eamxx/cmake/machine-files/compy.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ include (${EKAT_MACH_FILES_PATH}/kokkos/openmp.cmake)
include (${EKAT_MACH_FILES_PATH}/mpi/srun.cmake)

#Compy SLURM specific settings
set(EKAT_MPI_NP_FLAG "-p short -n" CACHE STRING "" FORCE)
set(EKAT_MPI_NP_FLAG "-p short --mpi=pmi2 -n" CACHE STRING "" FORCE)
3 changes: 2 additions & 1 deletion components/eamxx/src/physics/mam/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ add_subdirectory(${EXTERNALS_SOURCE_DIR}/mam4xx ${CMAKE_BINARY_DIR}/externals/ma
# EAMxx mam4xx-based atmospheric processes
add_library(mam
eamxx_mam_microphysics_process_interface.cpp
eamxx_mam_optics_process_interface.cpp)
eamxx_mam_optics_process_interface.cpp
eamxx_mam_aci_process_interface.cpp)
target_compile_definitions(mam PUBLIC EAMXX_HAS_MAM)
add_dependencies(mam mam4xx)
target_include_directories(mam PUBLIC
Expand Down
Loading