Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding cryo fields to coupler budget accounting #6229

Merged
merged 15 commits into from
Apr 16, 2024

Conversation

darincomeau
Copy link
Member

@darincomeau darincomeau commented Feb 9, 2024

Adds five new o2x coupling fields and terms to coupler budget for polar configurations:

Foxo_ismw - water from ice shelf basal melting (either prognostic or data)
Foxo_rrofl - water removed from Antarctica liquid runoff
Foxo_rrofi - water removed from Antarctica solid runoff

Foxo_ismh - heat from ice shelf basal melting
Foxo_rrofih - heat from removed ice runoff

Foxo_frazil_li - frazil ice formed in ice-shelf cavities ('land ice frazil')
Foxo_frazil_q - latent heat associated with land ice frazil

These new fields are used to calculate wpolar and hpolar, which are Antarctic Ice Sheet imbalances in polar configurations. These coupler budget terms replace the existing iceberg wberg and hberg entries in the budget table, which are included in the wpolar and hpolar totals.

Adds a driver namelist option (flds_polar) to only send these coupling fields in polar configurations.

Changes to ocean conservation analysis member:

  • Turns on terms in ocean conservation analysis member when data ice-shelf melt fluxes are used.
  • Removes landIceFreshwaterFluxesOn conditional on writing land ice frazil terms
  • Removes accumulated land ice frazil salinity terms for salt conservation, since land ice frazil is assumed to have 0 salinity.

Further discussion in E3SM-Ocean-Discussion#74

[BFB]
[NML]

@darincomeau
Copy link
Member Author

I was trying to confirm that this did not change coupler budget values in WCYCL1850 cases when I came across #6230. I did confirm this in the development branch, but I'll confirm again here once that issue is resolved.

@darincomeau
Copy link
Member Author

@jonbob and I are in discussions on how to avoid sending additional coupling fields in non-polar configurations.

@darincomeau
Copy link
Member Author

Added Do not merge tag as current implementation is not backwards compatible due to new drv namelist option.

@darincomeau
Copy link
Member Author

Note I moved the Registry conservation check unit changes to it's own PR, since it's strictly not needed here: #6257

@darincomeau
Copy link
Member Author

darincomeau commented Feb 25, 2024

Testing, water budget

Results from a one month ne30pg2_r05_IcoswISC30E3r5.CRYO1850-DISMF test:

Coupler budget:

(seq_diag_print_mct) NET WATER BUDGET (kg/m2s*1e6): period =  monthly: date =     10201     0
                           atm            lnd            rof            ocn         ice nh         ice sh            glc        *SUM*
         wfreeze     0.00000000     0.00000000     0.00000000    -0.17973798     0.17735453     0.00238345     0.00000000     0.00000000
           wmelt     0.00000000     0.00000000     0.00000000     7.58470353     2.20109620    -9.78896708     0.00000000    -0.00316735
           wrain   -31.60939529     5.37280561     0.00000000    26.08642354     0.01250205     0.13062941     0.00000000    -0.00703468
           wsnow    -2.35991690     1.12794625     0.00000000     0.79294270     0.18795323     0.25104383     0.00000000    -0.00003090
          wpolar     0.00000000     0.00000000     0.00000000     0.13523863     0.00000000     0.00000000     0.00000000     0.13523863
           wevap    33.96949591    -2.82779187     0.00000000   -31.06770644    -0.00804431    -0.06604609     0.00000000    -0.00009280
         wrunoff     0.00000000    -0.34373524     0.19473091     0.14641975     0.00000000     0.00000000     0.00000000    -0.00258458
         wfrzrof     0.00000000    -0.15069088    -0.00000000     0.15002219     0.00000000     0.00000000     0.00000000    -0.00066870
          wirrig     0.00000000     0.00000000     0.00000000     0.00000000     0.00000000     0.00000000     0.00000000     0.00000000
           *SUM*     0.00018371     3.17853386     0.19473091     3.64830592     2.57086170    -9.47095648     0.00000000     0.12165963

Ocean conservation analysis:

 MASS FLUXES
 MPAS-Ocean name          kg/s (F)       coupler name     short name     kg/m^2/s*1e6 (F/A)
 frazilFreshwaterFlux    -9.59743463E+07 o2x_Fioo_frazil  wfreeze             -0.18814768
 seaIceFreshwaterFlux     3.87015222E+09 x2o_Fioi_meltw   wmelt                7.58702915
 rainFlux                 1.33073551E+10 x2o_Faxa_rain    wrain               26.08767961
 snowFlux                 4.04511876E+08 x2o_Faxa_snow    wsnow                0.79300328
 icebergFreshwaterFlux    9.56158749E+07 x2o_Fioi_bergw   wberg                0.18744493
 evaporationFlux         -1.58475061E+10 x2o_Foxx_evap    wevap              -31.06738023
 riverRunoffFlux          7.28150055E+07 x2o_Foxx_rofl    wrunoff              0.14274621
 removedRiverRunoffFlux   1.92389165E+06                  wrunoff                              0.00377159
       SUM: river runoff  7.47388971E+07 x2o_Foxx_rofl    wrunoff SUM                          0.14651780
 iceRunoffFlux            2.59254293E+07 x2o_Foxx_rofi    wfrzrof              0.05082410
 removedIceRunoffFlux     5.06523065E+07                  wfrzrof                              0.09929856
       SUM: ice runoff    7.65777358E+07 x2o_Foxx_rofi    wfrzrof SUM                          0.15012265
 landIceFreshwaterFlux    3.00028819E+07                                       0.05881752
   landIceFrazilFlux     -4.09826274E+06 (already in wfreeze, do not sum                )     -0.00803422
 SUM VOLUME FLUXES        1.86289798E+09                                       3.65201689

@darincomeau
Copy link
Member Author

darincomeau commented Feb 25, 2024

The coupler wpolar term is 0.13523863. The freshwater AIS imbalance is the difference of the added terms, iceberg and ice-shelf melting, and the removed runoff terms, liquid and solid runoff around Antarctica, and land ice frazil . Note land ice frazil is subtracted here, since that mass is formed from ocean water, and then is removed from the ocean at the coupling interval (representing refreezing of the ice-shelf base).

From the ocean conservation analysis:
icebergFreshwaterFlux + landIceFreshwaterFlux - (removedRiverRunoffFlux + removedIceRunoffFlux + landIceFrazilFlux) =
0.18744493 + 0.05881752 - (0.00377159 + 0.09929856 + 0.00803422) = 0.13515808

So the totals calculated by the coupler and ocean agree to the first 3 digits.

@darincomeau
Copy link
Member Author

darincomeau commented Feb 25, 2024

Testing, heat budget

Results from a one month ne30pg2_r05_IcoswISC30E3r5.CRYO1850-DISMF test:

Coupler budget:

(seq_diag_print_mct) NET HEAT BUDGET (W/m2): period =  monthly: date =     10201     0
                           atm            lnd            rof            ocn         ice nh         ice sh            glc        *SUM*
         hfreeze     0.00000000     0.00000000     0.00000000     0.05997856    -0.05918321    -0.00079536     0.00000000     0.00000000
           hmelt     0.00000000     0.00000000     0.00000000    -2.57515374     0.32556949     2.25110384     0.00000000     0.00151959
          hnetsw  -167.55416973    35.39744238     0.00000000   130.23057885     0.06110429     1.84805976     0.00000000    -0.01698445
           hlwdn  -333.13726177    80.03177103     0.00000000   241.58017651     5.06359568     6.46531421     0.00000000     0.00359566
           hlwup   387.84017602   -98.56687798     0.00000000  -275.95809072    -6.28612925    -7.02731409     0.00000000     0.00176398
         hlatvap    84.99561077    -7.08858113     0.00000000   -77.70033380    -0.02273561    -0.18419232     0.00000000    -0.00023210
         hlatfus     0.78750427    -0.37639566     0.00000000    -0.26460498    -0.06271999    -0.08377333     0.00000000     0.00001031
          hiroff     0.00000000     0.05028555     0.00000000    -0.05006240     0.00000000     0.00000000     0.00000000     0.00022314
            hsen    19.66703061    -8.29588599     0.00000000   -11.43698307     0.01484109     0.05048082     0.00000000    -0.00051654
          hpolar     0.00000000     0.00000000     0.00000000    -0.04795899     0.00000000     0.00000000     0.00000000    -0.04795899
        hh2otemp     0.55956740     0.00000000     0.00000000    -0.56004950     0.00000000     0.00000000     0.00000000    -0.00048210
           *SUM*    -6.84154243     1.15175819     0.00000000     3.27749673    -0.96565751     3.31888352     0.00000000    -0.05906150

Ocean conservation analysis:

  ENERGY CONSERVATION CHECK

 HEAT FLUXES: explicit
 MPAS-Ocean name            W (raw sum)    coupler name    short name         W/m^2 (flux/A)
 frazilMass * LHF /dt       3.20034667E+13 o2x_Fioo_q      hfreeze               0.06273945
 seaIceHeatFlux            -1.31406202E+15 x2o_Fioi_melth  hmelt                -2.57608133
 shortWaveHeatFlux          6.64246475E+16 x2o_Foxx_swnet  hnetsw              130.21858263
 longWaveHeatFluxDown       1.23230985E+17 x2o_Faxa_lwdn   hlwdn               241.58147347
 longWaveHeatFluxUp        -1.40766806E+17 x2o_Foxx_lwup   hlwup              -275.95861839
 latentHeatFlux            -3.96346127E+16 x2o_Foxx_lat    hlatvap             -77.69951796
 snowFlux * LHF            -1.34985613E+14                 hlatfus              -0.26462519
 iceRunoffFlux * LHF       -8.65131575E+12                 hiroff               -0.01696000
 removedIceRunoffFlux *LHF -1.69026747E+13                 hiroff                               -0.03313593
       SUM: ice runoff*LHF -2.55539904E+13                 hiroff SUM                           -0.05009593
 sensibleHeatFlux          -5.83370009E+15 x2o_Foxx_sen    hsen                -11.43635964
 icebergHeatFlux           -3.27124856E+13 x2o_Fioi_bergh  hberg                -0.06412941
 landIceHeatFlux           -1.00119617E+13                                      -0.01962741
    landIceFrazilHeatFlux   1.36759028E+12 (already in hfreeze, do not sum                )      0.00268102
 SUM EXPLICIT HEAT FLUXES   1.95209392E+15                                       3.82687622

@darincomeau
Copy link
Member Author

darincomeau commented Feb 25, 2024

The coupler hpolar term is -0.04795899. The AIS heat imbalance is the difference of the added terms, associated iceberg and ice-shelf melting, and land ice frazil formation, and the removed term, associated solid runoff around Antarctica. There is no heat flux associated with liquid runoff sent from the coupler, so there is no liquid runoff term in this budget. Note the latent heat associated with land ice frazil is added here since the ocean forms the frazil (releasing heat) before removing the land ice frazil mass at the coupling interval.

From the ocean conservation analysis:
icebergHeatFlux + landIceHeatFlux + landIceFrazilHeatFlux - (removedIceRunoffFlux *LHF ) =
-0.06412941 -0.01962741 + 0.00268102 - (-0.03313593) = -0.04793987

So the totals calculated by the coupler and ocean agree to the first 3 digits.

@darincomeau
Copy link
Member Author

Testing, no impact on non-polar configurations

Confirmed with one month ne30pg2_r05_IcoswISC30E3r5.WCYCL1850 tests before this PR and with this PR, and values in the coupler budget do not change. The only change is wberg->wpolar and hberg -> hpolar.

@darincomeau
Copy link
Member Author

darincomeau commented Feb 25, 2024

Testing, backwards compatibility

I ran a ne30pg2_r05_IcoswISC30E3r5.WCYCL1850 before this PR, then created a new hybrid case off this PR using restarts from the case before this PR, and the job successfully ran.
Part of the reason the flds_polar driver namelist was added was to prevent the model from looking for the new coupling fields in restarts that did not have them, which throws numerous errors to e3sm.log file about setting the missing coupling fields to 0 for backwards compatibility. So this avoids errors in using restarts before this PR in non-polar configurations.

In the situation where a user has an existing case before this PR, then updates the code that includes this PR, rebuilds, and tries to run, the model will error and abort looking for the new driver namelist. Adding flds_polar = .false. to user_nl_cpl will get around this (./case.setup --reset is needed).

Copy link
Contributor

@cbegeman cbegeman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we've resolved the question of the land ice heat flux sign, I'm happy with these changes. You've shown that the coupler accounting agrees with the ocean component's accounting, and that's the main thing I was looking for. Let me know if you would like any of my help with further testing. Thanks for your work on this @darincomeau!

Copy link
Contributor

@xylar xylar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. It has already caught one mistake at least (the sign of the DISMF heat flux) and will be a big improvement in Polar runs going forward.

@darincomeau
Copy link
Member Author

darincomeau commented Feb 27, 2024

Additional Testing

Passes

SMS_Lm1_P512.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel
SMS_Lm1_P512.ne30pg2_r05_IcoswISC30E3r5.CRYO1850.chrysalis_intel
SMS_Lm1_P512.ne30pg2_r05_IcoswISC30E3r5.CRYO1850-DISMF.chrysalis_intel

ERS_Ld31_P512.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel (broken see #6230)
ERS.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel
ERS_Ld31_P512.ne30pg2_r05_IcoswISC30E3r5.CRYO1850.chrysalis_intel

Also ran some G-case tests, where the budgets aren't useful but wanted to make sure nothing unexpected happened:

SMS_Lm1_P512.TL319_IcoswISC30E3r5.GMPAS-JRA1p5.chrysalis_intel
SMS_Lm1_P512.TL319_IcoswISC30E3r5.GMPAS-JRA1p5-DIB-PISMF.chrysalis_intel
SMS_Lm1_P512.TL319_IcoswISC30E3r5.GMPAS-JRA1p5-DIB-DISMF.chrysalis_intel

@darincomeau darincomeau force-pushed the cpl/add-cryo-cpl-budgets branch from ba34b6a to 8ab1f4e Compare March 5, 2024 17:04
@darincomeau
Copy link
Member Author

Rebased on v3.0.0

Reran and PASSes

SMS_Lm1_P512.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel
SMS_Lm1_P512.ne30pg2_r05_IcoswISC30E3r5.CRYO1850-DISMF.chrysalis_intel
SMS_Lm1_P512.ne30pg2_r05_IcoswISC30E3r5.CRYO1850.chrysalis_intel

ERS_Ld31_P512.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel
ERS_Ld31_P512.ne30pg2_r05_IcoswISC30E3r5.CRYO1850.chrysalis_intel

@cbegeman
Copy link
Contributor

@darincomeau What do you think about including these changes to driver/ocn_comp_mct.F 34699cd? I think this is consistent with what we discussed: sending both sea ice and land ice frazil mass fluxes whenever non-zero. I added another conditional to avoid computations of melt potential in ice shelf cavities since that variable should only be used by the sea ice model. This commit is not yet tested.

I'll let you make the changes to wpolar and hpolar in driver-mct/main/seq_diag_mct.F90 separately.

What do you think about including these 2 changes in this PR and I'll open a separate one to fix up the ocean analysis members (conservation check and global stats)?

@darincomeau
Copy link
Member Author

@cbegeman @xylar this has been updated to include accounting for land ice frazil.

@darincomeau
Copy link
Member Author

@stephenprice @proteanplanet this one is ready to merged soon, I don't think we need anymore careful reviews but wanted to give you a heads up in case you had any concerns.

@rljacob this one adds coupling fields, and a new driver namelist option. The namelist option was added to only use the new coupling fields in polar configurations, in an attempt to minimize disruption with backwards compatibility and avoid throwing errors. I also made the same changes in driver-mct to driver-moab to keep the code in sync, but I don't know how to test that - I can remove those changes if that's preferred. Also wanted to give you a heads up here in case you had any concerns.

@proteanplanet
Copy link
Contributor

@darincomeau The cryo configuration is being added to https://github.com/E3SM-Project/SimulationScripts/blob/master/archive/PolarGroup/E3SM-Polar-Developer.sh, and we're also setting up an e3sm_cryo_developer test suite in https://github.com/E3SM-Project/E3SM/blob/master/cime_config/tests.py. I didn't see any PEM or PET tests. Even though the code suggests these won't be affected, I'll run them to be sure.

@darincomeau
Copy link
Member Author

Ok thanks @proteanplanet ! I agree it would be good to check PEM/PET, I've been trying to be as careful as possible here to avoid being disruptive. I can also run those since I'm already setup locally.

@darincomeau
Copy link
Member Author

Actually I'm just remembering I did run the e3sm_ice_developer test suite at some point here which includes PEM/PET, but I didn't post results (PASSed) since it was D-cases where the coupler budgets aren't relevant and there was no ocean conservation analysis member to check against.
@jonbob I can run an integration test suite if that'd be helpful.

@jonbob
Copy link
Contributor

jonbob commented Apr 11, 2024

passes:

  • PET_Ln9_PS.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel.allactive-mach-pet
  • ERS_Ld3.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel.allactive-nlmaps

waiting on:

  • PEM_Ln9.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel

jonbob added a commit that referenced this pull request Apr 15, 2024
Adding cryo fields to coupler budget accounting

Adds new o2x coupling fields and terms to coupler budget for polar
configurations:
* Foxo_ismw - water from ice shelf basal melting (either prognostic or
              data)
* Foxo_rrofl - water removed from Antarctica liquid runoff
* Foxo_rrofi - water removed from Antarctica solid runoff
* Foxo_ismh - heat from ice shelf basal melting
* Foxo_rrofih - heat from removed ice runoff
* Foxo_frazil_li - frazil ice formed in ice-shelf cavities ('land ice
                   frazil')
* Foxo_frazil_q - latent heat associated with land ice frazil
These new fields are used to calculate wpolar and hpolar, which are
Antarctic Ice Sheet imbalances in polar configurations. These coupler
budget terms replace the existing iceberg wberg and hberg entries in the
budget table, which are included in the wpolar and hpolar totals.

Adds a driver namelist option (flds_polar) to only send these coupling
fields in polar configurations.

Changes to ocean conservation analysis member:
* Turns on terms in ocean conservation analysis member when data
  ice-shelf melt fluxes are used.
* Removes landIceFreshwaterFluxesOn conditional on writing land ice
  frazil terms
* Removes accumulated land ice frazil salinity terms for salt
  conservation, since land ice frazil is assumed to have 0 salinity.

[NML]
[BFB]
@jonbob
Copy link
Contributor

jonbob commented Apr 15, 2024

Passed PEM_Ln9.ne30pg2_r05_IcoswISC30E3r5.WCYCL1850.chrysalis_intel as well. Merged to next

@jonbob jonbob merged commit e1e7058 into E3SM-Project:master Apr 16, 2024
11 checks passed
<values match="last">
<value compset="_MPASO%IBPISMF_">TRUE</value>
<value compset="_MPASO%IBDISMF_">TRUE</value>
<value compset="_MPASSI%DIB_">TRUE</value>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this logic is missing the case when DIB is not active, but DISMF or PISMF is active.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make an E3SM issue? I already merged this to master and have been in the process of blessing the expected NML and regular DIFFs (which are just due to a change in the field list)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, will do

@proteanplanet
Copy link
Contributor

This appears to have already been merged to next. Can @darincomeau 's changes be included in that?

@jonbob
Copy link
Contributor

jonbob commented Apr 16, 2024

merged to master and expected NML and regular DIFFs blessed -- except for pm-cpu which may not report today

@jonbob
Copy link
Contributor

jonbob commented Apr 17, 2024

NML and regular DIFFs blessed on pm-cpu

jonbob added a commit that referenced this pull request Apr 17, 2024
Fix logical on flds_polar driver namelist

In PR #6229, the logic on the new driver namelist option flds_polar was
not set to true in G-cases where data icebergs were not active, but
ice-shelf melt fluxes were. This caused the model to crash on ocn init
in ocn_comp_mct.F. This expands the logic for defining that namelist
option to TRUE to include these cases.

Fixes #6349

[BFB]
jonbob added a commit that referenced this pull request Apr 18, 2024
Fix logical on flds_polar driver namelist

In PR #6229, the logic on the new driver namelist option flds_polar was
not set to true in G-cases where data icebergs were not active, but
ice-shelf melt fluxes were. This caused the model to crash on ocn init
in ocn_comp_mct.F. This expands the logic for defining that namelist
option to TRUE to include these cases.

Fixes #6349

[BFB]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants