Add WOA23 initial condition

[xylar]

This merge adds the WOA 2023 climatology from 1991-2020 as the default initial condition in the global ocean test group.

The WOA23 data was processed using #604.

Checklist

• User's Guide has been updated
• Developer's Guide has been updated
• Documentation has been built locally and changes look as expected
• Document (in a comment titled Testing in this PR) any testing that was used to verify the changes
• New tests have been added to a test suite

[xylar]

Testing

I ran all of the QU240, EC30to60 and ECwISC30to60 test cases for the WOA23 initial condition on Chrysalis.

Some of the files_for_e3sm test cases have not yet completed but there is no sign of trouble, the metadata looks right:

MPAS_Initial_Condition = "World Ocean Atlas 2023 climatology 1991-2020" 

and I don't expect that test case to be affected by a different initial condition except for the metadata.

The results can be found in these 3 directories:

/lcrc/group/e3sm/ac.xylar/compass_1.2/chrysalis/test_20230420/qu240_woa23/ocean/global_ocean/QU240
/lcrc/group/e3sm/ac.xylar/compass_1.2/chrysalis/test_20230420/ec_woa23/ocean/global_ocean/EC30to60
/lcrc/group/e3sm/ac.xylar/compass_1.2/chrysalis/test_20230420/ecwisc_woa23/ocean/global_ocean/ECwISC30to60

[xylar]

@milenaveneziani and @vanroekel, I'm not sure the best way to proceed here. I guess I'm inclined to have you look at the EC30to60 initial condition:

/lcrc/group/e3sm/ac.xylar/compass_1.2/chrysalis/test_20230420/ec_woa23/ocean/global_ocean/EC30to60/WOA23/init/initial_state/initial_state.nc

and E3SM initial condition (after dynamic adjustment):

/lcrc/group/e3sm/ac.xylar/compass_1.2/chrysalis/test_20230420/ec_woa23/ocean/global_ocean/EC30to60/WOA23/files_for_e3sm/ocean_initial_condition/mpaso.EC30to60E2r3.20230420.nc

in whatever way makes sense to you (maybe just ParaVeiw?) and let me know if any issues jump out at you.

[milenaveneziani]

@xylar: yes, I am sorry for not checking in. This is definitely on my radar. I just had a couple of deadlines this week, but I will use paraview to look at the new initial condition hopefully by tomorrow.

[xylar]

@milenaveneziani, this is different from #604, just so that's clear. #604 is on the original WOA 2023 grid and this is on an MPAS mesh. So I'm not trying to nag you or push you, I just figured it was time to move on from #604 since no one had time to give immediate feedback (which is totally understandable). At least this way we know that what I did in #604 does actually work in MPAS-Ocean and compass as expected, we just need to look at the results and see if they don't look crazy.

Thanks for getting to this when you can!

[maltrud]

@xylar the IC name is mpaso.EC30to60E2r3.20230420.nc. does this imply that the grid is different from the standard EC30to60E2r2? I was going to offer to run a case with the new IC, but not if it is a new grid.

[vanroekel]

@maltrud the r3 has the same horizontal mesh as the r2 EC30to60E2 but the bathymetry is changed to GEBCO from ETOPO. So you should be able to run pretty easily with it

[xylar]

@maltrud, this isn't a mesh that we're going to use in E3SM as it is, I'm just testing out the WOA23 initial condition.

When it's time to go to E3SM, I'll change the E3SM verison to E3 and the revision number back to r1. Don't worry too much about the mesh name for now.

[xylar]

does this imply that the grid is different from the standard EC30to60E2r2? I was going to offer to run a case with the new IC, but not if it is a new grid.

Yeah, compass is generating a new jigsaw mesh for these tests so they're not the same as anything we're already using. I don't see a lot of value in running E3SM simulations yet, nor in making a WOA23 initial condition for EC30to60E2r2. We'll run testing later on but I'm just looking for a sanity check for now.

[xylar]

@maltrud the r3 has the same horizontal mesh as the r2 EC30to60E2 but the bathymetry is changed to GEBCO from ETOPO. So you should be able to run pretty easily with it

@vanroekel, yes that's true but (despite the name here), this is not that mesh either. It's a work-in-progress toward EC(wISC)30to60E3r1.

[vanroekel]

Oh I see. Thanks for the clarification @xylar

[xylar]

The confusion is coming from the fact that the files_for_e3sm step basically will keep producing the same mesh name it was last asked to produce. I don't know if there's a clean way to avoid that. Maybe having a dummy revision number unless someone sets a config option manually to give it a revision number?

[dengwirda]

@xylar and all, is it worthwhile/possible to look at a few vertical T/S profiles in addition to the surface fields?
I'm not clear on how to do this easily in e.g. paraview...

[xylar]

Actually, I suppose the way to handle that is to just bump the revision number immediately after we merge a mesh PR, so all subsequent testing has the next revision number until we actually use that number. I can go ahead and do that.

[xylar]

@xylar and all, is it worthwhile/possible to look at a few vertical T/S profiles in addition to the surface fields?
I'm not clear on how to do this easily in e.g. paraview...

The paraview extractor can pull out various depth indices. That's what I had in mind.

[xylar]

But, no, we don't have great tools for making profiles of initial conditions yet. @milenaveneziani, do you have anything up your sleeve?

[xylar]

Hopefully, #611 will reduce some of the confusion about mesh names in future testing.

[xylar]

@dengwirda, it should be possible for me to throw together a python script to use the transect plotting from mpas_tools and a geojson file of transects. It's after midnight here but I'll do that tomorrow and get back to you. Good idea!

Here's what I'll start from: MPAS-Dev/MPAS-Tools#324 (comment)

[dengwirda]

Thanks @xylar, I was considering whether assessing (some approximation to) the stability of the stratification may be a good sanity check on the remapping onto the MPAS layers, and/or might help identify columns where it's not quite right. As you say though, this would require a bunch of infrastructure to be put in place, and might be a bridge too far here.

[milenaveneziani]

@dengwirda, @xylar: yes, I do have scripts that make either a vertical profile across a certain transect (say T as a function of x and z) or vertical profiles for a property averaged over a certain region(T(z)).
The former requires the making of the transect mask files ahead of time and works best for scalars. The velocities are noisy because they are not interpolated (they are just the normalVelocity plotted at each edge of the transect mask). Still useful for me, just not as pretty to look at :).

I can share the scripts if interested.

[xylar]

a few EC30to60E3r1 transects after dynamic adjustment

[xylar]

a few ECwISC30to60E3r1 transects after dynamic adjustment

[xylar]

@dengwirda and @milenaveneziani (and anyone else), feel free to chuck transect geojson files my way and I can make more plots.

Alternatively you can follow these instructions to try it out yourselves:
MPAS-Dev/MPAS-Tools#505 (comment)

[milenaveneziani]

Beautiful figures @xylar, thanks!
Here is a good new geojson file that I created recently: /lcrc/group/e3sm/ac.milena/mpas-region_masks/regionsTransects4budget/arctic_atlantic_budget_regionsTransects20230313.geojson
It contains a few long zonal transects in the Atlantic Ocean (34S and 27N, land-to-land) as well as all transects bordering the North Atlantic subpolar gyre and the Arctic Ocean.

[xylar]

@milenaveneziani, thanks for that. Here are some plots based on your transects and mine for the SORRM initial condition (before dynamic adjustment):
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xylar/woa23/sorrm/init/

I will also post the same after dynamic adjustment:
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xylar/woa23/sorrm/dyn_adj/

And both of those for the WC14 mesh:
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xylar/woa23/wc/init/
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.xylar/woa23/wc/dyn_adj/

Keep an eye on those links.

[xylar]

I'm noticing an intermediate layer of low salinity in the FRIS cavity:

I need to investigate by looking at horizontal sections of the extrapolated WOA dataset but it looks like I may have made a mistake in the extrapolation process.

[milenaveneziani]

Keep an eye on those links.

will do! plots look great, thanks!

I generated the vtk files and now I am trying to transfer them to my laptop for viewing them with paraview. Just having some trouble with ssh and port 22 connections these days..

[xylar]

Just having some trouble with ssh

Sea surface height, secure shell or both? ;-)

[milenaveneziani]

ok, I was finally able to look at both the initial_state and after_adjustment file. I don't see anything weird about them (I mostly looked at T and S, various layers, just browsing through paraview). No weird blob is visible in the Arctic or anywhere else.

[milenaveneziani]

Approved based on viewing of new ic files in paraview.

[xylar]

Regarding #610 (comment), I think the extrapolation algorithm is doing exactly what it's meant to do. The weirdness comes from changes in FRIS bathymetry that mean that different inlets get cut off at different depths and the fact that there's a pretty strong salinity gradient from east to west.

The following are from ParaVeiw looking at the WOA23 data set on the 0.25 degree mesh at an intermediate stage of the extrapolation.

At shallower depths, there are contributions from the western and central Ronne, and Filchner, resulting in a moderate salinity:

Then, the central Ronne gets cut off at intermediate depths, so the center of the Ronne cavity suddenly becomes less saline:

And then at deeper depths, the western Ronne gets cut off and only the more saline water from Filchner is avaliable:

Obviously, this isn't super realistic but I don't see how we're going to do a ton better with just extrapolating. It seems like dynamic adjustment isn't too upset about this density distribution. More as soon as SORRM is finished with dynamic adjustment...

[vanroekel]

I took a look through a bunch of paraview slices and combined with figures posted by @xylar I don't see anything odd. So am approving this PR.

[xylar]

@vanroekel and @milenaveneziani, thank you so much for the feedback!

Anyone else, I certainly invite you to continue taking a look at the results. But I'm going to merge this for now. There will still be time to look over the initial condition and a (more) spun-up E3SM run before we approve the E3SM v3 meshes and initial conditions.