A question about nCellsAll in MPAS Ocean

[jacob-stu-allen]

Hello!

I'm adding source code modifications to mpas_ocn_surface_bulk_forcing.f90 in mpaso, and I'm confused about the nCells and nCellsAll integers commonly used in the mpas subroutines. For my modifications to work, I need to loop through each cell and make an if statement at each cell of windStressZonal. However, when I print out nCellsAll, it's much smaller than I expected (2252) and restricts my computation to about 190°E-210°E and 15°S-5°S.

How do I loop through each cell on the ocean surface? See subroutine, wind_bolster_user in the attached mpas_ocn_surface_bulk_forcing.f90.

My two modified files are attached. I would attach the ocean log, but it's ~40 mb as I printed out every lat, lon in the do loop starting on line 287.

Thanks in advance,
Jacob

[xylar]

@jacob-stu-allen, you are only printing output from processor 0 by default. You need to compile in debug mode to get output from all processors (each in its own log file).

nCells as I recall is defined differently in different places depending on how deep into the halo (cells owned by other processors) you need to go for a given operation. It is taken from nCellsArray with different indices to indicate how much of the halo to include.

I don't see any mention of nCellsAll in the file you're editing:
https://github.com/E3SM-Project/E3SM/blob/9eba6d6be2b1965da45f433652a4129684416e32/components/mpas-ocean/src/shared/mpas_ocn_surface_bulk_forcing.F
Perhaps you are working with an older version (or seeing that in another file)? In any case, nCellsAll is all owned and halo cells on the processor, which sounds likely to be overkill in your case but also will do no harm other than being a small performance hit.

[jacob-stu-allen]

Hello Xylar, thanks a lot, that makes sense now. I saw nCellsAll being used in another file and borrowed it. Is there some documentation on the owned/halo cells?

Performance wise, I'm guessing it would be best to use nCellsHalo(2)?

[philipwjones]

For the various nCells variables...
nCellsOwned - just the cells owned by local partition in the MPI task with no halo cells
nCellsAll - all the cells the task has access to, including all halos
nCellsHalo(i) - contains all local cells and halo cells up to halo depth i

How far into the halo you want to go depends on where the results of this calculation will be used throughout the code. nCellsOwned is the most efficient if you're confident no halo info is needed. nCellsAll is the safest if you don't want to invalidate any halo layers. Any calculations using horizontal neighbors will invalidate halo layers.

[jacob-stu-allen]

Great, thanks for this added information. My calculation requires only the surface wind stress data from a single cell at each time-step, so I think I should be able to use nCellsOwned. I appreciate it!