Add pio_finalize calls for CICE history and restart files #995
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…g with PIO. This cleans up some PIO internal data. The code was aborting when trying to write lots of files (>600) because we reinitialize PIO for each file. This is a short term fix. A longer term fix is to initialize PIO once and reuse as much as possible. Add a new test that runs CICE for a year writing hourly files (>8000 files). Test with pio2 and netcdf.
apcraig
requested review from
eclare108213,
dabail10,
DeniseWorthen and
anton-seaice
DeniseWorthen
approved these changes
Nov 14, 2024
eclare108213
approved these changes
Nov 14, 2024
anton-seaice
suggested changes
Nov 14, 2024
| integer(kind=int_kind) :: ierr | ||
| character(len=*), parameter :: subname = '(ice_pio_finalize)' | ||
|
|
||
| call pio_finalize(ice_pio_subsystem,ierr) |
There was a problem hiding this comment.
Mostly when there is an ierr code returned from a pio function, elsewhere there is a
call pio_seterrorhandling(File, PIO_RETURN_ERROR)
before the call, then the ierr is checked with ice_pio_check and the error handling is set back to internal
There was a problem hiding this comment.
Good catch, I've updated the implementation.
|
|
||
| integer(kind=int_kind) :: ierr | ||
| character(len=*), parameter :: subname = '(ice_pio_finalize)' | ||
|
|
There was a problem hiding this comment.
Suggested change
| #ifndef CESMCOUPLED |
This is handled elsewhere for CESMCOUPLED
There was a problem hiding this comment.
Good catch, I've updated the implementation.
anton-seaice
approved these changes
Nov 14, 2024
|
Thanks @apcraig |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR checklist
Add pio_finalize calls for CICE history and restart files
apcraig
Tested quick_suite and io_suite on derecho with 6 compilers. All tests pass and are bit for bit. New tests pass. One old test, restart_gx3_32x1_alt02_histall_iohdf5_iopio1 now is failing with an hdf5 dimscales error. I tried to look into the problem, but this seems to be a problem on with pio1 and hdf5. I am going to allow this to fail.
This fixes a "too many communicators" error caught in UFS and duplicated in standalone CICE when writing lots of files.
Add pio_finalize calls for CICE history and restart files when running with PIO. This cleans up some PIO internal data. The code was aborting when trying to write lots of files (>600) because we reinitialize PIO for each file. This is a short term fix. A longer term fix is to initialize PIO once and reuse as much as possible.
Add a new test that runs CICE for a year writing hourly files (>8000 files). Test with pio2 and netcdf.