Feature/bump covariance #123
Conversation
interpolator adding.
…t to state method. Update yaml 3DVar yaml file so 3DVar runs through (but produces nans currently).
* Update ci container * allow manual trigger of ci * temporary change to force ci to run * revert temporary change as ci now gets past cmake config step
Co-authored-by: Toby Searle <[email protected]>
…etOffice/orca-jedi into feature/add_simple_dirac_3dvar
creation of json schema again for 3dvar and errorcovariancetoolbox applications.
correlations in Dirac and 3DVar.
correlations in Dirac and 3DVar.
…e/orca-jedi into feature/bump_covariance
correlations in Dirac and 3DVar.
correlations in Dirac and 3DVar.
…e/orca-jedi into feature/bump_covariance
to revert change to not use missing-if-all-missing in atlas interpolator.
…emporary fix for "Adjoint interpolation only works for interpolation schemes that are linear" error in 3dvar.
from increments to fieldset and vice versa.
|
Hi Toby, This PR is now ready for review I think. The main thing this development does is add the ability to use BUMP to create background error correlations in the assimilation which means we can spread the observation information and the increment from any observation is now a blob rather than delta function. I added a new ctest to construct the "NICAS" grid for BUMP. I tried to use settings to make this as quick as possible but note it takes 44 seconds to run ( a bit slower than the other tests ). I've also put in unit tests to test the new functionality I added. All ctests pass. The bbb test has passed see http://fcm1/cylc-review/taskjobs/frld?&suite=mo-bundle-orca-jedi-bump The list of commits is a bit long because it was a branch of a branch which I rebased and it has listed the commit messages from the parent branch as well as the commits for the child. The file differences showing up in github look correct though so I presume it will merge correctly once approved. Thanks, Dan |
There was a problem hiding this comment.
I gave this my best first pass of a review and it is looking good to me! I have a few comments to aid my understanding so that I can give a proper review, as well as a few questions about the interface to the atlas-orca grids that it would be good to nail down.
At some point I would quite like to request a review from someone working on land DA to get their view on the general layout and if they have any additional thoughts.
|
One additional thing that just occured to me - it would be great if you could make some small changes to the README (can be very light touch) just to point to the additional dependencies, and where to go to find out how to write the configuration for the interface to SABER (I assume this is in the jedi technical docs... hopefully). |
Co-authored-by: Toby Searle <[email protected]>
I've added SABER to the dependencies. SABER is (unusually for JEDI) well documented in the standard JEDI documentation here https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/en/latest/inside/jedi-components/saber/index.html I've added a link to this to README too. Let me know if more is required. One question is there a way to see how the README.md would look in github before it goes back to develop? |
custom routine to determine the i and j points.
|
Toby, |
Thanks. I agree about the land-sea mask but for now this sufficient for my next step... Brace yourself the next step is the NEMOVAR interface! The interaction with orca-jedi is not too heavy due to the wonders of JEDI most interactions are through SABER. I did get this all working (again) without too much trouble but then I updated NEMOVAR and now I've got a job of work on... (See https://github.com/MetOffice/orca-jedi/tree/feature/nemovar_interface ) |
Description
Add spatial covariances to the 3dvar and Dirac tests generated using BUMP.
Let there be light! We now have assimilation with spatial covariances.
Developing this change I did encounter an atlas error "Exception: Not implemented: Adjoint interpolation only works for interpolation schemes that are linear" after adding missing values to increments. This was because of the "non_linear: missing-if-all-missing" in the atlas interpolation. I understand that this may be worked on in future so for now I have added a QC filter to remove any observations affected by missing values.
I was considering updating the orca2 background field which has what I consider an incorrect land mask, but in this case it is quite useful as one of the observations is in the mask and it is a good test that my QC filtering is working. I will revisit this when I start looking at the NEMOVAR interface as this will require that the correct orca2 mask is used (or at least that NEMOVAR and orca-jedi agree).
Issue(s) addressed
N/A
Dependencies
List the other PRs that this PR is dependent on:
N/A
Impact
Expected impact on downstream repositories or workflows:
No impact
Checklist