Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clearly separate fit and update steps for all ensemble assimilation algorithms #5628

Closed
Blunde1 opened this issue Jun 26, 2023 · 4 comments
Closed
Labels
need-design needs-discussion Issues requiring further discussions
Milestone

Comments

@Blunde1
Copy link
Contributor

Blunde1 commented Jun 26, 2023

Is your feature request related to a problem? Please describe.
All ensemble based assimilation algorithms involve a fitting procedure of ensemble to some parametric or semi-parametric model.
After the fit, it will employ an update to the realizations, based on the fit.
All statistical issues for ensemble based methods reside within the fit part of the algorithm.
To allow us to

Describe the solution you'd like
Create the API so that we at least have fit and update, similar to fit and predict in sklearn.
Note, in the iterative_ensemble_smoother we almost have this, but the fit and predict reside within the iterative procedure, while I would argue it should be outside.

Describe alternatives you've considered
Trying to solve the listed problems above without performing this separation is a difficult task.

@Blunde1 Blunde1 added need-design needs-discussion Issues requiring further discussions labels Jun 26, 2023
@Blunde1 Blunde1 added this to the Localisation milestone Jun 26, 2023
@tommyod
Copy link
Contributor

tommyod commented Jul 3, 2023

Can you elaborate a bit on this? What do you propose as signatures for fit and update in IES?

@Blunde1
Copy link
Contributor Author

Blunde1 commented Jul 3, 2023

fit inputs ensemble and outputs and reports on statistical quantities in the estimation
update inputs ensemble and statistical quantities and updates the realizations.

Thinking the standard EnKF, ES, or similar I would like the fit to output (and report) on the estimated covariance matrix. Note that is often not explicitly calculated. The update would be the actual MV-Normal conditional mean update used to update realizations.

Considering Alg. 1 For the IES in Evensen 2019, this would mean the fit calculates and reports on W_i (the "i" is missing in the paper, but step 8, and Ws should be indexed) and related statistical quantities. update performs step 9, but outside of the loop.

The fit + update steps are usually entangled in the data assimilation literature, and combined called "the analysis step". The statistical estimation (that we should report the quality on) is often hidden both notationally and in the numerical linear algebra. I would like to be more explicit about this, and at least be able to draw out (a report on) the estimates.

@dafeda
Copy link
Contributor

dafeda commented Jan 25, 2024

Can we close or update this issue now that we have a new API in the iterative_ensemble_smoother package?

@tommyod
Copy link
Contributor

tommyod commented Jan 25, 2024

I think we can close it, yes. In the future there could be further discussions on API, but if so they should be written as new concrete issues :)

@tommyod tommyod closed this as completed Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need-design needs-discussion Issues requiring further discussions
Projects
None yet
Development

No branches or pull requests

3 participants