From c0f6559b1e91fc2a9c50d7443c02fd2bb87c9106 Mon Sep 17 00:00:00 2001 From: Dhruv Balwada Date: Sun, 16 Jun 2024 18:08:27 -0400 Subject: [PATCH 01/10] Update README.md add statement on need. --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index eab9fc24..1fed45e2 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,10 @@ [![build-and-deploy-book](https://github.com/m2lines/L96_demo/actions/workflows/deploy.yml/badge.svg)](https://github.com/m2lines/L96_demo/actions/workflows/deploy.yml) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/m2lines/L96_demo/HEAD) +This repository provides a set of notebooks to pedagogically introduce the reader to the problem of parameterization in the climate sciences and how machine learning may be used to address it. + +The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. It could be used as a teaching tool, for self-study, or as a reference manual. + ## Structure and Organization of the Repo This project uses [Jupyter Book](https://jupyterbook.org/) to organize a collection of From c3fbdd03966f80cd7fc0b66dfb5c759476ab268b Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sun, 16 Jun 2024 22:08:50 +0000 Subject: [PATCH 02/10] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1fed45e2..96ed03c2 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [![build-and-deploy-book](https://github.com/m2lines/L96_demo/actions/workflows/deploy.yml/badge.svg)](https://github.com/m2lines/L96_demo/actions/workflows/deploy.yml) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/m2lines/L96_demo/HEAD) -This repository provides a set of notebooks to pedagogically introduce the reader to the problem of parameterization in the climate sciences and how machine learning may be used to address it. +This repository provides a set of notebooks to pedagogically introduce the reader to the problem of parameterization in the climate sciences and how machine learning may be used to address it. The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. It could be used as a teaching tool, for self-study, or as a reference manual. From 81e044318bb8c788be419e83dca43b8a0a707aea Mon Sep 17 00:00:00 2001 From: Dhruv Balwada Date: Mon, 17 Jun 2024 08:05:27 -0400 Subject: [PATCH 03/10] Update README.md --- README.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 96ed03c2..35d21800 100644 --- a/README.md +++ b/README.md @@ -3,9 +3,13 @@ [![build-and-deploy-book](https://github.com/m2lines/L96_demo/actions/workflows/deploy.yml/badge.svg)](https://github.com/m2lines/L96_demo/actions/workflows/deploy.yml) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/m2lines/L96_demo/HEAD) + This repository provides a set of notebooks to pedagogically introduce the reader to the problem of parameterization in the climate sciences and how machine learning may be used to address it. -The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. It could be used as a teaching tool, for self-study, or as a reference manual. +The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. + +**Statement of need**: Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. Since the book addresses people from multiple fields the level of pre-requisites required is minimal; a basic understanding of Python and some experience with PDEs or dynamical systems and solving them numerically (an introductory course in numerical methods) can be helpful. This book could be used as a teaching tool, for self-study, or as a reference manual. + ## Structure and Organization of the Repo @@ -51,6 +55,8 @@ This file lives in [conda-linux-64.lock](https://github.com/m2lines/L96_demo/blo ## Building the Book +Most readers interested in learning from this material could just run individual notebooks once they have setup the appropriate environment, or use the binder link provided at the top of this readme. However, some more advanced readers, particularly those wishing to contribute back, may be interested in building the book locally for testing purposes. + To build the book locally, you should first create and activate your environment, as described above. Then run @@ -86,6 +92,7 @@ $ jcache cache remove -p _build/.jupyter_cache ## Contributing + ### Pre-commit We use [pre-commit](https://pre-commit.com/) to keep the notebooks clean. From 378e994f2f3ca18b8af651ff194ec60f42bb83b5 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 17 Jun 2024 12:08:03 +0000 Subject: [PATCH 04/10] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 35d21800..e4f7726e 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ This repository provides a set of notebooks to pedagogically introduce the reader to the problem of parameterization in the climate sciences and how machine learning may be used to address it. -The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. +The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. -**Statement of need**: Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. Since the book addresses people from multiple fields the level of pre-requisites required is minimal; a basic understanding of Python and some experience with PDEs or dynamical systems and solving them numerically (an introductory course in numerical methods) can be helpful. This book could be used as a teaching tool, for self-study, or as a reference manual. +**Statement of need**: Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. Since the book addresses people from multiple fields the level of pre-requisites required is minimal; a basic understanding of Python and some experience with PDEs or dynamical systems and solving them numerically (an introductory course in numerical methods) can be helpful. This book could be used as a teaching tool, for self-study, or as a reference manual. ## Structure and Organization of the Repo @@ -55,7 +55,7 @@ This file lives in [conda-linux-64.lock](https://github.com/m2lines/L96_demo/blo ## Building the Book -Most readers interested in learning from this material could just run individual notebooks once they have setup the appropriate environment, or use the binder link provided at the top of this readme. However, some more advanced readers, particularly those wishing to contribute back, may be interested in building the book locally for testing purposes. +Most readers interested in learning from this material could just run individual notebooks once they have setup the appropriate environment, or use the binder link provided at the top of this readme. However, some more advanced readers, particularly those wishing to contribute back, may be interested in building the book locally for testing purposes. To build the book locally, you should first create and activate your environment, as described above. Then run From ab87d2e3802c415665b33c308a71b69a8a7eadce Mon Sep 17 00:00:00 2001 From: Dhruv Balwada Date: Mon, 17 Jun 2024 08:28:02 -0400 Subject: [PATCH 05/10] Update README.md --- README.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 35d21800..cb450ebd 100644 --- a/README.md +++ b/README.md @@ -6,10 +6,9 @@ This repository provides a set of notebooks to pedagogically introduce the reader to the problem of parameterization in the climate sciences and how machine learning may be used to address it. -The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. - -**Statement of need**: Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. Since the book addresses people from multiple fields the level of pre-requisites required is minimal; a basic understanding of Python and some experience with PDEs or dynamical systems and solving them numerically (an introductory course in numerical methods) can be helpful. This book could be used as a teaching tool, for self-study, or as a reference manual. +The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. Since the book addresses people from multiple fields the level of pre-requisites required is minimal; a basic understanding of Python and some experience with PDEs or dynamical systems and solving them numerically (an introductory course in numerical methods) can be helpful. This book could be used as a teaching tool, for self-study, or as a reference manual. +The easiest way to read the content (non-interactively) is to view it through the book's [website](https://m2lines.github.io/L96_demo/intro.html). For more interactive experience either use the binder link provided above, or setup the appropriate environments on your own machine and interact with each notebook indivdually. ## Structure and Organization of the Repo @@ -90,8 +89,15 @@ To remove cached notebooks, run $ jcache cache remove -p _build/.jupyter_cache ``` -## Contributing +## Contributing and reporting problems + +If you find any problems or mistakes in the material, think something is not clear, or spot errors in the codes, please [open a new issue](https://github.com/m2lines/L96_demo/issues/new) to report these or seek help. + +Also, we welcome any contributions that you would like to make. These can come in the form of: +- Providing solutions to any errors or clarity issues you or others may have spotted and reported on the issues page. +- Suggest and create new notebooks for any additional concepts that are not currently covered. +These contributions can also be made by [opening a new issue](https://github.com/m2lines/L96_demo/issues/new) and starting a discussion about what you would like to contribute, and eventually submitting changes in the form of a new pull request. ### Pre-commit @@ -168,7 +174,7 @@ $ git fetch upstream $ git merge upstream/main ``` -Ideally you will not have any merge conflicts. +Ideally, you will not have any merge conflicts. You are now ready to make a new feature branch. ## References From 2fabb311b5de2e06e3da410b6e4fdd412651b9f3 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 17 Jun 2024 12:30:29 +0000 Subject: [PATCH 06/10] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index d4893ba8..0c2375cf 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ This repository provides a set of notebooks to pedagogically introduce the reader to the problem of parameterization in the climate sciences and how machine learning may be used to address it. -The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. Since the book addresses people from multiple fields the level of pre-requisites required is minimal; a basic understanding of Python and some experience with PDEs or dynamical systems and solving them numerically (an introductory course in numerical methods) can be helpful. This book could be used as a teaching tool, for self-study, or as a reference manual. +The original goal for these notebooks in this Jupyter book was for our [M2LInES](https://m2lines.github.io/) team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. Now this material is presented here for anyone to learn from. The primary audience for this guide is researchers and students trained in climate science wanting to be introduced to machine learning or trained in machine learning and want to get acquainted with the parameterization problem in climate sciences. Since the book addresses people from multiple fields the level of pre-requisites required is minimal; a basic understanding of Python and some experience with PDEs or dynamical systems and solving them numerically (an introductory course in numerical methods) can be helpful. This book could be used as a teaching tool, for self-study, or as a reference manual. The easiest way to read the content (non-interactively) is to view it through the book's [website](https://m2lines.github.io/L96_demo/intro.html). For more interactive experience either use the binder link provided above, or setup the appropriate environments on your own machine and interact with each notebook indivdually. @@ -91,10 +91,10 @@ $ jcache cache remove -p _build/.jupyter_cache ## Contributing and reporting problems -If you find any problems or mistakes in the material, think something is not clear, or spot errors in the codes, please [open a new issue](https://github.com/m2lines/L96_demo/issues/new) to report these or seek help. +If you find any problems or mistakes in the material, think something is not clear, or spot errors in the codes, please [open a new issue](https://github.com/m2lines/L96_demo/issues/new) to report these or seek help. Also, we welcome any contributions that you would like to make. These can come in the form of: -- Providing solutions to any errors or clarity issues you or others may have spotted and reported on the issues page. +- Providing solutions to any errors or clarity issues you or others may have spotted and reported on the issues page. - Suggest and create new notebooks for any additional concepts that are not currently covered. These contributions can also be made by [opening a new issue](https://github.com/m2lines/L96_demo/issues/new) and starting a discussion about what you would like to contribute, and eventually submitting changes in the form of a new pull request. From 0411abcd017d501e45e9eb00b2b8b580a2656ecb Mon Sep 17 00:00:00 2001 From: Dhruv Balwada Date: Mon, 17 Jun 2024 09:22:07 -0400 Subject: [PATCH 07/10] Update references.bib Adding an extra ref. --- references.bib | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/references.bib b/references.bib index 6b6a5f0e..1668cc87 100644 --- a/references.bib +++ b/references.bib @@ -1,5 +1,12 @@ % L96 Jupyter Book +@article{christensen2022parametrization, + title={Parametrization in weather and climate models}, + author={Christensen, Hannah and Zanna, Laure}, + year={2022}, + publisher={Oxford University Press} +} + @article{Lorenz1995, title = {Predictability: a problem partly solved}, journal = {Seminar on Predictability}, From ed45a2a0dc6ca161d49c47d04ed0036c53278eb9 Mon Sep 17 00:00:00 2001 From: Dhruv Balwada Date: Mon, 17 Jun 2024 09:30:03 -0400 Subject: [PATCH 08/10] Update paper.md --- paper.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/paper.md b/paper.md index ddf19fe0..8b61b51a 100644 --- a/paper.md +++ b/paper.md @@ -157,6 +157,13 @@ The material in this Jupyter book is presented over five sections. The first sec The book was created by and as part of M2LInES, an international collaboration supported by Schmidt Futures, to improve climate models with scientific machine learning. The original goal for these notebooks in this Jupyter book was for our team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. This was done as a series of tutorials, each of which was led by a few team members and occurred with a frequency of roughly once every 2 weeks for about 6-7 months. This Jupyter book is a collection of the notebooks used during these tutorials, which have only slightly been edited for continuity and clarity. Ultimately, we are happy to share these resources with the scientific community to introduce our research ideas and foster the use of machine learning techniques for tackling climate science problems. +# Statement of Need +Parameterization of sub-grid processes is a major challenge in climate science. The exact details of this problem are often very context dependent (@christensen2022parametrization), ... + +As described above, these notebooks were originally created to introduce non-domain experts to ideas from the parameterization aspects of climate modeling and how machine learning could be used to potentially address these. Now they have been adapted to act as a pedagogical tool for self-learning, be used as a reference manual, or for teaching some modules in an introductory class on machine learning. +The book is organized in sections that are relatively independent; with the exception that the first section provides a general overview to the parameterization problem in climate models. +Each notebook covers material that can be discussed in roughly an hour-long lecture, and sections can be mixed and matched or ordered as needed depending on the overall learning objectives. + # Acknowledgements This work is supported by the generosity of Eric and Wendy Schmidt by recommendation of Schmidt Futures, as part of its Virtual Earth System Research Institute (VESRI). MAB acknowledges support from National Science Foundation's AGS-PRF Fellowship Award (AGS2218197). From e0d197edaa4c9be1059c7acbc9f5d76fac4c00f3 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 17 Jun 2024 13:30:15 +0000 Subject: [PATCH 09/10] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- paper.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/paper.md b/paper.md index 8b61b51a..2f5ea82a 100644 --- a/paper.md +++ b/paper.md @@ -158,10 +158,10 @@ The material in this Jupyter book is presented over five sections. The first sec The book was created by and as part of M2LInES, an international collaboration supported by Schmidt Futures, to improve climate models with scientific machine learning. The original goal for these notebooks in this Jupyter book was for our team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. This was done as a series of tutorials, each of which was led by a few team members and occurred with a frequency of roughly once every 2 weeks for about 6-7 months. This Jupyter book is a collection of the notebooks used during these tutorials, which have only slightly been edited for continuity and clarity. Ultimately, we are happy to share these resources with the scientific community to introduce our research ideas and foster the use of machine learning techniques for tackling climate science problems. # Statement of Need -Parameterization of sub-grid processes is a major challenge in climate science. The exact details of this problem are often very context dependent (@christensen2022parametrization), ... +Parameterization of sub-grid processes is a major challenge in climate science. The exact details of this problem are often very context dependent (@christensen2022parametrization), ... -As described above, these notebooks were originally created to introduce non-domain experts to ideas from the parameterization aspects of climate modeling and how machine learning could be used to potentially address these. Now they have been adapted to act as a pedagogical tool for self-learning, be used as a reference manual, or for teaching some modules in an introductory class on machine learning. -The book is organized in sections that are relatively independent; with the exception that the first section provides a general overview to the parameterization problem in climate models. +As described above, these notebooks were originally created to introduce non-domain experts to ideas from the parameterization aspects of climate modeling and how machine learning could be used to potentially address these. Now they have been adapted to act as a pedagogical tool for self-learning, be used as a reference manual, or for teaching some modules in an introductory class on machine learning. +The book is organized in sections that are relatively independent; with the exception that the first section provides a general overview to the parameterization problem in climate models. Each notebook covers material that can be discussed in roughly an hour-long lecture, and sections can be mixed and matched or ordered as needed depending on the overall learning objectives. From ee9d68c112215209456d534d8f35c756a01c38b4 Mon Sep 17 00:00:00 2001 From: Dhruv Balwada Date: Mon, 17 Jun 2024 10:15:43 -0400 Subject: [PATCH 10/10] Update paper.md --- paper.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/paper.md b/paper.md index 2f5ea82a..42e4e1e9 100644 --- a/paper.md +++ b/paper.md @@ -158,7 +158,7 @@ The material in this Jupyter book is presented over five sections. The first sec The book was created by and as part of M2LInES, an international collaboration supported by Schmidt Futures, to improve climate models with scientific machine learning. The original goal for these notebooks in this Jupyter book was for our team to work together and learn from each other; in particular, to get up to speed on the key scientific aspects of our collaboration (parameterizations, machine learning, data assimilation, uncertainty quantification) and to develop new ideas. This was done as a series of tutorials, each of which was led by a few team members and occurred with a frequency of roughly once every 2 weeks for about 6-7 months. This Jupyter book is a collection of the notebooks used during these tutorials, which have only slightly been edited for continuity and clarity. Ultimately, we are happy to share these resources with the scientific community to introduce our research ideas and foster the use of machine learning techniques for tackling climate science problems. # Statement of Need -Parameterization of sub-grid processes is a major challenge in climate science. The exact details of this problem are often very context dependent (@christensen2022parametrization), ... +Parameterization of sub-grid processes is a major challenge in climate modeling. The details of this problem may often be very context dependent (@christensen2022parametrization), but much can be learned by addressing the issue in a general and simpler sense. Also, a general approach allows non-domain experts, e.g. machine learning researchers, to engage and contribute more meaningfully. This JupyterBook aims to achieve this target with the help of a simple dynamical system model - Lorenz 96, such that the reader is introduced to the basic concepts with minimal superfluous complexity. It is possible to extend the concepts that are presented here to other dynamical systems, and even to more complex parameterization tasks (some examples can be found at https://m2lines.github.io/publications/), and we hope that researchers and learners aiming to do this find the concepts presented here as a useful stepping stone in this pursuit. As described above, these notebooks were originally created to introduce non-domain experts to ideas from the parameterization aspects of climate modeling and how machine learning could be used to potentially address these. Now they have been adapted to act as a pedagogical tool for self-learning, be used as a reference manual, or for teaching some modules in an introductory class on machine learning. The book is organized in sections that are relatively independent; with the exception that the first section provides a general overview to the parameterization problem in climate models.