Merge pull request #57 from JGCRI/dev

Dev

.github/workflows/build.yml

@@ -16,7 +16,7 @@ jobs:
 
       env:
         OS: ${{ matrix.os }}
-        PYTHON: '3.8'
+        PYTHON: '3.9'
 
       steps:
 
@@ -25,7 +25,7 @@ jobs:
         - name: Set up Python
           uses: actions/setup-python@master
           with:
-            python-version: 3.8
+            python-version: 3.9
 
         - name: Install dependencies
           run: |

.zenodo.json

@@ -1,8 +1,8 @@
 {
-    "title": "JGCRI/tethys: v1.3.1", 
-    "version": "v1.3.1", 
+    "title": "JGCRI/tethys: v2.0.0",
+    "version": "v2.0.0",
     "upload_type": "software", 
-    "publication_date": "2022-03-30", 
+    "publication_date": "2023-05-03",
     "creators": [
         {
             "affiliation": "Joint Global Change Research Institue (JGCRI) Pacific Northwest National Laboratory (PNNL)", 

LICENSE

@@ -1,13 +1,11 @@
-CERF
-===
+tethys
 
-Copyright (c) 2021:, Battelle Memorial Institute
+Copyright 2018-present Battelle Memorial Institute
 
-Open source under license BSD 2-Clause
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
 
-1.	Battelle Memorial Institute (hereinafter Battelle) hereby grants permission to any person or entity lawfully obtaining a copy of this software and associated documentation files (hereinafter “the Software”) to redistribute and use the Software in source and binary forms, with or without modification.  Such person or entity may use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and may permit others to do so, subject to the following conditions:
-  -	Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimers.
-  -	Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
-  -	Other than as used herein, neither the name Battelle Memorial Institute or Battelle may be used in any form whatsoever without the express written consent of Battelle.
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
 
-2.	THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BATTELLE OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -1,57 +1,11 @@
 [![build](https://github.com/JGCRI/tethys/actions/workflows/build.yml/badge.svg)](https://github.com/JGCRI/tethys/actions/workflows/build.yml)
-[![codecov](https://codecov.io/gh/JGCRI/tethys/branch/master/graph/badge.svg?token=EzEcfk940f)](https://codecov.io/gh/JGCRI/tethys)
+[![codecov](https://codecov.io/gh/JGCRI/tethys/branch/main/graph/badge.svg)](https://codecov.io/gh/JGCRI/tethys)
 [![docs](https://github.com/JGCRI/tethys/actions/workflows/docs.yml/badge.svg)](https://github.com/JGCRI/tethys/actions/workflows/docs.yml)
 [![DOI](https://zenodo.org/badge/104476654.svg)](https://zenodo.org/badge/latestdoi/104476654)
 
 # Summary
-Tethys serves as a critical step to link Xanthos (a global hydrologic model) and the Global Change Analysis Model (GCAM). The spatial resolution of GCAM is geopolitical regional scale for energy and economy systems (32 regions), and river basins for the land, agriculture, and water systems (235 water basins). GCAM is often used as a boundary condition and coupled to sectoral models, such as the Community Land Model and Xanthos, which typically operate at finer spatial and temporal scales than GCAM. For example, Xanthos is a globally gridded hydrology model that operates at monthly scale. Resolving such a mismatch in spatial and temporal scales facilitated coupling these models together. It is also helpful for understanding seasonal patterns of water use and acquiring high resolution water use data. The main objective of Tethys is to reconstruct global monthly gridded (0.5 geographic degree) water withdrawal datasets by spatial and temporal downscaling water withdrawal estimates at region/basin and annual scale. As an open-access software, Tethys applies statistical downscaling algorithms, to spatially and temporally downscale water withdrawal data from annual region/basin scale into monthly grid scale. In our study, the water withdrawals are separated into six sectors: irrigation, livestock, domestic, electricity (generation), manufacturing and mining.
+Demand for water is often modeled as part of integrated human-Earth systems models, such as GCAM, which operate at coarse spatiotemporal scales in order to model long-term global economic linkages. Tethys was designed to downscale projections from such models, producing high-resolution water demand data consistent with global dynamics. This allows coupling with gridded hydrology models, which represent physical processes on much finer scales.
 
-# Get Started with Tethys
-Set up Tethys using the following steps:
-1.  Install Tethys from GitHub using:
-    ```bash
-    python -m pip install git+https://github.com/JGCRI/tethys.git
-    ```
-
-2.  Download the example data using the following in a Python prompt:
-    ```python
-    import tethys
-    
-    # the directory that you want to download and extract the example data to
-    data_dir = "<my data download location>"
-    
-    # download and unzip the package data to your local machine
-    tethys.get_package_data(data_dir)
-    ```
-
-3.  Setup your configuration file (.ini).  Inside the "example" directory that you just downloaded (`data_dir` from step 2 above) there will be two example configuration files:
-- `data_dir`/example_1_3_1/config.ini
-- `data_dir`/example_1_3_1/configDemeter.ini
-Before you can run these examples please find and replace all absolute paths in each of these files from "C:/Z/models/tethysExampleFolders/example_v1_3_1/..." to "`data_dir`/example_1_3_1/...".
-
-4.  To run Tethys:
-
-    ```python
-    import tethys
-    
-    # the path and file name that my example configuration (.ini) file was downloaded to
-    config_file = 'data_dir/example_1_3_1/config.ini'
-    
-    # run Tethys 
-    tethys.run_model(config_file=config_file)
-    ```
-# Citation
-Zarrar Khan, Isaac Thompson, & Chris R. Vernon. (2022). JGCRI/tethys: v1.3.1 (v1.3.1). Zenodo. https://doi.org/10.5281/zenodo.6399488
-
-# Supporting Documents
-Li, X., Vernon, C.R., Hejazi, M.I., Link, R.P., Huang, Z., Liu, L. and Feng, L., 2018. Tethys – A Python Package for Spatial and Temporal Downscaling of Global Water Withdrawals. Journal of Open Research Software, 6(1), p.9. DOI: http://doi.org/10.5334/jors.197
-
-# Contact Us
-For questions, technical support and user contribution, please contact:
-
-Zarrar Khan <[email protected]>
-Chris Vernon <[email protected]>
-
-# Example Data
-Example data is available for download at https://zenodo.org/record/6399117/files/example_v1_3_1.zip?download=1.
-Once extracted, change the paths in `config.ini` to point to the relevant files and directories on your machine.
+# Documentation
+Check out [Getting Started](https://jgcri.github.io/tethys/getting_started.html) for installation and basic usage.
+Details on methodology and advanced usage are available on the [User Guide](https://jgcri.github.io/tethys/user_guide.html).

docs/source/getting_started.rst

@@ -1,108 +1,101 @@
 Getting Started
-==================================
+===============
+This page walks you through the steps of installing **tethys** and running an example. But first:
 
-About
------------------------------------
-Tethys is a spatiotemporal downscaling model for global water use constructed at the Joint Global Change Research Institute of the Pacific Northwest National Laboratory (http://www.globalchange.umd.edu). It serves to link several hydrological models to the Global Change Analysis Model (GCAM) by disaggregating geopolitical region and water basin scale data from GCAM into finer spatial and temporal resolutions.
+Motivation
+----------
+Integrated human-Earth systems models, such as GCAM, can project future water demand at a coarse, regionally-relevant scale by modeling long-term interactions between multiple sectors under a variety of scenarios, while gridded hydrology models simulate physical processes at a much finer spatial and temporal resolution. **tethys** facilitates coupling between these kinds of models by providing finer-scale water demand data while maintaining consistency with coarser-scale global dynamics.
 
-GCAM uses 32 geopolitical regions for energy and economy systems, and 235 water basins for land, agriculture, and water systems, with 5 year timesteps. Hydrological and other sectoral models often need gridded data with 0.5 or 0.125 geographic degree resolution and monthly timesteps in order to model physical processes heavily influenced by surface and subsurface features. Tethys applies statistical downscaling algorithms to reconstruct water withdrawal data at this resolution across six sectors: irrigation, livestock, domestic, electricity generation, manufacturing, and mining. The methodology and equations used are described in more detail in :ref:`downscaling-algorithms`.
-
-.. figure:: _static/workflow.png
+.. figure:: _static/motivation.png
   :width: 100%
-  :alt: workflow
+  :alt: spatial downscaling
   :align: center
   :figclass: align-center
-
-  *Major inputs and outputs of Tethys by six sectors*
-
-Prerequisites
------------------------------------
-* Python (tested on 3.9) https://www.python.org/downloads/ 
-* Java https://www.java.com/en/download/
 
-.. note:: Without Java installed, the dependency gcamreader will be unable to query the GCAM database files.
+While **tethys** is designed to integrate seamlessly with GCAM, it has the ability to downscale region-scale water demand data from other sources as well.
 
 
 Installation
------------------------------------
-Currently, tethys can be cloned from https://github.com/JGCRI/tethys using::
-
-    $ git clone https://github.com/JGCRI/tethys
-	
-The next commands need to be run from within the tethys directory you just downloaded, so change directory with::
+------------
+As a prerequisite, you'll need to have `Python <https://www.python.org/downloads/>`_ installed (version 3.9 or above), and if you plan on querying a GCAM database, `Java <https://openjdk.org>`_ must be installed and added to your path.
 
-	$ cd tethys
+**tethys** can be installed from GitHub using pip::
 
+  pip install git+https://github.com/JGCRI/tethys
 
-Once downloaded, install as a Python package by running *setup.py* from the command line::
+This will automatically install the dependencies. In order to avoid package version conflicts, consider using a virtual environment.
 
-	$ python setup.py install
-	
-This will automatically install the packages listed in :ref:`dependencies`. In order to avoid package version conflicts, consider creating a virtual environment for tethys.
+Try importing **tethys** to confirm that installation was successful:
 
-In the future, easy installation will be available via pip.
+.. code-block:: python
 
-.. _installing-package-data:
+  import tethys
+  
+  tethys.__version__  # should print the version number
 
-Installing Package Data
------------------------------------
-Example data is available for download at https://zenodo.org/record/6399117/files/example_v1_3_1.zip?download=1. 
 
-The data can also be directly downloaded for the latest release version as follows::
+Example Data
+------------
 
-	import tethys
-    
-    # the directory that you want to download and extract the example data to
-    data_dir = "<my data download location>"
-    
-    # download and unzip the package data to your local machine
-    tethys.get_package_data(data_dir)
+Example data and configurations can be downloaded from Zenodo `here <https://doi.org/10.5281/zenodo.7569651>`_, or by using the following:
 
-.. note:: Although the download is 2.1 GB, the extracted data will require around **9.6 GB** of storage space.
+.. code-block:: python
+  
+  tethys.get_example_data()
+  
+The download decompresses to about 4.5GB. By default, it will make a directory called ``example`` at the root of the **tethys** pacakge, but you can specify another path.
 
-Once extracted, change the paths in *config.ini* to point to the relevant files and directories on your machine.
 
 Run
------------------------------------
-Verify the installation was successful by running the following in Python::
+---
+With the example data downloaded, a simple configuration can be run
 
-	import tethys
-	
-Make sure the config file is properly set up and somewhere Python can find it (or use its absolute file path), then run::
+.. code-block:: python
 
-   dmw = tethys.model.run_model('config.ini')
-   
-Logging info should begin printing to the console, and after a few minutes downscaled data and diagnostics output files will be created.
+  # assuming you downloaded to the default location
+  config_file = tethys.default_download_dir + '/example/config_example.yml'
 
+  result = tethys.run_model(config_file)
 
-.. _dependencies:
 
-Dependencies
-------------
+Plotting
+--------
+**tethys** makes use of the `Xarray <https://docs.xarray.dev/en/stable/index.html>`_ package, which provides convenient plotting functionality.
 
-===========	================
-Dependency	Minimum Version
-===========	================
-configobj	5.0.6
-numpy		1.20.3
-pandas		1.2.4
-scipy		1.6.3
-requests	2.20.0
-gcamreader	1.2.5
-===========	================
-
-Optional Dependencies
----------------------
-
-=======================	================
-Dependency      		Minimum Version
-=======================	================
-build					0.5.1
-nbsphinx				0.8.6
-setuptools				57.0.0
-sphinx					4.0.2
-sphinx-panels			0.6.0
-sphinx-rtd-theme		0.5.2
-sphinx-mathjax-offline	0.0.1
-twine					3.4.1
-=======================	================
+.. code-block:: python
+  
+  from matplotlib import colors, pyplot as plt
+  
+  # higher dpi in order to see resolution
+  plt.figure(figsize=(10, 6), dpi=300)
+  
+  # powernorm the color palette in order to see more detail at the low end
+  result.outputs.Municipal.sel(year=2010).plot(norm=colors.PowerNorm(0.25), cmap='viridis_r')
+  
+  plt.show()
+
+
+Dashboard
+---------
+**tethys** uses `Dask <https://docs.dask.org/en/stable/>`_ for parallelization and to lazily compute results. You can launch the dask distributed client in order to view dashboard and monitor the progress of large workflows.
+
+.. note:: viewing the dashboard requires a few other dependencies not automatically installed by **tethys**
+
+.. code-block:: python
+  
+  from dask.distributed import Client
+  
+  # this configuration may need to be different depending on your machine
+  client = Client(threads_per_worker=8, n_workers=1, processes=False, memory_limit='8GB')
+  
+  # link to view the dask dashboard in your browser, probably localhost:8787
+  client.dashboard_link
+  
+  # run tethys AFTER launching the client
+  config_file = tethys.default_download_dir + '/example/config_demeter.yml'
+  result = tethys.run_model(config_file)
+  
+  # this configuration does not write outputs to a file,
+  # so plots are lazily computed when requested
+  result.outputs.Wheat.sel(year=2030).plot(norm=colors.PowerNorm(0.25), cmap='viridis_r')
+  plt.show()

docs/source/index.rst

@@ -1,23 +1,12 @@
 :notoc:
 
-tethys documentation
-====================
+About
+=====
+**tethys** is an open-source spatiotemporal downscaling model for global water demand, written in Python. **tethys** was built at the `Joint Global Change Research Institute <https://www.pnnl.gov/projects/jgcri>`_ in order to:
 
-**Date**: |today| **Version**: |version|
-
-**Useful links**:
-`Source Repository <https://github.com/JGCRI/tethys>`_ |
-`Issues & Ideas <https://github.com/JGCRI/tethys/issues>`_ |
-`Interactive Quickstarter <https://github.com/JGCRI/tethys/blob/main/notebooks/quickstarter.ipynb>`_
-
-**tethys** is an open-source geospatial Python package for evaluating and analyzing future electricity technology capacity expansion feasibility.  **tethys** was created to:
-
- 1) Evaluate the feasibility of a future scenario-driven electricity technology capacity expansion plan as generated by a parent model,
-
- 2) Site power plants in the least cost configuration when considering regional economics an on-the-ground barriers to siting,
-
- 3) Assist planners and modelers of alternate future realizations of the electricity system to gain an understanding of how siting costs and service area congestion may respond under certain stressors.
+ 1) Convert region-scale, annual water demand projections from the `Global Change Analysis Model <http://jgcri.github.io/gcam-doc/index.html>`_ (GCAM) into a gridded monthly format suitable for use with global hydrologic models, and
 
+ 2) Provide localized water demand projections across alternative futures, which are critical for understanding scarcity and planning for future need.
 
 .. panels::
     :card: + intro-card text-center
@@ -89,11 +78,16 @@ tethys documentation
             :text: To the development guide
             :classes: btn-block btn-secondary stretched-link
 
+**Date**: |today| **Version**: |version|
+
+**Useful links**:
+`Source Repository <https://github.com/JGCRI/tethys>`_ |
+`Issues & Ideas <https://github.com/JGCRI/tethys/issues>`_
 
 .. toctree::
    :maxdepth: 2
    :hidden:
-
+   
    getting_started
    user_guide
    modules