From 46cfe2fd9e5af3e7b08b743ed3c7320cfb0a9cdd Mon Sep 17 00:00:00 2001 From: Paul Natsuo Kishimoto Date: Tue, 5 Nov 2019 23:39:45 +0100 Subject: [PATCH] Update Sphinx docs --- R/README.md | 66 ----------------------- README.md | 56 +------------------- doc/conf.py | 2 +- doc/historical.rst | 9 ++++ doc/index.rst | 12 ++--- doc/model.rst | 4 +- doc/reference.rst | 24 +++++++++ doc/stats.rst | 5 -- doc/usage.rst | 129 +++++++++++++++++++++++++++++++++++++++++++++ python/run | 2 - 10 files changed, 173 insertions(+), 136 deletions(-) delete mode 100644 R/README.md create mode 100644 doc/historical.rst create mode 100644 doc/reference.rst delete mode 100644 doc/stats.rst create mode 100644 doc/usage.rst delete mode 100755 python/run diff --git a/R/README.md b/R/README.md deleted file mode 100644 index a6244810..00000000 --- a/R/README.md +++ /dev/null @@ -1,66 +0,0 @@ -R tools for the iTEM databases -============================== - -Installation ------------- - -Use [`devtools`](https://cran.r-project.org/package=devtools). From source (for instance, to develop the code locally): - -``` -$ git clone git@github.com:transportenergy/database.git -$ Rscript -e "devtools::install_local('database/R/item')" -``` - -Or without cloning the repository: - -```r -devtools::install_github('transportenergy/database/R/item') -``` - -Usage ------ - -### From R scripts - -``` r -library(item) - -data <- item::load_model_data(1) # Load version 1 of the iTEM models database -``` - -### Command line - -`run` is an executable that invokes `item::cli()`. It can be used without installing the package. - -``` -$ ./run -Loading item -Usage: ./run [OPTIONS] COMMAND -Command-line interface for the iTEM databases. - -… - -Commands: - mkdirs - debug - load_model_data -``` - - -Development ------------ - -Code conventions and packaging follow the [“R packages” book](http://r-pkgs.had.co.nz/). - -`test` is an executable that runs the tests in `./item/tests/testthat`. The environment variable `ITEM_TEST_DATA` must be defined in order for these tests to work: - -``` -$ export ITEM_TEST_DATA=../../data/model/database -$ ./test -Loading item -Loading required package: testthat -Testing item -Model database: . - -DONE ============================================ -``` diff --git a/README.md b/README.md index cc826ea4..b771789b 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,7 @@ Tools for the iTEM databases ============================ [![Build Status](https://travis-ci.org/transportenergy/database.svg?branch=master)](https://travis-ci.org/transportenergy/database) ![Codecov](https://img.shields.io/codecov/c/gh/transportenergy/database.svg) +[![Documentation Status](https://readthedocs.org/projects/transportenergy/badge/?version=latest)](https://transportenergy.readthedocs.io/en/latest/?badge=latest) [iTEM](https://transportenergy.org) maintains two databases: @@ -17,57 +18,4 @@ This repository contains tools for *both* databases, in *both* Python and R. The - Members of the [iTEM organizing group](https://transportenergy.org/participants/#organizing-group) use the tools to prepare templates for data submission, and to aggregate & clean submitted data. - iTEM participants, if they choose, maintain code that *transparently & reproducibly* derives the iTEM variables from immediate output files of their models. - The tools are well-tested and yield reproducible results. - -Contents --------- - -- `python/` — Python code for accessing & maintaining the database. See [`python/README.md`](https://github.com/transportenergy/database/blob/master/python/README.md). - - `item/data/` — *Meta*data about the iTEM databases. Actual data is stored separately; see below. - - `model/` — Metadata about the iTEM model database. Many of these files are in [YAML format](http://www.yaml.org/spec/1.2/spec.html) ([Wikipedia](https://en.wikipedia.org/wiki/YAML)), with more detailed comments in-line. - - `models.yaml` — description of the [iTEM participating models & teams](https://transportenergy.org/participants/). - - `dimensions/` — information about the [data dimensions of the model database](https://transportenergy.org/database/). - - Other directories: - - are named to match the keys in `models.yaml`, - - contain model-specific metadata, including but not limited to: - - `regions.yaml`: regional aggregation. - - `scenarios.yaml`: descriptions of scenarios submitted to each iTEM workshop. - - `concepts.yaml`, `measures.yaml`, `spec.yaml` — specifications of data to be submitted for for the 3rd iTEM MIP. - - `item_config_example.yaml` — an example configuration file. -- `R/` — R code for accessing & maintaining the database. See [`R/README.md`](https://github.com/transportenergy/database/blob/master/R/README.md). - - -Usage ------ - -The following instructions are not language-specific. -Both the Python and R tools will operate on data stored in the following way; see the language `README`s for specific information. - -1. Clone this repository and/or install the code in one or both languages. - -2. Create 1 or more separate directory to contain the input and output files. - A simple way to do this is to call `item mkdirs PATH`, where `PATH` is a base directory; this will create a tree of directories. - Use `item mkdirs --help` for more information. - -3. Copy the file [`item_config_example.yaml`](https://github.com/transportenergy/database/blob/master/python/item/data/item_config_example.yaml`) to `item_config.yaml` in any working directory where you intend to use this code. - Edit the file (see the inline comments) to point to the directories created in #2 above. - -4. Use the tools through the command-line interface: - - ``` - $ item - Usage: item [OPTIONS] COMMAND [ARGS]... - - Command-line interface for the iTEM databases. - - Options: - --path Override data paths (multiple allowed). - --help Show this message and exit. - - Commands: - debug Show debugging information, including paths. - help Show extended help for the command-line tool. - mkdirs Create a directory tree for the database. - model Manipulate the model database. - stats Manipulate the stats database. - template Generate the MIP3 submission template. - ``` +- The tools are [fully documented](https://transportenergy.readthedocs.io). diff --git a/doc/conf.py b/doc/conf.py index 65252cdc..38c996a5 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -187,7 +187,7 @@ # Example configuration for intersphinx: refer to the Python standard library. intersphinx_mapping = { - 'python': ('https://docs.python.org/', None), + 'python': ('https://docs.python.org/3/', None), 'pandas': ('https://pandas.pydata.org/pandas-docs/stable/', None), } diff --git a/doc/historical.rst b/doc/historical.rst new file mode 100644 index 00000000..b6ffb5cc --- /dev/null +++ b/doc/historical.rst @@ -0,0 +1,9 @@ +Historical & statistical data (:mod:`item.stats`) +************************************************* + +This module contains the code that implements the `iTEM Open Data project `_, the broader aims of which are described on the main iTEM website. + +The present documentation covers the code. + +.. automodule:: item.stats + :members: diff --git a/doc/index.rst b/doc/index.rst index c67aae82..8d25c33d 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -9,12 +9,12 @@ International Transport Energy Modeling (iTEM) toolkit `iTEM`_ maintains two databases: 1. A **model database** of transport energy projections assembled as part of - the `iTEM workshops`_, of which there have been four so far, with a fourth - planned for late 2019. Because of restrictions from workshop participants, + the `iTEM workshops`_, of which there have been four so far, with a fifth + planned for early 2020. Because of restrictions from workshop participants, the model database is currently *not public*, but only available to iTEM workshop participants. -2. A **statistics database** to form a common, public, “best available” +2. A **historical database** to form a common, public, “best available” baseline for model calibration and projections. The statistics database is currently *under development*. @@ -40,11 +40,11 @@ This repository contains tools for *both* databases, in *both* Python and R. The :maxdepth: 2 :caption: Contents: + usage model - stats + historical + reference -Indices and tables -================== * :ref:`genindex` * :ref:`modindex` diff --git a/doc/model.rst b/doc/model.rst index 933598bf..6ab97ac7 100644 --- a/doc/model.rst +++ b/doc/model.rst @@ -1,5 +1,5 @@ -``item.model``: Transport energy model projections -================================================== +Transport energy model projections (:mod:`item.model`) +****************************************************** .. automodule:: item.model :members: diff --git a/doc/reference.rst b/doc/reference.rst new file mode 100644 index 00000000..c4171eb5 --- /dev/null +++ b/doc/reference.rst @@ -0,0 +1,24 @@ +Reference +********* + +Repository organization +----------------------- + +- ``python/`` — Python code for accessing & maintaining the database. + + - ``item/data/`` — *Meta* -data about the iTEM databases. Actual data is stored separately; see below. + + - ``model/`` — Metadata about the iTEM model database. Many of these files are in `YAML format `_ (`Wikipedia `_), with more detailed comments in-line. + + - ``models.yaml`` — description of the `iTEM participating models & teams `_. + - ``dimensions/`` — information about the `data dimensions of the model database `_. + - Other directories: + + - are named to match the keys in ``models.yaml``, + - contain model-specific metadata, including but not limited to: + + - ``regions.yaml``: regional aggregation. + - ``scenarios.yaml``: descriptions of scenarios submitted to each iTEM workshop. + - ``concepts.yaml``, ``measures.yaml``, ``spec.yaml`` — specifications of data to be submitted for for the 3rd iTEM MIP. + - ``item_config_example.yaml`` — an example configuration file. +- ``R/`` — R code for accessing & maintaining the database. diff --git a/doc/stats.rst b/doc/stats.rst deleted file mode 100644 index 1ad45f8f..00000000 --- a/doc/stats.rst +++ /dev/null @@ -1,5 +0,0 @@ -``item.stats``: historical & statistical data -============================================= - -.. automodule:: item.stats - :members: diff --git a/doc/usage.rst b/doc/usage.rst new file mode 100644 index 00000000..6b1616df --- /dev/null +++ b/doc/usage.rst @@ -0,0 +1,129 @@ +Usage +***** + +The following instructions are not language-specific. +Both the Python and R tools will operate on data stored in the following way; see the language-specific sections below for more details. + +1. Clone the repository and/or install the code in one or both languages. + +2. Create 1 or more separate directory to contain the input and output files. + A simple way to do this is to call ``$ item mkdirs PATH``, where ``PATH`` is a base directory; this will create a tree of directories. + Use ``item mkdirs --help`` for more information. + +3. Copy the file ``item_config_example.yaml`` to ``item_config.yaml`` in any working directory where you intend to use this code. + Edit the file (see the inline comments) to point to the directories created in #2 above. + +4. Use the tools through the command-line interface:: + + $ item + Usage: item [OPTIONS] COMMAND [ARGS]... + + Command-line interface for the iTEM databases. + + Options: + --path Override data paths (multiple allowed). + --help Show this message and exit. + + Commands: + debug Show debugging information, including paths. + help Show extended help for the command-line tool. + mkdirs Create a directory tree for the database. + model Manipulate the model database. + stats Manipulate the stats database. + template Generate the MIP3 submission template. + +Python +====== + +Installation +------------ +Use `pip `_. +From source (for instance, to develop the code locally):: + + $ git clone git@github.com:transportenergy/database.git + $ pip install --editable database/python + +Or, without cloning the repository:: + + $ pip install --editable git://github.com/transportenergy/database#egg=item@subdirectory=python + + +Usage +----- + +From Python scripts:: + + import item + + data = item.load_model_data(1) + +Run tests +--------- + +Tests in ``python/item/tests`` can be run with `py.test `_. +The command-line option ``--local-data`` must be defined in order for these tests to work:: + + $ py.test --local-data=../../data/model/database item + ================ test session starts ================ + … + + +R +=== + +Installation +------------ + +Use `devtools `_. +From source (for instance, to develop the code locally):: + + $ git clone git@github.com:transportenergy/database.git + $ Rscript -e "devtools::install_local('database/R/item')" + +Or without cloning the repository:: + + devtools::install_github('transportenergy/database/R/item') + + +Usage +----- + +From R scripts:: + + library(item) + + # Load version 1 of the iTEM models database + data <- item::load_model_data(1) + +From the command-line: ``run`` is an executable that invokes ``item::cli()``. +It can be used without installing the package:: + + $ ./run + Loading item + Usage: ./run [OPTIONS] COMMAND + Command-line interface for the iTEM databases. + + … + + Commands: + mkdirs + debug + load_model_data + + +Development +----------- + +Code conventions and packaging follow the `“R packages” book `_. + +``test`` is an executable that runs the tests in ``R/item/tests/testthat``. +The environment variable ``ITEM_TEST_DATA`` must be defined in order for these tests to work:: + + $ export ITEM_TEST_DATA=../../data/model/database + $ ./test + Loading item + Loading required package: testthat + Testing item + Model database: . + + DONE ============================================ diff --git a/python/run b/python/run deleted file mode 100755 index 996c38ad..00000000 --- a/python/run +++ /dev/null @@ -1,2 +0,0 @@ -#!/bin/sh -python3 -m item $@