Update README and CLI help

transportenergy · Jul 8, 2019 · 1f1c707 · 1f1c707
1 parent 2b6e361
commit 1f1c707
Show file tree

Hide file tree

Showing 2 changed files with 76 additions and 61 deletions.
diff --git a/README.md b/README.md
@@ -1,11 +1,14 @@
-Tools for accessing & maintaining the iTEM databases
-====================================================
+Tools for the iTEM databases
+============================
+[![Build Status](https://travis-ci.org/transportenergy/database.svg?branch=master)](https://travis-ci.org/transportenergy/database)
 
-[iTEM](http://transportenergy.org) maintains two databases:
+[iTEM](https://transportenergy.org) maintains two databases:
 
-1. A **model database** of transport energy projections assembled as part of the [iTEM Workshops](https://transportenergy.org/workshops/), of which there have been two so far, with a third planned for October 2017. Because of restrictions from workshop participants, the model database is currently *not public*, but only available to iTEM workshop participants.
+1. A **model database** of transport energy projections assembled as part of the [iTEM Workshops](https://transportenergy.org/workshops/).
+   Because of restrictions from workshop participants, the model database is currently *not public*, but only available to iTEM workshop participants.
 
-2. A **statistics database** to form a common, public, “best available” baseline for model calibration and projections. The statistics database is currently *under development*.
+2. A **historical database** to form a common, public, “best available” baseline for model calibration and projections.
+   The historical database is currently *under development*.
 
 This repository contains tools for *both* databases, in *both* Python and R. The goals for these tools are that:
 
@@ -17,43 +20,53 @@ This repository contains tools for *both* databases, in *both* Python and R. The
 Contents
 --------
 
-- `data/` — *Meta*data about the iTEM databases. Actual data is stored separately; see below.
-  - `model/` — Metadata about the iTEM model database. Many of these files are in [YAML format](http://www.yaml.org/spec/1.2/spec.html) ([Wikipedia](https://en.wikipedia.org/wiki/YAML)), with more detailed comments in-line.
-    - `models.yaml` — description of the [iTEM participating models & teams](https://transportenergy.org/participants/).
-    - `dimensions/` — information about the [data dimensions of the model database](https://transportenergy.org/database/).
-    - Other directories:
-      - are named to match the keys in `models.yaml`,
-      - contain model-specific metadata, including but not limited to:
-        - `regions.yaml`: regional aggregation.
-        - `scenarios.yaml`: descriptions of scenarios submitted to each iTEM workshop.
-  - `item_config_example.yaml` — an example configuration file.
 - `python/` — Python code for accessing & maintaining the database. See [`python/README.md`](https://github.com/transportenergy/database/blob/master/python/README.md).
+  - `item/data/` — *Meta*data about the iTEM databases. Actual data is stored separately; see below.
+    - `model/` — Metadata about the iTEM model database. Many of these files are in [YAML format](http://www.yaml.org/spec/1.2/spec.html) ([Wikipedia](https://en.wikipedia.org/wiki/YAML)), with more detailed comments in-line.
+      - `models.yaml` — description of the [iTEM participating models & teams](https://transportenergy.org/participants/).
+      - `dimensions/` — information about the [data dimensions of the model database](https://transportenergy.org/database/).
+      - Other directories:
+        - are named to match the keys in `models.yaml`,
+        - contain model-specific metadata, including but not limited to:
+          - `regions.yaml`: regional aggregation.
+          - `scenarios.yaml`: descriptions of scenarios submitted to each iTEM workshop.
+    - `concepts.yaml`, `measures.yaml`, `spec.yaml` — specifications of data to be submitted for for the 3rd iTEM MIP.
+    - `item_config_example.yaml` — an example configuration file.
 - `R/` — R code for accessing & maintaining the database. See [`R/README.md`](https://github.com/transportenergy/database/blob/master/R/README.md).
 
 
 Usage
 -----
 
-The following instructions are not language-specific. Both the Python and R tools will operate on data stored in the following way; see the language `README`s for specific information.
+The following instructions are not language-specific.
+Both the Python and R tools will operate on data stored in the following way; see the language `README`s for specific information.
 
 1. Clone this repository and/or install the code in one or both languages.
 
-2. Create a separate directory to contain the databases. This directory should have the following structure:
-
-  - `DATA_DIR/` (any directory of your choosing)
-    - `model/`
-      - `raw/` for submitted (‘raw’) data such as:
-        - `MODEL.xlsx`
-        - `MODEL.csv`
-        - `MODEL.gdx`
-      - `processed/` for processed and cleaned information, such as:
-        - `MODEL.csv` — processed and cleaned data.
-        - `MODEL/*` — other metadata derived from the submitted data, such as:
-          - `notes.csv` — notes about the variables.
-          - `regions.csv` — list of regions appearing in the submitted data.
-      - `database/` for the iTEM models database in CSV format.
-    - `stats/`
-
-   Depending on the use-case, some or all of these directories may be left empty. Optionally, also create a CACHE_DIR, anywhere.
-
-3. Copy the file `item_config_example.yaml` to `item_config.yaml` in any working directory where you intend to use this code. Edit the file (see the inline comments) so that the `path/model` key points to `DATA_DIR/model`, and (optionally) the `path/cache` key points to `CACHE_DIR`.
+2. Create 1 or more separate directory to contain the input and output files.
+   A simple way to do this is to call `item mkdirs PATH`, where `PATH` is a base directory; this will create a tree of directories.
+   Use `item mkdirs --help` for more information.
+
+3. Copy the file [`item_config_example.yaml`](https://github.com/transportenergy/database/blob/master/python/item/data/item_config_example.yaml`) to `item_config.yaml` in any working directory where you intend to use this code.
+   Edit the file (see the inline comments) to point to the directories created in #2 above.
+
+4. Use the tools through the command-line interface:
+
+   ```
+   $ item
+   Usage: item [OPTIONS] COMMAND [ARGS]...
+
+     Command-line interface for the iTEM databases.
+
+   Options:
+     --path <KEY> <PATH>  Override data paths (multiple allowed).
+     --help               Show this message and exit.
+
+   Commands:
+     debug     Show debugging information, including paths.
+     help      Show extended help for the command-line tool.
+     mkdirs    Create a directory tree for the database.
+     model     Manipulate the model database.
+     stats     Manipulate the stats database.
+     template  Generate the MIP3 submission template.
+   ```
diff --git a/python/item/cli.py b/python/item/cli.py
@@ -1,29 +1,4 @@
-"""Command-line interface for the iTEM databases.
-
-This tool takes configuration options in one of two ways:
-
-1. From a file named item_config.yaml in the current directory. For
-instance, to override the path to the raw model data, put the
-following in item_config.yaml:
-
-\b
-    path:
-      'model raw': ../custom/data/location
-
-2. From command-line options. For instance, give the following:
-
-       $ ./run --path model_raw ../custom/data/location COMMAND
-
-Underscores are converted to spaces automatically.
-
-In a Python script, the following is equivalent:
-
-\b
-    import item
-    item.init_paths(model_raw='../custom/data/location')
-    …
-
-"""
+"""Command-line interface for the iTEM databases."""
 from textwrap import indent
 
 import click
@@ -47,6 +22,33 @@ def main(paths):
     init_paths(**paths)
 
 
+@main.command()
+def help():
+    """Show extended help for the command-line tool."""
+    print(__doc__, end='\n\n')
+    print("""This tool takes configuration options in one of two ways:
+
+1. From a file named item_config.yaml in the current directory. For
+   instance, to override the path to the raw model data, put the
+   following in item_config.yaml:
+
+   path:
+     'model raw': ../custom/data/location
+
+2. From command-line options. For instance, give the following:
+
+   $ ./run --path model_raw ../custom/data/location COMMAND
+
+Underscores are converted to spaces automatically.
+
+In a Python script, the following is equivalent:
+
+    import item
+    item.init_paths(model_raw='../custom/data/location')
+    …
+""")
+
+
 @main.command()
 def debug():
     """Show debugging information, including paths."""