Merge pull request #363 from khaeru/issue/196

Update documentation for v3.2

.github/PULL_REQUEST_TEMPLATE.md

@@ -30,12 +30,12 @@ the user behaviour, e.g. new features or fixes to existing behaviour. They are
 
 In ambiguous cases, strike them out and add a short explanation, e.g.
 
-- [x] ~Tests added.~ No change in behaviour, simply refactoring.
+- ~Add or expand tests.~ No change in behaviour, simply refactoring.
 -->
 
-- [ ] Tests added.
-- [ ] Documentation added.
-- [ ] Release notes updated.
+- [ ] Add or expand tests.
+- [ ] Add, expand, or update documentation.
+- [ ] Update release notes.
   <!-- Add a single line at the top of the “Next release” section of
        RELEASE_NOTES.rst, where '999' is the GitHub pull request number:
 

RELEASE_NOTES.rst

@@ -4,6 +4,7 @@ Next release
 All changes
 -----------
 
+- :pull:`363`: Expand documentation and revise installation instructions.
 - :pull:`362`: Raise Python exceptions from :class:`.JDBCBackend`.
 - :pull:`354`: Add :meth:`Scenario.items`, :func:`.utils.diff`, and allow using filters in CLI command ``ixmp export``.
 - :pull:`353`: Add meta functionality.

doc/source/api-backend.rst

@@ -25,10 +25,19 @@ Provided backends
 .. autoclass:: ixmp.backend.jdbc.JDBCBackend
    :members: read_file, write_file
 
-   JDBCBackend supports:
+   JDBCBackend **supports**:
 
    - Databases in local files (HyperSQL) using ``driver='hsqldb'`` and the *path* argument.
    - Remote, Oracle databases using ``driver='oracle'`` and the *url*, *username* and *password* arguments.
+   - Temporary, in-memory databases using ``driver='hsqldb'`` and the *url* argument.
+     Use the `url` parameter with the format ``jdbc:hsqldb:mem:[NAME]``, where [NAME] is any string::
+
+       mp = ixmp.Platform(
+           backend="jdbc",
+           driver="hsqldb",
+           url="jdbc:hsqldb:mem:temporary platform",
+       )
+
 
    JDBCBackend caches values in memory to improve performance when repeatedly reading data from the same items with :meth:`.par`, :meth:`.equ`, or :meth:`.var`.
 
@@ -41,10 +50,17 @@ Provided backends
 
    .. tip:: Modifying an item by adding or deleting elements invalidates its cache.
 
-   JDBCBackend has the following limitations:
+   JDBCBackend has the following **limitations**:
 
    - The `comment` argument to :meth:`Platform.add_unit` is limited to 64 characters.
 
+   JDBCBackend's implementation allows the following kinds of file input and output:
+
+   .. autosummary::
+
+      read_file
+      write_file
+
 .. automethod:: ixmp.backend.jdbc.start_jvm
 
 Backend API

doc/source/conf.py

@@ -99,6 +99,7 @@
 intersphinx_mapping = {
     'dask': ('https://docs.dask.org/en/stable/', None),
     'jpype': ('https://jpype.readthedocs.io/en/latest', None),
+    'message_ix': ('https://docs.messageix.org/en/latest/', None),
     'pandas': ('https://pandas.pydata.org/pandas-docs/stable/', None),
     'pint': ('https://pint.readthedocs.io/en/stable/', None),
     'python': ('https://docs.python.org/3/', None),

doc/source/install.rst

@@ -1,22 +1,31 @@
 Installation
 ************
 
-A desktop or laptop computer is sufficient for most purposes using :mod:`ixmp`.
 Most users will have :mod:`ixmp` installed automatically when `installing MESSAGEix`_.
+The sections below cover other use cases.
 
-The sections below cover other use cases:
+Ensure you have first read the :doc:`prerequisites <prereqs>` for understanding and using |MESSAGEix|.
+These include specific points of knowledge that are necessary to understand these instructions and choose among different installation options.
 
-- Installing *ixmp* to be used alone (i.e., with models or frameworks other than |MESSAGEix|):
+Use cases for installing ixmp directly include:
 
-  - see the sections `Install system dependencies`_,
-  - then `Install ixmp via Anaconda`_.
+- Installing *ixmp* to be used alone (i.e., with models or frameworks other than |MESSAGEix|).
+  Follow the sections:
 
-- Installing *ixmp* from source, for development purposes: see `Install ixmp from source`_.
+  - `Install system dependencies`_, then
+  - `Using Anaconda`_.
 
-- Installing the R API to *ixmp*:
+- Installing *ixmp* from source, for development purposes, e.g. to be used with a source install of :mod:`message_ix`.
+  Follow the sections:
+
+  - `Install system dependencies`_, then
+  - `From source`_.
+
+- Installing the :doc:`rixmp <api-r>` R package, to use the :mod:`ixmp`, alone, from R.
+  Again, to use |MESSAGEix| from R, it is sufficient to install ``rmessageix``; and not necessary to also/separately install ``rximp``.
 
   - Start with `Install system dependencies`_.
-  - Then install *ixmp* either via Anaconda, or from source.
+  - Then install :mod:`ixmp` from source.
   - Finally, see `Install rixmp`_.
 
 **Contents:**
@@ -27,12 +36,12 @@ The sections below cover other use cases:
 Install system dependencies
 ===========================
 
-GAMS
-----
+GAMS (required)
+---------------
 
 :mod:`ixmp` requires `GAMS`_.
 
-1. Download GAMS for your operating system; either the `latest version`_ or `version 29`_ (see note below).
+1. Download GAMS for your operating system; either the `latest version`_ or, for users not familiar with GAMS licenses, `version 29`_ (see note below).
 
 2. Run the installer.
 
@@ -59,40 +68,62 @@ GAMS
    If you only have a license for an older version of GAMS, install both the older and the latest versions.
 
 
-Graphviz
---------
+Graphviz (optional)
+-------------------
 
 :meth:`ixmp.reporting.Reporter.visualize` uses `Graphviz`_, a program for graph visualization.
 Installing ixmp causes the python :mod:`graphviz` package to be installed.
 If you want to use :meth:`.visualize` or run the test suite, the Graphviz program itself must also be installed; otherwise it is **optional**.
 
-If you `Install ixmp via Anaconda`_, Graphviz is installed automatically via `its conda-forge package`_.
+If you install :mod:`ixmp` using Anaconda, Graphviz is installed automatically via `its conda-forge package`_.
 For other methods of installation, see the `Graphviz download page`_ for downloads and instructions for your system.
 
 
-Install ``ixmp`` via Anaconda
-=============================
+Install :mod:`ixmp`
+===================
+
+Using Anaconda
+--------------
 
 After installing GAMS, we recommend that new users install Anaconda, and then use it to install :mod:`ixmp`.
 Advanced users may choose to install :mod:`ixmp` from source code (next section).
 
-4. Install Python via `Anaconda`_.
+4. Install Python via either `Miniconda`_ or `Anaconda`_. [1]_
    We recommend the latest version; currently Python 3.8.
 
 5. Open a command prompt.
-   We recommend Windows users use the “Anaconda Prompt” to avoid permissions issues when installing and using :mod:`ixmp`.
+   We recommend Windows users use the “Anaconda Prompt” to avoid issues with permissions and environment variables when installing and using :mod:`ixmp`.
    This program is available in the Windows Start menu after installing Anaconda.
 
-6. Install the ``ixmp`` package::
+6. Configure conda to install :mod:`ixmp` from the conda-forge channel [2]_::
+
+    $ conda config --prepend channels conda-forge
+
+7. Create a new conda enviroment.
+   This step is **required** if using Anaconda, but *optional* if using Miniconda.
+   This example uses the name ``ixmp_env``, but you can use any name of your choice::
+
+    $ conda create --name ixmp_env
+    $ conda activate ixmp_env
+
+6. Install the ``ixmp`` package into the current environment (either ``base``, or another name from step 7, e.g. ``ixmp_env``)::
 
     $ conda install -c conda-forge ixmp
 
+.. [1] See the `conda glossary`_ for the differences between Anaconda and Miniconda, and the definitions of the terms ‘channel’ and ‘environment’ here.
+.. [2] The ‘$’ character at the start of these lines indicates that the command text should be entered in the terminal or prompt, depending on the operating system.
+       Do not retype the ‘$’ character itself.
+
+.. note:: When using Anaconda (not Miniconda), steps (5) through (8) can also be performed using the graphical Anaconda Navigator.
+   See the `Anaconda Navigator documentation`_ for how to perform the various steps.
 
-Install ``ixmp`` from source
-============================
+
+From source
+-----------
 
 4. (Optional) If you intend to contribute changes to *ixmp*, first register a Github account, and fork the `ixmp repository <https://github.com/iiasa/ixmp>`_.
    This will create a new repository ``<user>/ixmp``.
+   (Please also see :message_ix:doc:`contributing`.)
 
 5. Clone either the main repository, or your fork; using the `Github Desktop`_ client, or the command line::
 
@@ -113,83 +144,96 @@ Install ``ixmp`` from source
     $ pytest
 
 
-Install development tools
--------------------------
+Install ``rixmp``
+=================
 
-Developers making changes to the :mod:`ixmp` source **may** need one or more of the following tools.
-Users developing models using existing functionality **should not** need these tools.
+``rixmp`` is the R interface to :mod:`ixmp`; see :doc:`its documentaiton <api-r>`.
+You only need to install ``rixmp`` if you intend to use :mod:`ixmp` from R, rather than from Python.
 
-Git
-   Use one of:
+Install :mod:`ixmp` **from source**, per the section above.
+Then:
 
-   - https://git-scm.com/downloads
-   - https://desktop.github.com
-   - https://www.gitkraken.com
+8. `Install R <https://www.r-project.org>`_.
+   Ensure that your ``PATH`` environment variable is configured correctly so that the ``Rscript`` executable is available.
 
-   In addition, set up an account at https://github.com, and familiarize yourself with forking and cloning repositories, as well as pulling, committing and pushing changes.
+   .. warning::
+      Ensure the the R version installed is either 32- *or* 64-bit (and >= 3.5.0), consistently with GAMS and Java.
+      Having both 32- and 64-bit versions of R, or mixed 32- and 64-bit versions of different packages, can cause errors.
 
-Java Development Kit (JDK)
-   - Install the Java Development Kit (JDK) for Java SE version 8 from https://www.oracle.com/technetwork/java/javase/downloads/index.html
+9. Open a command prompt in the :file:`ixmp/` directory.
+   Type the following commands to build, then install, ``rixmp`` and its dependencies, including reticulate_::
 
-     .. note:: At this point, ixmp is not compatible with JAVA SE 9.
+    $ R CMD build rixmp
 
-   - Follow the `JDK website instructions <https://docs.oracle.com/cd/E19182-01/820-7851/inst_cli_jdk_javahome_t/>`_ to set the ``JAVA_HOME`` environment variable; if ``JAVA_HOME`` does not exist, add it as a new system variable.
+10. Check that there is only one :file:`*.tar.gz` or :file:`.zip` file in the folder, then run::
 
-   - Update your ``PATH`` environment variable to point to the JRE binaries and server installation (e.g., :file:`C:\\Program Files\\Java\\jdk[YOUR JDK VERSION]\\jre\\bin\\`, :file:`C:\\Program Files\\Java\\jdk[YOUR JDK VERSION]\\jre\\bin\\server`).
+      # On Windows
+      $ R CMD INSTALL rixmp_*.zip
 
-     .. warning:: Do not overwrite the existing ``PATH`` environment variable, but add to the list of existing paths.
+      # Other operating systems
+      $ R CMD INSTALL rixmp_*.tar.gz
 
+11. (Optional) Install `IRKernel`_, which allows running R code in Jupyter notebooks (see the link for instructions).
 
+12. (Optional) Check that the R interface works by using the built-in test suite to run the R tutorial notebooks::
 
-Install ``rixmp``
-=================
+    $ pytest -m rixmp
 
-See also the :ref:`rixmp documentation <rixmp>`.
 
-1. `Install R <https://www.r-project.org>`_.
+Troubleshooting
+===============
+
+Run ``ixmp show-versions`` on the command line to check that you have all dependencies installed, or when reporting issues.
 
-   .. warning::
-      Ensure the the R version installed is either 32- *or* 64-bit (and >= 3.5.0), consistently with GAMS and Java.
-      Having both 32- and 64-bit versions of R, or mixed 32- and 64-bit versions of different packages, can cause errors.
+For Anaconda users experiencing problems during installation of ixmp, check that the following paths are part of the ``PATH`` environment variable, and add them if missing::
 
-2. Enter the directory ``rixmp/`` and use R to build and install the package and its dependencies, including reticulate_::
+    C:\[YOUR ANACONDA LOCATION]\Anaconda3;
+    C:\[YOUR ANACONDA LOCATION]\Anaconda3\Scripts;
+    C:\[YOUR ANACONDA LOCATION]\Anaconda3\Library\bin;
 
-    $ cd rixmp
-    $ Rscript -e "install.packages(c('knitr', 'reticulate'), repos='http://cran.rstudio.com/')"
-    $ R CMD build .
 
-3. Check that there is only one ``*tar.gz`` in the folder, then run::
+Install development tools
+=========================
 
-    $ R CMD INSTALL rixmp_*
+Developers making changes to the :mod:`ixmp` source **may** need one or more of the following tools.
+Users developing models using existing functionality **should not** need these tools.
 
-4. (Optional) Run the built-in test suite to check that *ixmp* and *rixmp* functions, as in *Install ixmp from source 6.* (installing
-   the R ``devtools`` package might be a pre-requisite). In the ``ixmp`` directory type::
+Git
+   Use one of:
 
-    $ pytest -m rixmp
+   - https://git-scm.com/downloads
+   - https://desktop.github.com
+   - https://www.gitkraken.com
 
-5. (Optional) To use rixmp in Jupyter notebooks, install the `IR kernel <https://irkernel.github.io>`_.
+Java Development Kit (JDK)
+   - Install the Java Development Kit (JDK) for Java SE version 8 from https://www.oracle.com/technetwork/java/javase/downloads/index.html
 
-6. (Optional) Install `Rtools <https://cran.r-project.org/bin/windows/Rtools/>`_ and add the path to the environment variables.
+     .. note:: At this point, ixmp is not compatible with JAVA SE 9.
 
-.. _reticulate: https://rstudio.github.io/reticulate/
+   - Follow the `JDK website instructions`_ to set the ``JAVA_HOME`` environment variable; if ``JAVA_HOME`` does not exist, add it as a new system variable.
 
+   - Update your ``PATH`` environment variable to point to the JRE binaries and server installation (e.g., :file:`C:\\Program Files\\Java\\jdk[YOUR JDK VERSION]\\jre\\bin\\`, :file:`C:\\Program Files\\Java\\jdk[YOUR JDK VERSION]\\jre\\bin\\server`).
 
-Troubleshooting
-===============
+     .. warning:: Do not overwrite the existing ``PATH`` environment variable, but add to the list of existing paths.
 
-Run ``ixmp show-versions`` on the command line to check that you have all dependencies installed, or when reporting issues.
+Rtools
+   https://cran.r-project.org/bin/windows/Rtools/
 
-For Anaconda users experiencing problems during installation of ixmp, check that the following paths are part of the ``PATH`` environment variable, and add them if missing::
+   For installing or modifying some R packages on Windows.
 
-    C:\[YOUR ANACONDA LOCATION]\Anaconda3;
-    C:\[YOUR ANACONDA LOCATION]\Anaconda3\Scripts;
-    C:\[YOUR ANACONDA LOCATION]\Anaconda3\Library\bin;
 
 .. _`installing MESSAGEix`: https://docs.messageix.org/en/latest/getting_started.html
 .. _`Anaconda`: https://www.continuum.io/downloads
 .. _`GAMS`: http://www.gams.com
 .. _`latest version`: https://www.gams.com/download/
 .. _`version 29`: https://www.gams.com/29/
+.. _Graphviz: https://www.graphviz.org
 .. _`its conda-forge package`: https://anaconda.org/conda-forge/graphviz
-.. _`Graphviz download page`: https://www.graphviz.org/download/
+.. _Graphviz download page: https://www.graphviz.org/download/
+.. _Miniconda: https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html
+.. _conda glossary: https://docs.conda.io/projects/conda/en/latest/glossary.html
+.. _Anaconda Navigator documentation: https://docs.anaconda.com/anaconda/navigator/
 .. _`Github Desktop`: https://desktop.github.com
+.. _reticulate: https://rstudio.github.io/reticulate/
+.. _IRkernel: https://irkernel.github.io/installation/
+.. _JDK website instructions: https://docs.oracle.com/cd/E19182-01/820-7851/inst_cli_jdk_javahome_t/