Merge pull request #182 from khaeru/feature/backend

Add Backend, JDBCBackend, Model, GAMSModel classes

.gitignore

@@ -53,7 +53,7 @@ rixmp/source/.Rhistory
 
 # pytest and related
 *.pid
-.coverage
+.coverage*
 .pytest_cache/
 coverage.xml
 htmlcov/

RELEASE_NOTES.md

@@ -1,6 +1,7 @@
 
 # Next Release
 
+- [#182](https://github.com/iiasa/ixmp/pull/182): Add new Backend, Model APIs and JDBCBackend, GAMSModel classes.
 - [#188](https://github.com/iiasa/ixmp/pull/188): Enhance reporting.
 - [#177](https://github.com/iiasa/ixmp/pull/177): add ability to pass `gams_args` through `Scenario.solve()`
 - [#175](https://github.com/iiasa/ixmp/pull/175): Drop support for Python 2.

doc/source/api-backend.rst

@@ -0,0 +1,145 @@
+.. currentmodule:: ixmp.backend
+
+Storage back ends (:mod:`ixmp.backend`)
+=======================================
+
+By default, the |ixmp| is installed with :class:`ixmp.backend.jdbc.JDBCBackend`, which can store data in many types of relational database management systems (RDBMS) that have Java DataBase Connector (JDBC) interfaces—hence its name.
+
+However, |ixmp| is extensible to support other methods of storing data: in non-JDBC RDBMS, non-relational databases, local files, memory, or other ways.
+Developers wishing to add such capabilities may subclass :class:`ixmp.backend.base.Backend` and implement its methods.
+
+
+Provided backends
+-----------------
+
+.. automodule:: ixmp.backend
+   :members: BACKENDS
+
+.. currentmodule:: ixmp.backend.jdbc
+
+.. autoclass:: ixmp.backend.jdbc.JDBCBackend
+   :members: read_gdx, write_gdx
+
+   JDBCBackend supports:
+
+   - ``dbtype='HSQLDB'``: HyperSQL databases in local files.
+   - Remote databases. This is accomplished by creating a :class:`ixmp.Platform` with the ``dbprops`` argument pointing a file that specifies JDBC information. For instance::
+
+       jdbc.driver = oracle.jdbc.driver.OracleDriver
+       jdbc.url = jdbc:oracle:thin:@database-server.example.com:1234:SCHEMA
+       jdbc.user = USER
+       jdbc.pwd = PASSWORD
+
+   It has the following methods that are not part of the overall :class:`Backend` API:
+
+   .. autosummary::
+      :nosignatures:
+
+      read_gdx
+      write_gdx
+
+.. automethod:: ixmp.backend.jdbc.start_jvm
+
+Backend API
+-----------
+
+- :class:`ixmp.Platform` implements a *user-friendly* API for scientific programming.
+  This means its methods can take many types of arguments, check, and transform them—in a way that provides modeler-users with easy, intuitive workflows.
+- In contrast, :class:`Backend` has a *very simple* API that accepts arguments and returns values in basic Python data types and structures.
+- As a result:
+
+  - :class:`Platform <ixmp.Platform>` code is not affected by where and how data is stored; it merely handles user arguments and then makes, usually, a single :class:`Backend` call.
+  - :class:`Backend` code does not need to perform argument checking; merely store and retrieve data reliably.
+
+.. autodata:: ixmp.backend.FIELDS
+
+.. currentmodule:: ixmp.backend.base
+
+.. autoclass:: ixmp.backend.base.Backend
+   :members:
+
+   In the following, the bold-face words **required**, **optional**, etc. have specific meanings as described in `IETF RFC 2119 <https://tools.ietf.org/html/rfc2119>`_.
+
+   Backend is an **abstract** class; this means it **must** be subclassed.
+   Most of its methods are decorated with :meth:`abc.abstractmethod`; this means they are **required** and **must** be overridden by subclasses.
+
+   Others, marked below with "OPTIONAL:", are not so decorated.
+   For these methods, the behaviour in the base Backend—often, nothing—is an acceptable default behaviour.
+   Subclasses **may** extend or replace this behaviour as desired, so long as the methods still perform the actions described in the description.
+
+   Backends:
+
+   - **must** only raise standard Python exceptions.
+
+   Methods related to :class:`ixmp.Platform`:
+
+   .. autosummary::
+      :nosignatures:
+
+      close_db
+      get_auth
+      get_nodes
+      get_scenarios
+      get_units
+      open_db
+      set_log_level
+      set_node
+      set_unit
+
+   Methods related to :class:`ixmp.TimeSeries`:
+
+   - Each method has an argument `ts`, a reference to the TimeSeries object being manipulated.
+   - ‘Geodata’ is otherwise identical to regular timeseries data, except value are :class:`str` rather than :class:`float`.
+
+   .. autosummary::
+      :nosignatures:
+
+      check_out
+      commit
+      delete
+      delete_geo
+      discard_changes
+      get
+      get_data
+      get_geo
+      init_ts
+      is_default
+      last_update
+      preload
+      run_id
+      set_data
+      set_as_default
+      set_geo
+
+   Methods related to :class:`ixmp.Scenario`:
+
+   - Each method has an argument `s`, a reference to the Scenario object being manipulated.
+
+   .. autosummary::
+      :nosignatures:
+
+      clone
+      delete_item
+      get_meta
+      has_solution
+      init_s
+      init_item
+      item_delete_elements
+      item_get_elements
+      item_set_elements
+      item_index
+      list_items
+      set_meta
+
+   Methods related to :class:`message_ix.Scenario`:
+
+   - Each method has an argument `ms`, a reference to the Scenario object being manipulated.
+
+   .. warning:: These methods may be moved to ixmp in a future release.
+
+   .. autosummary::
+      :nosignatures:
+
+      cat_get_elements
+      cat_list
+      cat_set_elements

doc/source/api-model.rst

@@ -0,0 +1,36 @@
+.. currentmodule:: ixmp.model
+
+Mathematical models (:mod:`ixmp.model`)
+=======================================
+
+By default, the |ixmp| is installed with :class:`ixmp.model.gams.GAMSModel`, which performs calculations by executing code stored in GAMS files.
+
+However, |ixmp| is extensible to support other methods of performing calculations or optimization.
+Developers wishing to add such capabilities may subclass :class:`ixmp.model.base.Model` and implement its methods.
+
+
+Provided models
+---------------
+
+.. automodule:: ixmp.model
+   :members: get_model, MODELS
+
+.. autoclass:: ixmp.model.gams.GAMSModel
+   :members:
+
+
+Model API
+---------
+
+.. autoclass:: ixmp.model.base.Model
+   :members: name, __init__, run
+
+   In the following, the words REQUIRED, OPTIONAL, etc. have specific meanings as described in `IETF RFC 2119 <https://tools.ietf.org/html/rfc2119>`_.
+
+   Model is an **abstract** class; this means it MUST be subclassed.
+   It has two REQURIED methods that MUST be overridden by subclasses:
+
+   .. autosummary::
+      name
+      __init__
+      run

doc/source/api-python.rst

@@ -3,8 +3,188 @@
 Python (:mod:`ixmp` package)
 ============================
 
-.. automodule:: ixmp
-   :members: Platform, TimeSeries, Scenario
+The |ixmp| application progamming interface (API) is organized around three classes:
+
+.. autosummary::
+
+   Platform
+   TimeSeries
+   Scenario
+
+Platform
+--------
+
+.. autoclass:: Platform
+   :members:
+
+   Platforms have the following methods:
+
+   .. autosummary::
+      add_region
+      add_region_synonym
+      add_unit
+      check_access
+      regions
+      scenario_list
+      set_log_level
+
+
+TimeSeries
+----------
+
+.. autoclass:: TimeSeries
+   :members:
+
+   A TimeSeries is uniquely identified on its :class:`Platform` by three
+   values:
+
+   1. `model`: the name of a model used to perform calculations between input
+      and output data.
+
+      - In TimeSeries storing non-model data, arbitrary strings can be used.
+      - In a :class:`Scenario`, the `model` is a reference to a GAMS program
+        registered to the :class:`Platform` that can be solved with
+        :meth:`Scenario.solve`. See :attr:`ixmp.model.MODELS`.
+
+   2. `scenario`: the name of a specific, coherent description of the real-
+      world system being modeled. Any `model` may be used to represent mutiple
+      alternate, or 'counter-factual', `scenarios`.
+   3. `version`: an integer identifying a specific iteration of a
+      (`model`, `scenario`). A new `version` is created by:
+
+      - Instantiating a new TimeSeries with the same `model` and `scenario` as
+        an existing TimeSeries.
+      - Calling :meth:`Scenario.clone`.
+
+      Optionally, one `version` may be set as a **default version**. See
+      :meth:`set_as_default`.
+
+   TimeSeries objects have the following methods:
+
+   .. autosummary::
+      add_geodata
+      add_timeseries
+      check_out
+      commit
+      discard_changes
+      get_geodata
+      is_default
+      last_update
+      preload_timeseries
+      remove_geodata
+      remove_timeseries
+      run_id
+      set_as_default
+      timeseries
+
+
+Scenario
+--------
+
+.. autoclass:: Scenario
+   :show-inheritance:
+   :members:
+
+   A Scenario is a :class:`TimeSeries` associated with a particular model that
+   can be run on the current :class:`Platform` by calling :meth:`solve`. The
+   Scenario also stores the output, or 'solution' of a model run; this
+   includes the 'level' and 'marginal' values of GAMS equations and variables.
+
+   Data in a Scenario are closely related to different types in the GAMS data
+   model:
+
+   - A **set** is a named collection of labels. See :meth:`init_set`,
+     :meth:`add_set`, and :meth:`set`. There are two types of sets:
+
+     1. Sets that are lists of labels.
+     2. Sets that are 'indexed' by one or more other set(s). For this type of
+        set, each member is an ordered tuple of the labels in the index sets.
+
+   - A **scalar** is a named, single, numerical value. See
+     :meth:`init_scalar`, :meth:`change_scalar`, and :meth:`scalar`.
+
+   - **Parameters**, **variables**, and **equations** are multi-dimensional
+     arrays of values that are indexed by one or more sets (i.e. with
+     dimension 1 or greater). The Scenario methods for handling these types
+     are very similar; they mainly differ in how they are used within GAMS
+     models registered with ixmp:
+
+     - **Parameters** are generic data that can be defined before a model run.
+       They may be altered by the model solution. See :meth:`init_par`,
+       :meth:`remove_par`, :meth:`par_list`, :meth:`add_par`, and :meth:`par`.
+     - **Variables** are calculated during or after a model run by GAMS code,
+       so they cannot be modified by a Scenario. See :meth:`init_var`,
+       :meth:`var_list`, and :meth:`var`.
+     - **Equations** describe fundamental relationships between other types
+       (parameters, variables, and scalars) in a model. They are defined in
+       GAMS code, so cannot be modified by a Scenario. See :meth:`init_equ`,
+       :meth:`equ_list`, and :meth:`equ`.
+
+   .. autosummary::
+      add_par
+      add_set
+      change_scalar
+      clear_cache
+      clone
+      equ
+      equ_list
+      get_meta
+      has_equ
+      has_par
+      has_set
+      has_solution
+      has_var
+      idx_names
+      idx_sets
+      init_equ
+      init_par
+      init_scalar
+      init_set
+      init_var
+      load_scenario_data
+      par
+      par_list
+      remove_par
+      remove_set
+      remove_solution
+      scalar
+      set
+      set_list
+      set_meta
+      solve
+      var
+      var_list
+
+Configuration
+-------------
+
+.. currentmodule:: ixmp.config
+
+.. autoclass:: Config
+
+   When imported, :mod:`ixmp` reads configuration from the first file named
+   ``config.json`` found in one of the following directories:
+
+   1. The directory given by the environment variable ``IXMP_DATA``, if
+      defined,
+   2. ``${XDG_DATA_HOME}/ixmp``, if the environment variable is defined,
+   3. ``$HOME/.local/share/ixmp``, or
+   4. ``$HOME/.local/ixmp`` (deprecated; retained for compatibility with ixmp
+      <= 1.1).
+
+   The file may define either or both of the following configuration keys, in
+   JSON format:
+
+   - `DB_CONFIG_PATH`: location for database properties files. A
+     :class:`ixmp.Platform` instantiated with a relative path name for the
+     `dbprops` argument will locate the file first in the current working
+     directory, then in `DB_CONFIG_PATH`, then in the four directories above.
+   - `DEFAULT_DBPROPS_FILE`: path to a default database properties file.
+     A :class:`ixmp.Platform` instantiated with no arguments will use this
+     file.
+   - `DEFAULT_LOCAL_DB_PATH`: path to a directory where a local directory
+     should be created. A :class:`ixmp.Platform` instantiated with
+     `dbtype='HSQLDB'` will create or reuse a database in this path.
 
 
 Testing utilities