Merge pull request #761 from iiasa/enh/2023-W46

Rename 'report'

.github/workflows/pytest.yaml

@@ -32,14 +32,15 @@ jobs:
         - "3.9"
         - "3.10"
         - "3.11"  # Latest version supported by message_ix
+        # - "3.12"  # Pending JPype support; see iiasa/ixmp#501
 
         # Below this comment are newly released or development versions of
         # Python. For these versions, binary wheels are not available for some
         # dependencies, e.g. llvmlite, numba, numpy, and/or pandas. Compiling
         # these on the job runner requires a more elaborate build environment,
         # currently out of scope for the message_ix project.
 
-        # - "3.12.0-alpha.1"  # Development version
+        # - "3.13.0-alpha.1"  # Development version
 
       fail-fast: false
 
@@ -121,15 +122,6 @@ jobs:
           --numprocesses=auto --dist=loadgroup || ( [ $? -eq 127 -a "${{ runner.os }}" = "Windows" ] && echo "Spurious exit code 127 from pytest" && exit 0)
       shell: bash
 
-    - name: Test documentation build using Sphinx
-      if: ${{ startsWith(matrix.os, 'ubuntu') }}
-      env:
-        # For pull_request triggers, GitHub creates a temporary merge commit
-        # with a hash that does not match the head of the branch. Tell it which
-        # branch to use.
-        SPHINXOPTS: -D linkcode_github_remote_head=${{ github.head_ref }}
-      run: make --directory=doc html
-
     - name: Upload test coverage to Codecov.io
       uses: codecov/codecov-action@v3
 
@@ -141,6 +133,8 @@ jobs:
     steps:
     - uses: actions/checkout@v3
     - uses: actions/setup-python@v4
+      with:  # Same as the "Latest version supported by message_ix", above
+        python-version: "3.11"
 
     - name: Force recreation of pre-commit virtual environment for mypy
       if: github.event_name == 'schedule'  # Comment this line to run on a PR

.gitignore

@@ -1,4 +1,5 @@
 # Python, R, etc.
+__pycache__
 *-checkpoint.ipynb
 *.bak
 *.dat
@@ -38,11 +39,12 @@ dist
 .cache/**
 tests/data/nightly
 
-# pytest and related
+# Python tooling, pytest, etc.
 .benchmarks
 .coverage*
 .mypy_cache
 .pytest_cache
+.ruff_cache
 coverage.xml
 htmlcov
 prof/

INSTALL.rst

@@ -52,8 +52,8 @@ GAMS (required)
 Graphviz (optional)
 -------------------
 
-:meth:`.reporting.Reporter.visualize` uses `Graphviz`_, a program for graph visualization.
-Installing message_ix causes the python :mod:`graphviz` package to be installed.
+:meth:`.Reporter.visualize` uses `Graphviz`_, a program for graph visualization.
+Installing message_ix causes the `graphviz <https://graphviz.readthedocs.io>`__ Python package to be installed.
 If you want to use :meth:`.visualize` or run the test suite, the Graphviz program itself must also be installed; otherwise it is **optional**.
 
 If you install MESSAGEix `Using Anaconda`_, Graphviz is installed automatically via `its conda-forge package`_.
@@ -91,7 +91,7 @@ Using Anaconda
 7. Install and configure the `mamba solver`_, which is faster and more reliable than conda's default solver::
 
     $ conda install conda-libmamba-solver
-    $ conda config --set solver libmamba 
+    $ conda config --set solver libmamba
 
 8. Create a new conda environment and activate it.
    This step is **required** if using Anaconda, but *optional* if using Miniconda.
@@ -150,7 +150,7 @@ From source
     # or:
     $ git clone [email protected]:USER/message_ix.git
 
-7. (Conditional) If you cloned your fork, add the main repository as a remote git repository. 
+7. (Conditional) If you cloned your fork, add the main repository as a remote git repository.
    This will allow keeping up to date with changes there and importing tags, which also needs to be done for the install tests to succeed::
 
     $ git remote add upstream [email protected]:iiasa/message_ix.git
@@ -235,7 +235,7 @@ Common issues
 “No JVM shared library file (jvm.dll) found”
 --------------------------------------------
 
-Error messages like this when running ``message-ix --platform=default list`` or when creating a :class:`.Platform` object (e.g. :code:`ixmp.Platform()` in Python) indicate that :mod:`message_ix` (via :mod:`ixmp` and JPype) cannot find Java on your machine, in particular the Java Virtual Machine (JVM).
+Error messages like this when running ``message-ix --platform=default list`` or when creating a :class:`ixmp.Platform` object (for instance, :py:`ixmp.Platform()` in Python) indicate that :mod:`message_ix` (via :mod:`ixmp` and JPype) cannot find Java on your machine, in particular the Java Virtual Machine (JVM).
 There are multiple ways to resolve this issue:
 
 1. If you have installed Java manually, ensure that the ``JAVA_HOME`` environment variable is set system-wide; see for example `these instructions`_ for Windows users.
@@ -250,7 +250,7 @@ To check which JVM will be used by ixmp, run the following in any prompt or term
 “No module named 'pyam'”
 ------------------------
 
-The package `pyam-iamc <https://pypi.org/project/pyam-iamc/>`_ is one of the "reporting" extra dependencies of :mod:`message_ix`.
+The package `pyam-iamc <https://pypi.org/project/pyam-iamc/>`_ is one of the "report" extra dependencies of :mod:`message_ix`.
 These extra dependencies are not installed automatically, but can be installed using::
 
     # If message_ix is installed using pip

RELEASE_NOTES.rst

@@ -1,16 +1,27 @@
 Next release
 ============
 
+Migration notes
+---------------
+Update code that imports from the following modules:
+
+- :py:`message_ix.reporting` → use :py:`message_ix.report`.
+- :py:`message_ix.reporting.computations` → use :py:`message_ix.report.operator`.
+
+Code that imports from the old locations will continue to work, but will raise :class:`DeprecationWarning`.
+
 All changes
 -----------
 
 - Drop support for Python 3.7, which `reached end-of-life on 2023-06-27 <https://peps.python.org/pep-0537/#lifespan>`__ (:pull:`738`).
   :mod:`message_ix` now requires Python version 3.8 or greater.
+- Rename :mod:`message_ix.report` (:pull:`761`).
 - New reporting operator :func:`.model_periods` and automatic keys ``y::model`` and ``y0`` (:pull:`738`).
 - Improve readability of LaTeX equations in docs (:pull:`721`).
+- Replace :py:`MESSAGE_ITEMS` and :py:`MACRO_ITEMS` with :attr:`.MESSAGE.items` and :attr:`.MACRO.items`, respectively (:pull:`761`).
 - Bugfix: :meth:`.Scenario.add_macro` would not correctly handle configuration that mapped a MESSAGE (commodity, level) to MACRO sector when the commodity and sector names were different (:pull:`719`).
 - Expand :doc:`macro` documentation, particularly code documentation (:issue:`315`, :pull:`719`).
-- Bugfix: :func:`.computations.as_message_df` would error if a particular dimension was supplied via the `common` argument but not present in `qty` (:pull:`719`).
+- Bugfix: :func:`.operator.as_message_df` would error if a particular dimension was supplied via the `common` argument but not present in `qty` (:pull:`719`).
 
 .. _v3.7.0:
 
@@ -85,7 +96,7 @@ Migration notes
 
      df = scen.vintage_and_active_years().query(f"{scen.y0} <= year_vtg")
 
-- The :ref:`default reports <default-reports>` (tables in IAMC format) available in a :class:`~message_ix.reporting.Reporter` have changed keys to e.g. ``message::default`` with **two** colons.
+- The :ref:`default reports <default-reports>` (tables in IAMC format) available in a :class:`.Reporter` have changed keys to e.g. ``message::default`` with **two** colons.
   Code using e.g. ``message:default`` (one colon) should be updated to use the current keys.
 
   This matches fixed behaviour upstream in :mod:`genno` version 1.12 to avoid unintended confusion with keys like ``A:i``: ``i`` (after the first colon) is the name for the sole dimension of a 1-dimensional quantity, whereas ``default`` in ``message::default`` is a tag.
@@ -173,7 +184,7 @@ All changes
 - Expand documentation :doc:`install` for installing GAMS under macOS (:pull:`460`).
 - Add new Westeros :doc:`tutorial <tutorials>` on add-on technologies (:pull:`365`).
 - Expand documentation of :ref:`dynamic constraint parameters <section_parameter_dynamic_constraints>` (:pull:`454`).
-- Adjust :mod:`message_ix.reporting` to use :mod:`genno` / :mod:`ixmp.reporting` changes in `ixmp PR #397 <https://github.com/iiasa/ixmp/pull/397>`_ (:pull:`441`).
+- Adjust :mod:`message_ix.report` to use :mod:`genno` / :mod:`ixmp.report` changes in `ixmp PR #397 <https://github.com/iiasa/ixmp/pull/397>`_ (:pull:`441`).
 
 
 v3.2.0 (2021-01-24)
@@ -194,10 +205,10 @@ Migration notes
 All changes
 -----------
 
-- :pull:`407`: Use :mod:`.reporting` in tutorials; add :mod:`.util.tutorial` for shorthand code used to streamline tutorials.
+- :pull:`407`: Use :mod:`.report` in tutorials; add :mod:`.util.tutorial` for shorthand code used to streamline tutorials.
 - :pull:`407`: Make :class:`.Reporter` a top-level class.
 - :pull:`415`: Improve :func:`.make_df` to generate empty, partially-, or fully-filled data frames with the correct columns for any MESSAGE or MACRO parameter.
-- :pull:`415`: Make complete lists of :data:`.MESSAGE_ITEMS`, :data:`.MACRO_ITEMS` and their dimensions accessible through the Python API.
+- :pull:`415`: Make complete lists of :attr:`.MESSAGE_ITEMS <.MESSAGE.items>`, :attr:`.MACRO_ITEMS <.MACRO.items>` and their dimensions accessible through the Python API.
 - :pull:`421`: Fix discounting from forward-looking to backward-looking and provide an explanation of the period structure and discounting in documentation of :doc:`time`.
 
 
@@ -242,7 +253,7 @@ All changes
 - :pull:`190`: Add generic mathematical formulation of :ref:`technologies that store commodities <gams-storage>`, such as water and energy.
 - :pull:`343`, :pull:`345`: Accept :class:`.MESSAGE_MACRO` iteration control parameters through :meth:`.solve`; document how to tune these to avoid numerical issues.
 - :pull:`340`: Allow cplex.opt to be used by `message_ix` from multiple processes.
-- :pull:`328`: Expand automatic reporting of emissions prices and mapping sets; improve robustness of :meth:`Reporter.convert_pyam`.
+- :pull:`328`: Expand automatic reporting of emissions prices and mapping sets; improve robustness of :func:`.Reporter.convert_pyam <genno.compat.pyam.operator.add_as_pyam>`.
 - :pull:`321`: Move :meth:`.Scenario.to_excel`, :meth:`.read_excel` to :class:`ixmp.Scenario`; they continue to work with :class:`message_ix.Scenario`.
 - :pull:`323`: Add `units`, `replace_vars` arguments to :meth:`.Reporter.convert_pyam`.
 - :pull:`308`: Expand automatic reporting of add-on technologies.
@@ -314,7 +325,7 @@ All changes
 - :pull:`187`: Test for cumulative bound on emissions.
 - :pull:`182`: Fix cross-platform cloning.
 - :pull:`178`: Bugfix of the ``PRICE_EMISSION`` variable in models with non-equidistant period durations.
-- :pull:`176`: Add :mod:`message_ix.reporting` module.
+- :pull:`176`: Add :mod:`message_ix.report` module.
 - :pull:`173`: The meth:`~.Scenario.solve` command now takes additional arguments when solving with CPLEX. The cplex.opt file is now generated on the fly during the solve command and removed after successfully solving.
 - :pull:`172`: Add option to set ``COMMODITY_BALANCE`` to equality.
 - :pull:`154`: Enable documentation build on ReadTheDocs.

doc/api.rst

@@ -31,10 +31,12 @@ The full API is also available from R; see :doc:`rmessageix`.
 
 .. currentmodule:: message_ix
 
+.. automodule:: message_ix
+
 ``message_ix`` package
 ----------------------
 
-|MESSAGEix| models are created using the :py:class:`message_ix.Scenario` class. Several utility methods are also provided in the module :py:mod:`message_ix.utils`.
+|MESSAGEix| models are created using the :py:class:`message_ix.Scenario` class. Several utility methods are also provided in the module :py:mod:`message_ix.util`.
 
 .. autoclass:: message_ix.Scenario
    :members:
@@ -90,8 +92,8 @@ Model classes
    MESSAGE_MACRO
    GAMSModel
    DEFAULT_CPLEX_OPTIONS
-   MESSAGE_ITEMS
-   ~message_ix.macro.MACRO_ITEMS
+   Item
+   ItemType
 
 .. autodata:: DEFAULT_CPLEX_OPTIONS
 
@@ -195,12 +197,23 @@ Model classes
    :exclude-members: defaults
    :show-inheritance:
 
+   .. autoattribute:: items
+      :no-value:
+
+      Keys are the names of items (sets, parameters, variables, and equations); values are :class:`.Item` instances.
+      These include all items listed in the MESSAGE mathematical specification, i.e. :ref:`sets_maps_def` and :ref:`parameter_def`.
+
 .. autoclass:: MACRO
    :members:
+   :exclude-members: items
    :show-inheritance:
 
+   .. autoattribute:: items
+      :no-value:
+
 .. autoclass:: MESSAGE_MACRO
    :members:
+   :exclude-members: items
    :show-inheritance:
 
    MESSAGE_MACRO solves the MESSAGE and MACRO models iteratively, connecting changes in technology activity and resource demands (from MESSAGE) to changes in final demands and prices (from MACRO).
@@ -218,22 +231,15 @@ Model classes
 
    .. seealso:: :meth:`.Scenario.add_macro`
 
-.. autodata:: MESSAGE_ITEMS
-   :annotation: = dict(…)
-
-   Keys are the names of items (sets, parameters, variables, and equations); values are :class:`dict` specifying their type and dimensionality, with keys 'ix_type', 'idx_sets', and in some cases 'idx_names'.
-   These include all items listed in the MESSAGE mathematical specification, i.e. :ref:`sets_maps_def` and :ref:`parameter_def`.
+   .. autoattribute:: items
+      :no-value:
 
-   .. seealso:: :meth:`.MESSAGE.initialize`, :data:`.MACRO_ITEMS`
+.. autodata:: DIMS
+.. autoclass:: Item
+   :members:
 
 .. currentmodule:: message_ix.macro
 
-.. autodata:: MACRO_ITEMS
-   :annotation: = dict(…)
-
-   .. seealso:: :data:`.MESSAGE_ITEMS`
-
-
 .. _utils:
 
 Utility methods
@@ -248,3 +254,6 @@ Testing utilities
 
 .. automodule:: message_ix.testing
    :members: make_dantzig, make_westeros
+
+.. automodule:: message_ix.testing.nightly
+   :members: