Code

Migrate message_data.reporting#116

Code

Merged

khaeru merged 220 commits intomainiiasa/message-ix-models:mainfrom migrate-reportiiasa/message-ix-models:migrate-reportCopy head branch name to clipboardAug 31, 2023

+2,667-15

Commits

Commits on Aug 31, 2023

• Move files into message_data; new __init__.py

  khaeru

  committed
  6474c03

• Merge branch 'master' into reporting3

  khaeru

  committed
  4c6cec2

• Update documentation

  khaeru

  committed
  009a76d

• Simplify model.create

  khaeru

  committed
  c8a0f0b

• Fix reporter.as_pyam call to reporter.convert_pyam (#77)

  khaeru

  committed
  3e1b888

• Move reporting config file to repo-wide metadata directory

  khaeru

  committed
  321b0e2

• Add CLI option for selecting reporting config file

  khaeru

  committed
  a0bee6c

• Adjust docs to reporting config file location

  khaeru

  committed
  b160cd0

• Adjust IAMC conversion

  khaeru

  committed
  0cb7f50

• Corrected context scenario attributes used to retrieve scenario model and scenario name

  

  OFR-IIASA

  authored and

  khaeru

  committed
  e19044b

• Reorganize reporting config

  khaeru

  committed
  e3f613a

• Tweak handling of URL CLI argument

  khaeru

  committed
  14ccd5b

• Fixed test for technologies.py

  

  francescolovat

  authored and

  khaeru

  committed
  0291845

• Added technology.yaml to the Default reporting configuration section of docu

  

  francescolovat

  authored and

  khaeru

  committed
  620aefd

• Adjust inclusion of technology.yaml in tools docs

  khaeru

  committed
  ae3ca8c

• Allow empty combine/inputs/select in reporting config YAML

  Show description for 32c4e42
  khaeru

  committed
  32c4e42

• Specify convert_pyam(year_time_dim=...) in add_iamc_table

  Show description for 7c4df12
  khaeru

  committed
  7c4df12

• Handle varying dimensionality in reporting.computations.combine; test

  khaeru

  committed
  377411f

• Make combine/inputs/weight default to 1; expand docs

  khaeru

  committed
  45024c5

• Document more reporting.core methods; use infer_keys in add_aggregate

  khaeru

  committed
  e52c501

• Add test_report_bare_res

  khaeru

  committed
  2aa3f9e

• Streamline docs configuration per sphinx-quickstart 2.3.1

  khaeru

  committed
  f6f20e8

• Adjust imports from ixmp.reporting

  khaeru

  committed
  151e353

• Comment definition of 'USD' unit in data/report/global.yaml

  khaeru

  committed
  8d244ef

• Test applying units to reported quantities (#104)

  Show description for 502cfbb
  khaeru

  committed
  502cfbb

• Add unit, replace_vars args for reporting IAMC conversion

  khaeru

  committed
  b7e7451

• Harmonize docstrings in reporting.core

  khaeru

  committed
  89f9ff6

• Test IAMC units conversion

  khaeru

  committed
  86456c2

• Test IAMC variable name replacement

  khaeru

  committed
  546e368

• Pass units through reporting.computations.combine

  khaeru

  committed
  d17705f

• Add common replacements to reporting.util.collapse()

  khaeru

  committed
  0e98e93

• Simplify config handling & passing config to ixmp

  khaeru

  committed
  94812e6

• Tidy reporting.core imports; re-add deepcopy for testing

  khaeru

  committed
  f8fbe58

• Report GDP variables (#110)

  Show description for be6bc0b
  

  francescolovat

  and

  khaeru

  committed
  be6bc0b

• Use improvements from iiasa/ixmp#312

  khaeru

  committed
  b337244

• Copy cleanups from iiasa/message_data#116

  khaeru

  committed
  baf95c4

• Bump Reporter.add_queue(..., max_tries=2) from 1

  khaeru

  committed
  bf97f24

• Don't re-set pint application registry

  khaeru

  committed
  4283222

• Update tests for iam-units registry

  khaeru

  committed
  c2cf559

• Add reporting.computations.update_scenario

  khaeru

  committed
  1e84fbb

• Report multiple scenarios with 'mix-data report --from-file=...'

  khaeru

  committed
  74c4ef7

• Document continuous reporting

  khaeru

  committed
  5d27d20

• Cast columns to 'str' in reporting.util.collapse

  khaeru

  committed
  a07cde9

• Add model.transport.check

  khaeru

  committed
  2e09349

• Add reporting.register() for callbacks; expand transport.report

  khaeru

  committed
  d7bcf29

• Adjust tests to use session_context where possible

  khaeru

  committed
  6b7ab89

• Adjust model.bare.create_res and other code to use set_info()

  khaeru

  committed
  257a009

• Update docs for model.transport, model.disutility; expand docs ToC

  khaeru

  committed
  fc4a76b

• Adjust imports from ixmp.reporting

  khaeru

  committed
  bfbc41f

• Adjust reporting imports

  khaeru

  committed
  d425a9a

• Separate CLIs into consistent .cli submodules

  Show description for 6e8a774
  khaeru

  committed
  6e8a774

• Add --modules/-m for 'mix-data report'

  khaeru

  committed
  f8ff4e7

• Add transport plots

  khaeru

  committed
  476e377

• Filter 0-length items from "mix-data report --module ..."

  khaeru

  committed
  6c68fb0

• Add reporting.CONFIG for use in code that does not load global.yaml

  khaeru

  committed
  e63e7cb

• Make report() a top-level method of the .reporting module

  khaeru

  committed
  2246ff5

• Report PRICE_COMMODITY and derived quantities (#87)

  Show description for be4498d
  

  GamzeUnlu95

  and

  khaeru

  committed
  be4498d

• CH4 initial reporting

  Show description for b2b7712
  

  Jihoon

  authored and

  khaeru

  committed
  b2b7712

• Resolve conflict for rebase

  

  Jihoon

  authored and

  khaeru

  committed
  b4a465c

• Add minor comments and bring back examples

  

  Jihoon

  authored and

  khaeru

  committed
  a710090

• Adjust to yaml format changes

  

  Jihoon

  authored and

  khaeru

  committed
  35c082d

• Add successfully an aggregate variable

  

  Jihoon

  authored and

  khaeru

  committed
  b9db904

• Add reporting.utils.REPLACE rule for CH4 fugitive emissions

  khaeru

  committed
  326b027

• Use CH4 output labels directly in aggregation

  Show description for 4672568
  khaeru

  committed
  4672568

• Compare auto-renamed CH4 liquids results with verbose method

  khaeru

  committed
  bab017b

• Further use CH4 output labels directly in aggregation

  khaeru

  committed
  f58aa68

• Add reporting.computations.select

  khaeru

  committed
  9245360

• Update reporting.util.collapse and .REPLACE for CH4

  khaeru

  committed
  139d08c

• Select technologies for CH4 IAMC reporting

  khaeru

  committed
  a4f5179

• Correct reporting.util.REPLACE for CH4 technologies

  khaeru

  committed
  87dbde2

• Remove verbose CH4 reporting to IAMC

  khaeru

  committed
  11a7e77

• Including fugitive and biomass burning in CH4 IAMC reporting

  khaeru

  committed
  a1ec4f4

• Simplify reporting of CH4 land use emissions

  khaeru

  committed
  42a18ea

• Tidy small/inadvertent changes in global.yaml

  khaeru

  committed
  79f155d

• Configure reporting of CH4 emissions from GLOBIOM

  khaeru

  committed
  8236b30

• Adjust test_iamc_replace_vars

  khaeru

  committed
  874a246

• Update replacements for reporting CH4 emissions from GLOBIOM

  khaeru

  committed
  bef3adf

• Sort technology names involved in CH4 emissions reporting

  khaeru

  committed
  22cf075

• Use output names directly for CH4 emissions from final energy use

  khaeru

  committed
  730fa85

• Expand docs of reporting.util.collapse

  khaeru

  committed
  96818bc

• Correct intersphinx cross-references in reporting docs

  

  francescolovat

  authored and

  khaeru

  committed
  23da9ce

• Correct code example typo in docstring of reporting.core.add_aggregate()

  

  francescolovat

  authored and

  khaeru

  committed
  de0dac2

• Import apply_units reporting comp from ixmp

  khaeru

  committed
  755144f

• Report emissions of F-gases

  Show description for a5da3ac
  khaeru

  committed
  a5da3ac

• Add computations.gwp_factors

  khaeru

  committed
  60d1f01

• Simplify YAML configuration for emissions reporting

  khaeru

  committed
  f61b145

• Add reporting.util.collapse_gwp_info

  khaeru

  committed
  e8d441c

• Remove gwp.csv

  khaeru

  committed
  b70da1c

• Expand comments for F-gas emissions reporting

  khaeru

  committed
  6cb3bfd

• Hide IAMC format YAML keys from Reporter.describe()

  khaeru

  committed
  b2dd927

• Add reporting.computations.select

  khaeru

  committed
  dc12836

• Add reporting.computations.apply_units

  khaeru

  committed
  ba0bfc3

• Update F-gas emissions reporting in global.yaml

  khaeru

  committed
  5ccc485

• Tidy logging in reporting.core

  khaeru

  committed
  9224dbd

• Remove computations.apply_units, .select

  Show description for f861310
  khaeru

  committed
  f861310

• Rewrite computations.combine using ixmp built-in computations

  khaeru

  committed
  fc0b48c

• Log a warning and continue on missing columns in util.collapse_gwp_info

  khaeru

  committed
  68eca2b

• Replace non-units for price quantities

  khaeru

  committed
  73a6d4d

• Report secondary energy (#120)

  Show description for facf452
  

  francescolovat

  and

  khaeru

  committed
  facf452

• Set "output dir" reporting config key

  khaeru

  committed
  8721299

• Move general Plot class to .reporting from .transport.report

  khaeru

  committed
  2470157

• Fix test_report_bare_res

  Show description for e8d2278
  khaeru

  committed
  e8d2278

• Update MESSAGEix-Transport usage docs; use Windows-friendly file names in tests (#158)

  Show description for 2818d4e
  

  francescolovat

  and

  khaeru

  committed
  2818d4e

• Silence a warning with Series.str.replace(regex=True) in .reporting.util

  Show description for 105d454
  khaeru

  committed
  105d454

• Add doc/repro.rst; config for sphinx.ext.{autosummary,viewcode}

  khaeru

  committed
  230e870

• Use message_data.testing in transport tests; blacken

  khaeru

  committed
  7539db6

• Tidy .transport.report and related code

  khaeru

  committed
  f0fae86

• Blacken .reporting.{cli,core}

  khaeru

  committed
  fd623be

• Use directory from "report --output=" option for plots

  khaeru

  committed
  56483da

• Remove upstreamed reporting code

  khaeru

  committed
  3d586ae

• Adjust imports; remove tests of upstreamed reporting code

  khaeru

  committed
  7fabe17

• Update /reporting/doc/index.rst; add genno to intersphinx config

  khaeru

  committed
  9a5713f

• Remove unused/unsupported arguments in .reporting.util.collapse()

  khaeru

  committed
  ef8a42b

• Override genno.config handler for "iamc:"; simplify prepare_reporter()

  khaeru

  committed
  f3d4c2c

• Remove obsolete test_iamc_replace_vars

  khaeru

  committed
  5a0af84

• Update report/global.yaml for genno

  khaeru

  committed
  0c3c2fd

• Update reporting documentation

  khaeru

  committed
  65c4bc4

• Merge .reporting.core to .reporting.__init__

  khaeru

  committed
  be57b10

• Add tests of .reporting.util.collapse()

  khaeru

  committed
  8547674

• Adjust transport reports for genno.compat.pyam

  khaeru

  committed
  527f262

• Allow reporting of unsolved scenarios

  khaeru

  committed
  7111e44

• Move mark_time() to .logging; add --verbose common CLI parameter

  khaeru

  committed
  2ef3914

• Lint (isort, wrap at 88 lines), comment, and document 8 files

  khaeru

  committed
  1b53075

• Make prepare_reporter(…, key=) arg optional

  khaeru

  committed
  002561d

• Tidy import, use of .logging

  khaeru

  committed
  c3eb20c

• Update .reporting.cli

  Show description for a0828c7
  khaeru

  committed
  a0828c7

• Update handling of default key in .reporting.prepare_reporter()

  khaeru

  committed
  19f73f9

• Update uses of deprecated Context methods/attributes in existing

  khaeru

  committed
  5f6340e

• Import from message_ix_models.util.{click,logging}, not old locations

  khaeru

  committed
  1d59500

• Adjust imports of migrated code

  khaeru

  committed
  3dcd6b0

• Quiet prepare_reporter() on an unsolved Scenario

  khaeru

  committed
  198f814

• Use private_data_path in the reporting CLI

  khaeru

  committed
  110cdd1

• Disambiguate reference to .reporting.prepare_reporter

  khaeru

  committed
  a8460ec

• Move data files to correct location

  

  OFR-IIASA

  authored and

  khaeru

  committed
  bb62283

• Move from xlsx to using csv to track changes

  

  OFR-IIASA

  authored and

  khaeru

  committed
  a9a5ab4

• Adapt and document syntax

  

  OFR-IIASA

  authored and

  khaeru

  committed
  1121c19

• Raise log level for "report --dry-run"

  khaeru

  committed
  1c8c641

• Don't set "report --output" to cwd by default (allow None)

  khaeru

  committed
  15aa957

• Move as_quantity() to .reporting.util

  khaeru

  committed
  dfb7b95

• Use local_data_path() to direct reporting outputs

  khaeru

  committed
  86e1599

• reporting.prepare_reporter() accepts a pre-populated Reporter

  khaeru

  committed
  6e12fc7

• Handle empty dict config in prepare_reporter()

  khaeru

  committed
  86eeb1d

• Use extra callbacks supplied via an argument in prepare_reporter()

  khaeru

  committed
  0de3a42

• Remove callback argument to prepare_reporter()

  khaeru

  committed
  7c78e22

• Guard against mypy errors in prepare_reporter()

  khaeru

  committed
  dfccdc8

• Update ENGAGE to v4.1.8 (#301)

  Show description for 3cc8aa8
  

  3 peopleauthored and

  khaeru

  committed
  3cc8aa8

• Move utilities for simulating solution data to .reporting.sim

  khaeru

  committed
  dfaa9b3

• Add .reporting.computations.make_output_path

  khaeru

  committed
  382fff3

• Handle "var:" key in "iamc:" reporting config sections

  khaeru

  committed
  2ea0397

• Adapt genno to some standard message_data reporting config

  khaeru

  committed
  06ff0dc

• Use "*" wildcards per genno 1.11

  khaeru

  committed
  2aeb9b0

• Quiet genno logging in add_simulated_solution()

  khaeru

  committed
  8ad230c

• Don't squash existing parameter data in add_simulated_solution()

  khaeru

  committed
  a14a4d8

• Use silence_log in add_simulated_solution()

  khaeru

  committed
  c408353

• Promote legacy reporting docs to nav menu; cross-link with others

  khaeru

  committed
  5f3702b

• Check that simulated_qty() do not lack dimension labels

  khaeru

  committed
  6316195

• Check for duplicate data in simulate_qty()

  khaeru

  committed
  f4b2d08

• Satisfy mypy for #337

  khaeru

  committed
  2200972

• Update files/directories to reduce lint-check exclusions (#384)

  Show description for 416042a
  

  OFR-IIASA

  authored and

  khaeru

  committed
  416042a

• Use "::" in genno keys in data/report/global.yaml

  Show description for d6b6ee9
  khaeru

  committed
  d6b6ee9

• Only infer full set of dimensions in prepare_reporter() if none given

  khaeru

  committed
  05dc7e2

• Use Context to pass options to report(), prepare_reporter()

  Show description for a06d2b2
  khaeru

  committed
  a06d2b2

• Simplify reporting CLI; use --dry-run via common_params()

  khaeru

  committed
  abbf4ee

• Adjust reporting tests to new signatures

  khaeru

  committed
  f43702e

• Expand commodity mapping in .reporting.util.REPLACE_DIMS

  Show description for 71f81e7
  khaeru

  committed
  71f81e7

• Use "message::default" as default key in report CLI

  khaeru

  committed
  b437472

• Move model_periods and usage from transport to general reporting

  khaeru

  committed
  a6af3a6

• Simplify prepare_reporter()

  khaeru

  committed
  73eff7b

• Roll import of reporting callbacks into register()

  khaeru

  committed
  3f311ea

• Invoke legacy reporting through "mix-models report" CLI

  khaeru

  committed
  5db34be

• Copy legacy reporting config files for materials from material-R12-rebase

  Show description for 005fcb5
  khaeru

  committed
  005fcb5

• Correct path to units file in report/materials.yaml

  khaeru

  committed
  4929867

• Always call Path.expanduser() and .mkdir() on reporting output paths

  khaeru

  committed
  d5c147e

• Add remove_all_ts() reporting computation; use for NAVIGATE

  khaeru

  committed
  95e17d8

• Use timeseries_only=True in remove_all_ts(); correct usage

  khaeru

  committed
  96aa839

• Always print result *and* path from reporting CLI

  khaeru

  committed
  9974f49

• Reduce redundant logging in prepare_reporter()

  khaeru

  committed
  0cd73cc

• Add a TODO in remove_all_ts()

  khaeru

  committed
  ada9489

• Fix NameError in .reporting.register()

  khaeru

  committed
  2211d53

• Satisfy mypy

  khaeru

  committed
  3b78a00

• Test invocation of iamc_report_hackathon.report()

  Show description for d51aa4c
  khaeru

  committed
  d51aa4c

• Adjust legacy reporting invocation by .reporting.report()

  Show description for 46bd9a6
  khaeru

  committed
  46bd9a6

• Warn about old args to .iamc_report_hackathon.report()

  khaeru

  committed
  0ef27f0

• Satisfy mypy in .transport.test_data.assert_units()

  khaeru

  committed
  265702d

• Simplify pass-through of legacy reporting args from CLI

  khaeru

  committed
  09134d5

• Expand types handled by as_quantity()

  khaeru

  committed
  b7509a7

• Adjust prepare_reporter for #82

  khaeru

  committed
  16213f5

• Sketch an integrated NAVIGATE workflow

  khaeru

  committed
  836c147

• Address warnings in Sphinx docs build

  Show description for 751363c
  khaeru

  committed
  751363c

• Allow specifying an initial year to .computations.remove_ts()

  Show description for c7c1e7d
  khaeru

  committed
  c7c1e7d

• Address untyped/implicit optional errors from mypy

  khaeru

  committed
  f65a1e9

• Add navigate_aggregates.csv

  

  GamzeUnlu95

  authored and

  khaeru

  committed
  c16c20a

• Add .reporting.util.add_replacements()

  khaeru

  committed
  1847a8d

• Handle partial sums in reporting "iamc:" config handler

  khaeru

  committed
  2649fa1

• Consolidate legacy reporting config for NAVIGATE

  Show description for f6e2eb2
  khaeru

  committed
  f6e2eb2

• Address mypy errors for #422

  khaeru

  committed
  6049f3c

• Handle missing "var" key in "iamc:" reporting config section

  khaeru

  committed
  757e1d1

• Add .reporting.computations.from_url()

  khaeru

  committed
  ddc281d

• Add .reporting.computations.get_ts()

  khaeru

  committed
  5c70163

• Add .reporting.util.copy_ts()

  khaeru

  committed
  ee40d68

• Ensure "y::model" and "y0" keys are in all reporters

  khaeru

  committed
  574ca98

• Debug .reporting.computations.from_url()

  Show description for 0a049f6
  khaeru

  committed
  0a049f6

• Debug .reporting.util.copy_ts()

  Show description for 3593ec1
  khaeru

  committed
  3593ec1

• Lint .reporting.computations

  khaeru

  committed
  f1fd123

• Add compound_growth() genno computation, tests

  khaeru

  committed
  46357e1

• Adjust test_generate_workflow_cli

  khaeru

  committed
  4738cb3

• Sort imports in 4 files

  khaeru

  committed
  48c3edb

• Adjust and sort imports in .report

  khaeru

  committed
  66cffd8

• Migrate enhanced silence_log() from message_data

  khaeru

  committed
  48a3555

• Update and integrate .report documentation

  khaeru

  committed
  63b385e

• Add .report.cli to "mix-models" CLI commands

  khaeru

  committed
  09fac8e

• Capitalize TODOs in .report.computations

  khaeru

  committed
  61cd48b

• Add #116 to doc/whatsnew

  khaeru

  committed
  bbac6e1

• Work around python/mypy#15843 in lint CI workflow

  khaeru

  committed
  d0b5faf

• Update test of silence_log()

  khaeru

  committed
  589b033

• Adjust compound_growth for pandas 2.1.0

  khaeru

  committed
  6da4b49

• Satisfy mypy in two places with genno 1.18.0

  khaeru

  committed
  b0b131a

• Adjust .report tests migrated from message_data

  khaeru

  committed
  e3596c6

• Adjust package names and relative paths in .report

  khaeru

  committed
  ba805c9

• Force use of "iamc:" genno config handler during migration

  Show description for 87ab426
  khaeru

  committed
  87ab426