Return vintage_and_active_years() for a specific technology

[khaeru]

(Text by @OFR-IIASA—thanks!)

This issue aims to summarize an ongoing discussion on ECE Slack about a new, desirable feature.

The aim of the feature is to ease the creation of technology specific parameters which correlate vintage and active years, but limited to those that are feasible given a specific technology's technical_lifetime.

Possibilities for implementation that were discussed:

Option 1

Combine/modify two existing functionalities as suggested by @khaeru:

1. scen.vintage_and_active_years() returns a dataframe with the index year_vtg and year_act and correlates all the possible activity years within the optimization time horizon (>= firstmodelyear) with all possible vintage years, including also vintage years prior to the firstmodelyear.
2. scen.years_active() returns a list of activity years for any given node/technology/vintage_year.

The results of scen.vintage_and_active_years() would then be filtered based on scen.years_active().

Note that it would be necessary to also allow for the possibility to retrieve activity years prior to firstmodelyear, meaning that scen.vintage_and_active_years() would need to be modified so that activity years < firstmodelyear are also returned. This is necessary so that historical reporting can be carried out. Possibly a filter option is required e.g. as for the global model we report historic values from 1990 onwards.

Option 2

As suggested by @behnam-zakeri and extended by @OFR-IIASA:

from itertools import product
tec = "coal_ppl"
node = "R11_AFR"

def new_function_name(node, tec, vtg_lower=0, act_lower=0):
    """Returns vintage and corresponding activity years.
    
    Parameter
    ---------
    node : str
        Node name.
    tec : str
        Technology name.
    vtg_lower : int
        Vintage year as of which data should be returned.
    act_lower : int
        Activity year as of which data should be returned.
        
    Returns
    -------
    pandas.DataFrame
        with columns 'year_vtg' and 'year_act', in which each row is a
        valid pair.
    """
    
    # Retrieve model years for vintages should be added 
    yr_vtg = [x for x in scen.set("year") if x >= vtg_lower]
    
    # Retrieve model years for which activity should be added
    yr_act = [x for x in yr_vtg if x >= act_lower]
    
    # Retrieve technical lifetime
    tec_lifetime = scen.par("technical_lifetime",
                            filters={"node_loc": node,
                                     "technology": tec}
                            ).year_vtg.tolist()
    
    return(pd.DataFrame([x for x in product(yr_vtg, yr_act)
                         if x[0] in tec_lifetime and x[1] in scen.years_active(node, tec, x[0])],
                        columns=["year_vtg", "year_act"]))

new_function_name(node, tec, 1990, 2020)

Option 3

Build on ScenarioInfo.yv_ya as suggested by @khaeru. This is however in message-ix-models, not this repo.

Option 4

Build on some code used in message_data as a workaround to the fact that duration_period_sum [incomplete sentence?]

This information can also be used to filter for example the results from ScenarioInfo.yv_ya based on the technical_lifetime of a technology.

import pandas as pd
horizon = sorted([int(i) for i in scen.set('year')])

# Retrieve parameter duration period
dur = scen.par('duration_period')\
          .sort_values(by='year')\
          .drop('unit', axis=1)\
          .set_index('year')
df_dur = pd.DataFrame(index=horizon[:-1], columns=horizon)
for i in df_dur.index:
    for j in [x for x in df_dur.columns if x >= i]:
        df_dur.loc[i, j] = int(
            dur.loc[(dur.index >= i) & (dur.index <= j)].sum()
        )

Option 5

Make use the functionality that creates the set map_tec_lifetime.

This is only generated as part of the solving of a scenario, but contains precisely the information required.

[khaeru]

Some thoughts here:

• Option 1 is the way to go.
  • Not option 2: This duplicates the code in the existing methods. Instead of adding many arguments to new functions, we can just remind users via the docs that all the functionality of pandas is available to them:

    df = (
        scenario.new_method_name("coal_ppl")
        .query(f"node_loc == 'R11_AFR' and year_act >= {x} and year_vtg >= {y}")
    )

  • Not option 3: The reason is: code in message-ix-models is for use in the MESSAGEix-GLOBIOM modeling tools and workflows. It is only tested/generalized for those uses. Anything that is potentially applicable to any MESSAGEix models could be moved to this package (e.g. .tools.add_year); however, this incurs a higher standard of generalizability and applicability, thus more work to test. Since (per Option 1) we are talking about an extension to existing functionality, we may as well keep it here.
  • Not option 5: this code is currently buried in Java, so needs to be extricated as part of Disentangle message_ix and ixmp_source #254, and we can't/shouldn't independently invoke that where it now lives. But this is a helpful reminder that any code which resolves this issue can be reused to help with 254.
• vintage_and_active_years() and years_active() are not the best naming choices, because they're not mutually consistent. (I made this problem worse with message_ix_models.ScenarioInfo.yv_ya). We could consider and implement consistent names while addressing this issue, and deprecate the old ones.

[OFR-IIASA]

The function vintage_active_years takes the argument ya_args, which can be used to retrieve the activity_years for a specific node, tec, yr_vtg.

Parameters
----------
ya_args : tuple of (node, tec, yr_vtg), optional
    Arguments to :meth:`years_active`.

How about modifying this so that if yr_vtg is not provided, i.e. if only two arguments are provided, then the activity years for all vintage years included in the technologies technical_lifetime are returned?

[khaeru]

I would suggest that whoever tackles this start by writing some tests to clarify the new expected behaviours.

[OFR-IIASA]

@khaeru I have extended the tests for the function vintage_and_active_years() in the above mentioned PR. For the time being I have created a new testing script. This also includes the original test from test_core.py. We can later discuss where the tests should be located.