Adding new years to a MESSAGE scenario

[behnam-zakeri]

PR checklist

• Tests added.
• Documentation added.
• Description in RELEASE_NOTES.md added.

[OFR-IIASA]

@behnam2015 please review scripts to make these pep8 conform. Once this is done, you will see that stickler-ci tests pass.

[OFR-IIASA]

@behnam2015 If I understand correctly, then this script essentially makes a new scenario from an existing scenario by copying all individual sets and parameters. In my opinion, that seems like a large workload as opposed to just adding datapoints, where required to an existing scenario; and should also save a lot of time in terms of processing and probably also reduce the number of lines of code. Also, as opposed to having this as a standalone tool, I think it would be desirable to have this part of the message_ix class, so that you can simply call e.g. "scen.toolbox.add_year([2025])". I also think that changing the firstmodelyear or changing the macro initial year shouldn't be part of the script's job. The script should be limited to simply adding additional datapoints. @gidden and @danielhuppmann would you agree? Are there any technical specifics that wouldn't allow this?

[OFR-IIASA]

Talking to Daniel it seems that due to the implementation in gams and java, it is not possible to insert a year into an existing model. Everything needs to be defined from the beginning, as otherwise the new year being added would end up being inserted after the last year, as both of the above dont know about the correct sequence of years.

[behnam-zakeri]

@OFR-IIASA thank you for the feedback and evaluating the code. There was one pep8 issue I resolved it, hopefully it passes the test now.

[behnam-zakeri]

@OFR-IIASA as you mentioned, the year sequence was the reason that the new years are added to an empty scenario and all parameters are then copied from the reference scenario to the new one.

[gidden]

would you mind taking a look here @khaeru? I will dig into #190

[khaeru]

This will be a solid addition! I am adding some notes here, in advance of a meeting on 2019-05-02 to discuss how to execute them:

• Use click rather than argparse for command-line interface & argument handling.
• Follow the PEP8 prescriptive naming conventions, in particular:
  • remove leading f_ from method names.
• Dissolve the class addNewYear to individual methods. Searching the code for self. shows that the class instance, self, is only used to refer to other method names. In this case, the class doesn't add anything (except indentation 😁).
• Use message_ix.tools instead of message_ix.toolbox namespace.
• Link the README for the new tool into the documentation build.

[behnam-zakeri]

Thanks a lot @khaeru for the comments and for your great additions! The use of click for comman-line handling looks very good and concise. I reformatted the class to a function, now called add_year. Originally this was a class to reduce the number of arguments passed over different functions, but it seems it's not needed anymore.

[khaeru]

Thanks for all the effort to tidy this up @behnam2015! It's a solid improvement to MESSAGEix that will be useful for many.

@gidden, since you requested my review, did you want to give this a second review?

[gidden]

Hi folks, happy to approve as-is. There are a few housecleaning things I would request here:

1. This introduces a new "standard" for calling CLIs (namely via the -m option). My preference here would be we choose a standard and stick to it. See e.g., setup.py:

   entry_points = {
        'console_scripts': [
            'messageix-config=message_ix.cli:config',
            'messageix-dl=message_ix.cli:dl',
        ],
}

I have no strong preference between the options here, but would suggest we maintain consistency. Let me know what you think!

2. Can you please add an issue to https://github.com/conda-forge/message-ix-feedstock/ to also test this CLI in the feedstock?

[khaeru]

1. This introduces a new "standard" for calling CLIs (namely via the -m option). My preference here would be we choose a standard and stick to it. See e.g., setup.py:

   entry_points = {
        'console_scripts': [
            'messageix-config=message_ix.cli:config',
            'messageix-dl=message_ix.cli:dl',
        ],
}

I have no strong preference between the options here, but would suggest we maintain consistency. Let me know what you think!

@behnam2015 and I used click in this PR to have a tidy CLI; but replacing the existing CLI was out of scope.

I think it should be a target for a future release to have a single entry point (message-ix) and then use click to dispatch to different commands. Once the overall CLI uses click, then the add_year tool/CLI can be converted to just one of the commands, and it can be called with message-ix add_year [options] [args]. For the moment, IMO it makes sense to keep it separate.

[gidden]

Ok, great advance here! Thanks all who helped out!