Add functionality in tools.costs to specify cost reduction scenarios and values

[measrainsey]

This PR adds the functionality for a user to add specific cost reduction values and scenario assumptions in the costs tool.

This PR closes:

• tools.costs does not allow for module-specific cost reduction scenarios and values #251

The logic is as follows (for both the cost reduction values and the reduction scenario categories):

1. If values/categories are specified in the non-energy module for a technology, then this value is used. If such values exist already in the energy module, the value in the non-energy module takes precedence (similar to how base year costs are implemented in this way) and replaces the energy module's values.
2. If no values/categories are specified in the non-energy module for a technology but that technology is mapped to an energy technology, then the mapped energy technology's values/categories are used.
3. If no values/categories are specified in the non-energy module or a mapped technology energy module for a technology, then no cost reduction is assumed.

For the time being, I have added some dummy examples using the materials module. The following cases I tested for:

• Specifying scenario assumptions and cost reduction values for a technology that only exists in the materials module.
• Specifying only scenario categories for a materials technology that is mapped to an energy technology. So while the cost reduction values associated with the categories for the energy technology is used, the customized scenario assumptions in the materials takes precedence.
• Specifying cost reduction values that already exists in the energy module with its own cost reduction values (therefore replacing the original cost reduction values).
• Specifying scenario assumptions and cost reduction values for a technology that does not have these values specified in the energy module. Therefore, if running energy module, then this technology has no cost reduction (because empty). But if running materials module, then the specified assumptions and values will be used.

I can remove the dummy examples after reviews say this methodology/PR makes sense.

How to review

For @GamzeUnlu95: Could you check if the methodology implemented in this PR would work for the costs differentiation you want to achieve for steel and cement technologies?

For @khaeru and/or @glatterf42 : Read the diff and note that the CI checks all pass.

PR checklist

• Continuous integration checks all ✅
• Add or expand Modify tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update doc/whatsnew.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.8%. Comparing base (aa49621) to head (5993c55).
Report is 16 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #255     +/-   ##
=======================================
- Coverage   76.7%   75.8%   -0.9%     
=======================================
  Files        203     203             
  Lines      15704   15733     +29     
=======================================
- Hits       12050   11931    -119     
- Misses      3654    3802    +148     

Files with missing lines	Coverage Δ
message_ix_models/tests/tools/costs/test_decay.py	100.0% <100.0%> (ø)
message_ix_models/tools/costs/decay.py	100.0% <100.0%> (ø)

... and 7 files with indirect coverage changes

[GamzeUnlu95]

The changes look good and align with the intended purpose of specifying customized cost reduction values for material module technologies. We can proceed with merging this. I will use it to implement differentiation for steel and cement technologies, which will then be merged into the ssp-dev branch.

[glatterf42]

Thanks for the PR, this looks mostly good to me! I would add more type hints and adjust the unit test rather than removing it, if possible :)

[glatterf42]

Somewhat strange to remove tests, but we seem to cover all lines touched by the PR even without it. Is there truly no need for this unit test any longer?
Even if this function is only ever called by another function, a unit test runs faster than the whole exterior function and grants confidence when debugging the code that this particular part still works or pinpoints the failure.
If it's not too much work, I think I'd like to see the test adjusted rather than removed.

[measrainsey]

I removed the test for get_cost_reduction_data() because the function was completely removed/replaced, but I now added new tests for the two new functions: _get_module_scenarios_reduction() and _get_module_cost_reduction()

[measrainsey]

The changes look good and align with the intended purpose of specifying customized cost reduction values for material module technologies. We can proceed with merging this. I will use it to implement differentiation for steel and cement technologies, which will then be merged into the ssp-dev branch.

Thanks @GamzeUnlu95 ! I removed the dummy data I created to avoid confusion when you need to use the module.

[glatterf42]

Great work, looks good to me now :)
Feel free to completely type the new tests, but no need (yet ;)

[glatterf42]

Amazing, even with type hints in the tests already :)
If you want to completely type this function: t_exp seems to be a set[str, str, str] or set[str, ...] if you want to be more generic :)

[measrainsey]

thanks @glatterf42 ! :)

i tried to add type hints for t_exp -- both set[str, str, str] and set[str, ...] bring up mypy linting errors, but set[str] seems to work, so hopefully that's okay?

[glatterf42]

Of course, I was confusing set and tuple 🤦‍♂️