Run specs for random scenarios

[michieldenhaan]

The ETM has various 'laws' that should hold under any circumstances. E.g. supply and demand of the hydrogen, collective heat and ccus mekkos should always balance, inputs and outputs of (most) nodes should be equal etc.

I think it would be very useful if we have something like Turk::Scenario.new(random, n=10) that generates n random scenarios. With random I mean:

• Random area
• Random target year
• Random amount of inputs set
• Random values for those inputs

to make testing for these 'laws' more robust. Currently, we have to explicitly specify all of the above. As a result we typically only test a few very specific situations.

Curious to hear your thoughts, @ChaelKruip, @antw

[antw]

The random element makes me a little uneasy, since it's quite possible that the scenario might set inputs which have nothing to do with the law being tested. It's possible that one of these laws might be broken, but for us to miss it because the wrong random numbers are generated. It might be that a law is only broken when a certain combination of inputs are set, and with 811 inputs the probability of happening upon that combination is quite low.

However I don't have a better suggestion, so I've added this to the backlog.

[michieldenhaan]

Thanks! I fully agree with your concerns. My worry is that we currently only test one or two predefined settings for most 'laws'. Supplementing this with X random combinations that change every night may allow us to find flaws that would otherwise have gone unnoticed. But it is definitely not fool proof.