Feature/transforms speedup

[SamRWest]

Speeds up two slow parts of the generate_commodity_groups transform by replacing nested for loops with vectorised groupby calls.

Also adds unit tests for the added functions and small test data files (from austimes) for verification.
Pytest runs and coverage reporting are now integrated into CI as well.

CSV-only regression test results show decent overall speedup:

Total runtime: 286.65s (main: 310.07s)
Change in runtime (negative == faster): -23.42s (-7.6%)
Change in correct rows (higher == better): +469 (+0.6%)
Change in additional rows: -12 (-0.0%)

But not sure whether the additional row metric increasing or decreasing is good, can you advise?

Update: full GAMS CI run here didn't show any change to correct/additional rows :)

Total runtime: 293.08s (main: 314.38s)
Change in runtime (negative == faster): -21.30s (-6.8%)
Change in correct rows (higher == better): +0 (+0.0%)
Change in additional rows: +0 (+0.0%)

[olejandro]

Thanks @SamRWest, looks great! I left some minor comments. I guess, we'll need to merge the benchmarks PR first?

[olejandro]

But not sure whether the additional row metric increasing or decreasing is good, can you advise?

It really depends. :-) Some of them could be additional, because casing is not the same as in the benchmarks - doesn't matter for GAMS; some need extra work - e.g. a dimension is inccorrect; some should have not been generated...

[SamRWest]

Thanks @SamRWest, looks great! I left some minor comments. I guess, we'll need to merge the benchmarks PR first?

Yep. Looks like they're both ready to merge now, just need a couple of approvals.
But yes, do #184 first please.

[olejandro]

@siddharth-krishna please go ahead merging if you don't have any further comments.
It is fun collaborating across the time zones!

[siddharth-krishna]

Looks good! Go ahead with the merge when ready.