Enable modelling of cellular grid structures

[lensum]

What it is about

I'm trying to enable the modelling (and subsequent optimization) of cellular grid structures in oemof-solph. The cellular grid is prominently supported/proposed by the VDE-ETG. It is similar to Multi-Microgrid structures, where the microgrids are allowed to exchange energy with each other, but in a cellular grid each cell can contain sub-cells, which can contain sub-cells themselves (and so on). So there are more "levels" possible than in a multi-microgrid structure.

As the structure of a multi-microgrid / low-level cellular grid offers itself to be disaggregated, I'd like to enable a method to solve the resulting problem in a distributed way (e.g. Dantzig-Wolfe). As of yet, this is not part of the pull request and I have little knowledge and no experience with distributed optimization.

What functionality does it implement?

The overall functionality is described above, but I will try and go into detail a little more by explaining the classes I added. The usage is also displayed in the new example called cellular.

Model

Edit 2023-05-18: The model is now capable to handle cellular structures. Minor changes were necessary, mainly in the way how the flows and nodes are collected from the energysystem. A dict can now be passed to the Model-object under the energysystem keyword. This is the indication, that a cellular structure is to be optimized. Hint: There is still one large pyomo Model build for the whole cellular energy system. Not one per cell. This will change when and if I get the Dantzig-Wolfe reformulation working.

Update 2023-06-22: As pointed out in the comments below, a list will be used instead of a dict to pass the cells to the Model-class.

Other comments and questions

• I have not yet written any tests and documentation is consistently lacking.
• I am woefully unexperienced when it comes to open source collaboration, but I'm trying my best. Please make me aware of any misbehaviour of mine.
• Lastly: Feedback is highly welcome! I started this with an "stop thinking and make it work" attitude, which might not be the best approach (Edit 2023-05-18: Indeed, it was not :D ). It was the only viable option, as I don't have much knowledge about e.g. oemof-network and limited time on my hands. It might very well be, that there are simpler, better and more user-friendly ways to implement this. If you know them, I appreciate you pointing them out to me.

Related Issues

I know that #840 was working on something similar, but to the best of my knowledge the development stopped.

[lensum]

In case someone follows this PR, I want to provide a short status report.

The example given in examples\cellular\cellular.py is working, as far as I have tested. More extensive testing still needs to be done and I anticipate a few problems:

1. Right now, the implementation does not allow simultaneous energy im- and export for cells. I assumed this to be a necessary constraint, initially, however when looking at multi-level cell structures, it became clear to me that this will not work (i.e.: if cell_b and cell_c are both connected as "children" to cell_a and energy shall be moved from cell_b to cell_c, cell_a needs to simultaneously import and export energy). This should be easily fixable within the CellConnectorBlock class. This will probably also affect issue 2)
2. The implementation of max_power in the CellConnector class is redundant, as the flows connecting the CellConnector with the buses also have the ability to restrict the maximum power via max-Parameter. This could lead to inconsistent parametriziation and should be alleviated by changing the _max_input_rule and _max_output_rule in the CellConnectorBlock class.
3. The usage of CellConnector objects is way too complicated. Two CellConnector objects and four Flows is just too much hassle. There has got to be an easier way to do this. My next attempt will be to somehow connect the cells with automatically generated CellConnector objects, which will be part of the upmost cell. The specifics are still hazy to me as well, but I hope to reduce the number of CellConnector objects with this measure to one per connection, instead of the current two and the necessary Flows maybe to two (maybe with the bidirectional functionality, which I didn't use so far).

Also, there are a lot of more or less current #TODO in the code, which I hope to resolve soon.

If anyone has any questions or recommendations, feel free to reach out! I plan on atteding the upcoming oemof-meeting, so discussion in person is also possible.

Edit: if anyone could tell me whether the bidirectional flow is working with all other flow-functionalities, that would be a great help!

[p-snft]

Hi @lensum,

I'm not really into the topic, so please please forgive me my lack of understanding. I have a few questions:

• What are the advantages of modelling a grid as a cellular structure vs. having everything in one energy system without cells? Won't the results be the same?
• If flows can go into both directions at the same time, what is the difference between a CellConnector and a Bus? If the former has a limited capacity, then why it is not the Link?

If this is mostly about data analysis, we might just think about having something like balancing groups in post-processing.

[lensum]

Hey @p-snft,

no worries, these are exactly the kind of questions I was hoping for.

You are correct that the results are the same. This is also reflected by the fact that the pyomo model is (currently) build "monolithic" (there is only one pyomo model which contains all cells). The benefits I nonetheless see are twofold:

1. I think an energy system model should reflect the setup of the actual energy system. So when one wants to investigate a cellular energy system, it might be beneficial to be able to actually model it as such.
2. I hope to implement a Dantzig-Wolfe (DW) reformulation/decomposition, which should reduce the calculation time when optimizing large cellular structures. It requires a special structure of the constraint matrix (see the Wikipedia entry under "Required form"). My assumption is that the cellular modelling approach eases the identification and selection of the constraint blocks, because each cell is represented as a block and the coupling constraints are the connections between the blocks/cells. When DW is used, the results will still be the same (or at least very close to it), but it will be possible to optimize larger cellular energy systems (possibly with a higher accuracy) in the same amount of time.

Concerning the question Bus, CellConnector or Link: I have a post-it right next to my monitor reading "are busses an option to connect cells?". So I definitely see this as an option. But it wouldn't make a CellConnector class obsolete, as I would need something inside the cells to connect the Bus to, would it? And it would still need an in- and output on the cell-side (inside the cell, connected to the bus inside the cell) and an in- and output on the grid-side (outside the cell, connected to the bus outside the cell).
I haven't looked at the capabilities of the Link class, but will make sure to do so!

[lensum]

Hey again @p-snft ,
thank you very much for your feedback! I sat down and implemented the changes and it significantly reduced my "footprint" on the code, which I appreciate. It's a much better status to report on at the upcoming meeting.

I also updated the initial text. If I should just delete the old text instead (for a cleaner look), feel free to point that out to me.

[jokochems]

Hi @lensum,

sorry, was meaning to look at this feature earlier and really liked your presentation at the user & dev meeting as well as the idea to allow for a decomposition of oemof.solph models by slightly re-arranging / ordering things.

Please find some comments of mine to dedicated code sections. Overall,

• I think, you could facilitate even a bit more (by using a list instead a dict for the energy_system) and
• I'm a bit sceptical towards adding a facade without any additional functionality inside oemof.solph.

Also, I think, simply dropping the sections that are crossed out in the original post would facilitate its readability.

[lensum]

Hey @jokochems,

I implemented most of the suggested changes. Thanks a ton! I'm now working on satisfying the checks before I add tests.
Would it be okay to ask you for a review, once I'm done?

[lensum]

coveralls complains about decreased coverage, but the lines that aren't covered weren't written by me, as far as I can see. Should I provide tests for those lines regardless?

[lensum]

All requirements for a pull request known to me are fulfilled, therefore I'm removing the "Draft" status.

Also, I took the liberty to change the title of the documentation from "Welcome to oemof's documentation" to "Welcome to oemof solph's documentation". I did this because I seem to remember that the term "oemof" should no longer be used synonymously with "solph" and this rather prominent display bothered be somehow.

[jokochems]

Hey @jokochems,

I implemented most of the suggested changes. Thanks a ton! I'm now working on satisfying the checks before I add tests. Would it be okay to ask you for a review, once I'm done?

Sure, I can have a look. But I'll leave the final decisions to @p-snft.

[jokochems]

Hello @lensum,

to me everything looks good. I think, the tiny decrease in coverage can be ignored.
I left some minor comments in the docs with suggestions for slight rephrasing or additions. Also, I think introducing a new method in _models.Model could help to avoid code duplication.

Overall, these are all optional aspects. I would be fine with it as it currently is and leave it up to @p-snft to decide.

Thanks a lot! Looking forward to your upcoming decomposition activities.

[jokochems]

If I'm not mistaken, this is the same as above. Couldn't we add a tiny method for this that we call twice to avoid code duplication?

[p-snft]

I like the Idea but I think there is room for improvement. As we introduce new options in the API, I feel like these should be discussed first.

[p-snft]

Suggested change

	energysystem : EnergySystem object or list of EnergySystem objects (experimental)
	energysystem : EnergySystem object or (experimental) list of EnergySystem objects

Maybe this way, to highlight that just the second option is experimental?

[lensum]

I'm fine with that! It's easier to understand, indeed. I needed to shorten the line, so I deleted the of EnergySystem objects after list. It now reads energysystem : EnergySystem object or list (experimental) and I would change it to energysystem : EnergySystem object or (experimental) list. The explanation that the list needs to hold EnergySystem objects moved into the text below.
Is that okay with you as well?

I apologize for the cumbersome discription.

[p-snft]

I see. The line energysystem : EnergySystem object or (experimental) list sounds reasonable. Maybe also add the keyword in the description, so it reads:

    energysystem : EnergySystem object or (experimental) list
        Object that holds the nodes of an oemof energy system graph.
        Experimental: If a list  is passed, the list needs to hold
        EnergySystem objects and a cellular structure is assumed.
        In this case, the first element needs to be the upmost energy cell
        (structurally containing all other cells).

[lensum]

Good idea! The latest commit includes this change.