Merge pull request #98 from Illuminator-team/dev

New release 3.0.0-beta

docs/cluster-setup.md

@@ -8,11 +8,11 @@ During simulation, the *server* engage with the *clients* to run the simulations
 information is exchanged between Rasberry Pi's using network sockets.
 The **server** provides a Dashboard to visualize the results, and saves them to a `.csv` files for later analysis. 
 
+
 <div align="center">
 	<img align="center" src="_static/img/Structure.jpg" width="500">
 </div>
 
-
 ## Hardware Requirements
 - A Raspberry Pi to use as a *server*.
 - One or more Raspebrry Pi's to use as *clients*.

docs/conf.py

@@ -38,7 +38,9 @@
               ]
 
 templates_path = ['_templates']
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'user/depricated-user-guide.md']
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'user/depricated-user-guide.md', 
+                    'tutorials/example.ipynb' # remove to include the example notebook
+                    ]
 
 
 

docs/developer/software-architecture.md

@@ -0,0 +1,112 @@
+# Software Architecture
+
+The Illuminator is modular Python applications to simulate energy systems. In this seciton, we provide an overview of its sotware architecture. The diagram below describes the components of the Illuminator. 
+
+<div align="center">
+    <img align="center" src="../_static/img/components.png" width="800">
+</div>
+
+
+## Users
+Users of the Illuminator take one of two roles:
+
+* **Moder Developer**: uses the Illuminator to create energy models that can be used in any simulation
+* **Energy Analys**: use sthe Illuminator to define and run simulation scenarios.
+
+<!-- ## Containers
+Containers represent applications that are separately runnable/deployable units of software. Below, we refer to containers as 'applicaitons' -->
+
+## Components
+
+### Mosaik Simulation Framework
+
+A framework that serves as a core platform for executing energy system simulations. [Mosaik](https://mosaik.readthedocs.io/en/latest/index.html) is an external dependency, and as such the Illuminator interacts with it thru its API.
+
+### Model Builder Application
+
+A Python applicaiton that model developers use to define/modify  energy models for the Illuminator. New models are developed using the **Builder** componente, which provides a custom interface for creating and registering energy models to the **Model Library**. The purpose of the **Builder** componet is ease the defintion of energy models using a jargon that **energy system engineers** are more familiar with. For example, by defining new models in term of *inputs, outputs, states, etc.*
+
+The **Model Library** component stores energy models that can be use in a simulation, so that they can be accessed by the **Mosaik Simulation Framework** during runtime.
+
+### Model Builder Application
+
+A Python application that facilated the development and registration of energy models for the **Model Library**, in wich the **Builder** component provides an interface that can be used to define energy models in terms of *parameters, inputs, output, states, etc*. This applicaiton stores energy models in the **Model Library** and mades them available to the **Mosaik Simulation Framework**. 
+
+Models in the **Model Library** are metadata and business logic containers. No computations are performed by this component.
+
+### Simulation Engine Application
+
+A Python application to run simulations via the Mosaik API. This application consists of four compoennts. The **Scenario API** provides a wrapper to prepare and start simulations in the **Mosaik Simulation Framework**. Therefore, simulations, computations  and the management of output data are delegated to the **Mosaik Simulation Framework**.
+The Senario API uses the **Scenario Schema** to validate simulation definitions writen as YAML files written by the **Energy Analysis**. The **Scenario Schema** defines the format that YAML files must be written on. 
+
+The **Illuminator CLI** is an appliccation implemented using [Typer](https://typer.tiangolo.com/), which provides a command line interface to run simulation locally, and automate the deployment of the Illuminator in a Raspberry Pi cluster. The **Illuminator CLI** uses the **Scernario API** and the **Cluster PI** components for its functionality. 
+
+Finally, the **Cluster Pi** component consists of a set of tools for setting up the Illuminator to tha Raspberry Py cluster that will run scenario simulatios. 
+
+### Dashboard
+
+An application used by the **Energy Analyst** to visualise the results and logs of simulatiosn in real-time. This is not implemented in the current version.
+
+-----
+## Use Cases
+
+There are three main use cases for the users of the Illuminator:
+
+1.  Extending the model library: a *Model developer* wants to add a new model to the  **Model Library**
+2.  Creating a simullation scenario: an *Energy Analyst* wants to define a simulation scenario using a YAML file and execute the simulation.
+3.  Set up a raspberry Pi cluster: a **user** wants to set up the Illuminator in a cluster of Raspberry Pi's to run simulations.
+
+### Extending the Model Library
+
+Energy model should be added to the **Model Library** as follows:
+
+1. Create a Python module with the name of the model. For example, `ExampleModel.py`
+1. In the file, create an IlluminatorModel object for the model. This defines which inputs, output, parameters, states, triggers, etc. a particular model has.  For example:
+
+```Python
+from illuminator.builder import IlluminatorModel
+# Defines a model'a paramters, inputs, outputs...
+example_model = IlluminatorModel(
+    parameters={"param1": "addition"},
+    inputs={"in1": 10, "in2": 20},
+    outputs={"out1": 0},
+    states={"out1": 0},
+    time_step_size=1,
+    time=None
+)
+```
+
+2.  Create a class that inherits from `ModelConstructor`, and impement the `step()` method. Name the new class with the same name as the Python module it contains it. The new class will become a *model type* in the Illuminator. Instances of this model type will be created by the **Scenario API**
+
+For example,
+
+```python
+from illuminator builder import ModelConstructor
+
+class ExampleModel (ModelConstructor):
+
+    def step():
+        """Computes this in every time step"""
+
+        # The computation logic goes here:
+
+        # return the time for the next time step
+        return time + self._model.time_step_size
+```
+
+3. To test the new model is implemented correctly, try:
+
+```python
+# TODO: add example
+```
+
+### Creating Simulation Scenarios
+
+TODO
+
+### Setting Up Cluster Pi
+
+Refer to [Cluster Pi setup.](../cluster-setup.md)
+
+
+

docs/diagrams/components.puml

@@ -0,0 +1,55 @@
+@startuml
+!include https://raw.githubusercontent.com/plantuml-stdlib/C4-PlantUML/master/C4_Component.puml
+
+scale 1.5
+LAYOUT_LANDSCAPE()
+title Component Diagram for the Illuminator
+
+AddElementTag("not implemented", $bgColor="#3c6658")
+
+Person(modelDeveloper, "Model Developer", "A person who wants to develop new models")
+Person(energyAnalyst, "Energy Analyst", "A person who creates simulations of energy systems")
+
+System_Ext(mosaik, "Mosaik Simulation Framework", "A Smart Grid co-simulation framework.  It provides a platform for executing  simualtions of energy systems")
+
+System_Boundary(illuminatorSys, "Illuminator", $descr="A Python package to simulate energy system") {
+
+    Container(dashboard, "Dashboard", "?", "An application for visualizing simulation results and logs in real-time", $tags="not implemented")
+
+    Container_Boundary(builderApp, "Model Builder Application", "Python", $descr="a Python application to develop energy models for the Illuminator") {
+        Component(modelBuilder, "Builder", "Python", "A custom interface for creating and registering energy system models")
+        Component(modelLibrary, "Model Libarry", "Python", "A collection of energy system models that can be use in a simulation")
+
+        Rel(modelBuilder, modelLibrary, "registers models to", "Python")
+        Rel(modelDeveloper, modelBuilder, "defines energy models using", "Python")
+
+    }
+    Container_Boundary(simulationEngine, "Simulation Engine Application", "Python", $descr="A Python application to simulate energy systems") {
+        Component(scenarioApi, "Scenario API", "Python", "generates and starts simulation scenarios.  It serves as a wrapper for the Mosaik API")
+        Component(clusterPi, "Cluster Pi", "Python, Bash", "tools for setting up a cluster of Raspberry Pi's for running simulations")
+        Component(cli, "Illuminator CLI", "Python, Typer", "a commnad line application to run simulation scenarios, and support the set up of a cluster of Raspberry Pi's")
+
+        Component(scenarioSchema, "Scenario Schema", "Python, YAML", "provides a formal schema for defining simulation scenarios using YAML files")
+
+        Rel(cli, scenarioApi, "uses", "Python")
+        Rel(cli, clusterPi, "uses", "Python")
+        Rel(scenarioApi, scenarioSchema, "validates scenario simulations using", "Python")
+
+}
+
+    Rel(scenarioApi, mosaik, "starts simulators in", "Mosaik API")
+    Rel(mosaik, dashboard, "provides simulation results and logs to", "Collector Model")
+
+}
+
+
+Rel(mosaik, modelLibrary, "access energy models from", "Python")
+
+Rel(energyAnalyst, scenarioApi, "defines simulation scenarios for", "YAML")
+Rel(energyAnalyst, cli, "runs simulation scenarios using", "Terminal")
+
+Rel(energyAnalyst,dashboard, "visualizes simulation results and logs in", "GUI")
+
+
+SHOW_LEGEND()
+@enduml

docs/index.rst

@@ -33,13 +33,12 @@ and the simulation engine is based on `Mosaik. <https://mosaik.offis.de/>`_
    :maxdepth: 2
    :caption: Tutorials
 
-   tutorials/example.ipynb
-
 .. toctree::
    :maxdepth: 2
    :caption: Developer's Documentation
 
    developer/set-up.md
+   developer/software-architecture.md
    developer/developer-docstrings.md
    developer/testing-explained.md
    developer/writing-tests.md

docs/references/engine-api.rst

@@ -1,10 +1,30 @@
-Engine's API
+Engine API
 =====================
 
-.. automodule:: illuminator.engine 
+The simulation engine provides a wrapper around Mosaik simulation to simplify the process of creating and running simulations. 
+
+Python Interface
+---------------------
+
+.. autoclass:: illuminator.engine.Simulation
     :members:
     :show-inheritance:
 
+Utility Functions
+---------------------
+
+.. autofunction:: illuminator.engine.apply_default_values
+
+.. autofunction:: illuminator.engine.build_connections
+
+.. autofunction:: illuminator.engine.compute_mosaik_end_time
+
+.. autofunction:: illuminator.engine.connect_monitor
+
+.. autofunction:: illuminator.engine.create_world
+
+.. autofunction:: illuminator.engine.generate_mosaik_configuration
+
 
 
 .. 

docs/user/config-file.md

@@ -68,26 +68,26 @@ monitor:
 
 | Keyword | Description | Optional | Default |
 |---------|-------------|----------|---------|
-| **scenario:** | a set of global values for a simulation. |  |  |
-|`name` | A name for the simulation, internally this name will be asssigned to what the Mosaik World created during runtime. |  |  |
-| `start_time` | start time for the simulation. Must be a timestamp in ISO 8601 format |  |  |
-| `end_time` | end time for the simulation. Must be a timestamp in ISO 8601 format.  |  |  |
-| `time_resolution` | number of seconds between simulation steps | ☑ | 900 (15 min)
-| **models:** | a list of models for the simulation | |  |
-|  `name` | a name for the model. Must be unique for each simulation |  |   |
-| `type`  | type of model. This must correspond with the name of the model registered in the Illuminator. | |  |
-| `inputs`  | a set of input-names and initial values for the model. The model type determines which names and values are applicable to each model, and they must be declared accordingly. Inputs are optional | | If the value is set to `null`, the default value will be used. See the respective model type for details.|
-| `outputs` | a set of output-names and initial values for the model. Similar to *inputs* valid names and values for each model are determined by the model *type*. See the respective model type for details. | | If the value is set to `null`, the default value will be used. |
-| `parameters`  | a set of name-value pairs for the model. Parameters declared constants for a model during runtime. | ☑ | If ommited, the default values will be used. See the respective model type for details. |
-| `states` | a set of name-value pairs considered as states for the model. The values modify the internal initial values of a state. | ☑ | If ommited, the default values will be used. See the respective model type for details. |
-| `triggers` | names of inputs, output or states that are use as triggers for a particular model. Triggers can only be declared by models that implement the *event-based paradigm*. See the respective model type to know if it accepts triggers. |  ☑ | |
-| `connect` | to declare in which client a model runs when using a Raspberry Pi cluster. | ☑  | |
-| `ip` | Ip of the client manchine that will run the model. Only IP version 4 format. |  |   |
-| `port` | TCP port to use to connect to the client machine| ☑   |   |
+| **scenario:** | a set of global values <br>for a simulation. |  |  |
+|`name` | A name for the simulation, internally <br>this name will be asssigned to what <br>the Mosaik World created during runtime. |  |  |
+| `start_time` | start time for the simulation.<br>Must be a timestamp in ISO 8601 format |  |  |
+| `end_time` | end time for the simulation. <br>Must be a timestamp in ISO 8601 format.  |  |  |
+| `time_resolution` | number of seconds between <br>simulation steps | &#9745; | 900 (15 min)
+| **models:** | a list of models for <br>the simulation | |  |
+|  `name` | a name for the model. Must <br>be unique for each simulation |  |   |
+| `type`  | type of model. This must correspond <br>with the name of the model <br>registered in the Illuminator. | |  |
+| `inputs`  | a set of input-names and initial <br>values for the model. The model <br>type determines which names and <br>values are applicable to each model, <br>and they must be declared accordingly. <br>Inputs are optional | | If the value is set to `null`, <br>the default value will be <br>used. See the respective model <br>type for details.|
+| `outputs` | a set of output-names and initial <br>values for the model. Similar to <br>*inputs* valid names and values <br>for each model are determined by <br>the model *type*. See the respective <br>model type for details. | | If the value is set to `null`, <br>the default value will be used. |
+| `parameters`  | a set of name-value pairs for <br>the model. Parameters declared constants <br>for a model during runtime. | &#9745; | If ommited, the default values <br>will be used. See the <br>respective model type for details. |
+| `states` | a set of name-value pairs considered <br>as states for the model. The values modify <br>the internal initial values of a state. | &#9745; | If ommited, the default <br>values will be used. See the <br>respective model type for details. |
+| `triggers` | names of inputs, output or states <br>that are use as triggers for a particular model. <br>Triggers can only be declared by models <br>that implement the *event-based paradigm*. <br>See the respective model type to know if <br>it accepts triggers. |  &#9745; | |
+| `connect` | to declare in which client a model runs <br>when using a Raspberry Pi cluster. | &#9745;  | |
+| `ip` | Ip of the client manchine that will run <br>the model. Only IP version 4 format. |  |   |
+| `port` | TCP port to use to connect to the <br>client machine| &#9745;   |   |
 | **connections:** |  how models connect to each other. |  |  |
-| `from`  | origin of the connection declared as `<model-name>.<output-name>`. Input names use here must also appear as *inputs* in the models section.   |   |  |
-| `to` | destination of the connection declared as `<model-name>.<input-name>`. Output names use here must also appear as *outputs* in the models section. |   | 
+| `from`  | origin of the connection declared as <br>`<model-name>.<output-name>`. Input names <br>use here must also appear as *inputs* in<br>the models section.   |   |  |
+| `to` | destination of the connection declared as <br>`<model-name>.<input-name>`. Output names <br>use here must also appear as *outputs* in <br>the models section. |   | 
 | **monitor:**  | 
-| `file` | path to a CSV file to store results of the simulation. File will be created if necessary. |  &#9745; | a `out.csv` file saved to the current directory |
-|`items` | a list of which inputs, outputs or states of models that most be monitored during runtime. Items must be declared as `<model-name>.<name>`, where *name* is an input, output or stated clared in the *models* section. No duplicated values are allowed  |  |   |
+| `file` | path to a CSV file to store results of <br>the simulation. File will be created if <br>necessary. |  &#9745; | a `out.csv` file saved to <br>the current directory |
+|`items` | a list of which inputs, outputs or states <br>of models that most be monitored during <br>runtime. Items must be declared as <br>`<model-name>.<name>`, where *name* is an <br>input, output or stated clared in the <br>*models* section. No duplicated values <br>are allowed  |  |   |
 

examples/PV_test.yaml

@@ -0,0 +1,65 @@
+scenario:
+  name: "SimpleTest" # in mosaik so called world
+  start_time: '2012-01-01 00:00:00' # ISO 8601 start time of the simulation
+  end_time: '2012-01-01 01:00:00' # duration in seconds 
+  time_resolution: 900 # time step in seconds (optional). Defaults to 15 minutes (900 s)
+
+
+models: # list of models for the energy network
+
+- name: CSVB # name for the model (must be unique)
+  type: CSV # name of the model registered in the Illuminator
+  parameters: # a CSV model must have a start time and a file as parameters
+    start: '2012-01-01 00:00:00' # ISO 8601 start time of the simulation
+    file_path: './examples/solar-sample.csv' # path to the file with the data
+    delimiter: ','
+    date_format: 'YYYY-MM-DD HH:mm:ss'
+
+- name: PV1
+  type: PV # models can reuse the same type
+  parameters:
+    m_area: 1.26
+    NOCT: 44
+    m_efficiency_stc: 0.198
+    G_NOCT: 800
+    P_STC: 250
+    peak_power: 600
+    m_tilt: 14
+    m_az: 180
+    cap: 500
+    output_type: 'power'
+  inputs: 
+    G_Gh: null 
+    G_Dh: null
+    G_Bn: null
+    Ta: null
+    hs: null
+    FF: null
+    Az: null
+  outputs:
+    pv_gen: 1
+    g_aoi: null
+
+
+connections:
+- from: CSVB.G_Gh # start model, pattern: model_name.output_name/input_name
+  to: PV1.G_Gh # end model
+- from: CSVB.G_Dh
+  to: PV1.G_Dh
+- from: CSVB.G_Bn
+  to: PV1.G_Bn
+- from: CSVB.Ta
+  to: PV1.Ta
+- from: CSVB.hs
+  to: PV1.hs
+- from: CSVB.FF
+  to: PV1.FF
+- from: CSVB.Az
+  to: PV1.Az
+
+
+monitor:
+  file: './out.csv' # optional with default, path to the results file for the scenario. This should be optional  # a list of models, its inputs, output and states to be monitored and logged
+  items:
+  - CSVB.G_Gh
+  - PV1.pv_gen