From 76fe7ec9112f87e360eb7e9d29b15ca9f4e4b889 Mon Sep 17 00:00:00 2001 From: Mark Vrijlandt Date: Tue, 15 Oct 2024 17:21:42 +0200 Subject: [PATCH] update readme, alembic revision script --- README.md | 145 +-- dev-requirements.txt | 22 +- esdl-mapeditor-config/README.md | 6 - esdl-mapeditor-config/esdl_config.py | 927 ------------------ .../nwn_dispatcher_config.template.json | 264 ----- .../nwn_dispatcher_service.json | 188 ---- .../nwn_dispatcher_service_table.json | 209 ---- .../nwn_dispatcher_service_table_both.json | 389 -------- esdl-mapeditor-config/settings.py | 128 --- requirements.txt | 14 +- scripts/create_dev_environment.sh | 5 - ....sh => db_models_generate_new_revision.sh} | 3 +- scripts/update_dependencies.sh | 8 - 13 files changed, 104 insertions(+), 2204 deletions(-) delete mode 100644 esdl-mapeditor-config/README.md delete mode 100644 esdl-mapeditor-config/esdl_config.py delete mode 100644 esdl-mapeditor-config/nwn_dispatcher_config.template.json delete mode 100644 esdl-mapeditor-config/nwn_dispatcher_service.json delete mode 100644 esdl-mapeditor-config/nwn_dispatcher_service_table.json delete mode 100644 esdl-mapeditor-config/nwn_dispatcher_service_table_both.json delete mode 100644 esdl-mapeditor-config/settings.py delete mode 100755 scripts/create_dev_environment.sh rename scripts/{install_dependencies.sh => db_models_generate_new_revision.sh} (72%) delete mode 100755 scripts/update_dependencies.sh diff --git a/README.md b/README.md index d903411..827696a 100644 --- a/README.md +++ b/README.md @@ -1,90 +1,107 @@ # Omotes REST service -This webservice allows the MapEditor to call the Omotes SDK to submit new workflow jobs, see their status and retrieve +This webservice allows the MapEditor to call the Omotes SDK to submit new workflow jobs, see their +status and retrieve the results. The webservice is based on TNOs Flask REST API Template. ## Usage -Copy `.env-template` to `.env` and fill with the appropriate values. +Copy `.env.template` to `.env` and fill with the appropriate values. To set up the components (for windows run in `Git Bash`): -``` + +```bash ./scripts/setup.sh ``` + To start the components: -``` + +```bash ./scripts/start.sh ``` + To stop the components: -``` + +```bash ./scripts/stop.sh ``` -# Flask REST API Template - -This is a skeleton application for a REST API. It contains a modular setup that should prevent annoying circular imports -that are sometimes an issue when scaling up Flask applications. It also contains a -generic library of code developed in other projects (under tno/shared). - -The key dependencies are: - -- [Flask-smorest](https://flask-smorest.readthedocs.io): A REST API framework built on top - of [marshmallow](https://marshmallow.readthedocs.io/). - -## Using the application +### Optional start with local code and exposed postgres (dev mode) -The application can run in Docker or locally. There is a docker-compose.dev.yml file which is meant for infrastructure -services, which are not part of your application. The purpose is to run that always, and then you can either use the -docker-compose.yml to start the API, or run it locally. +There is a dev mode for start which will use the local code for `omotes-rest`, `omotes-sdk-python` +and `omotes-sdk-protocol` (assuming the repos are all in the same folder as this `omotes-rest` +repo), instead of released docker images, and it will expose the postgres port: -The Makefile contains a number of common commands. To get started, you can run `make dev` and it should build the Docker -image and start the API in Gunicorn with the automatic reloader. If you wish to use the Flask local dev server, either -modify the docker-compose.yml file or start it outside of Docker, for example with `make dev-local`. - -Before running the application locally though, you are advised to create a virtual environment. Install the dependencies -using `pip install -r requirements.txt`, or `make requirements`. Then, copy the .env-template file to .env. - -Either way, the API should start on http://localhost:9200. Access the autogenerated docs -through http://localhost:9200/openapi or http://localhost:9200/redoc. - -## How to use this template - -You would typically copy this template, and replace all instances of "flask_rest_api" with the name of your application. -Application specific code would then go under `tno//`. Adding code under `tno/shared` is of -course also possible, but keep in mind that the goal of that folder is to allow for sharing between multiple -repositories. A proper way to actually set that up still needs to be figured out though. - -Your endpoints would go under `tno//__init__.py`. - -## Notable features - -There is a very permissive setup of CORS, so that an arbitrary frontend can perform requests to this REST PAI. - -The application contains a setup of structlog, a structured logging framework. This makes it trivial to perform more -advanced logging, and will by default output JSON logs in production and colored tab-separated logs in development. - -For dependency management we use pip-tools, which is a combination of pip-compile (which generates a requirements.txt -from a requirements.in file) and pip-sync (which synchronizes your virtualenv to the exact state as specified in the -requirements.txt). - -There is a basic configuration of mypy (see mypy.ini) for static type checking. +```bash +./scripts/start-dev.sh +``` -## Design decisions +# Directory structure + +The following directory structure is used: + +- `ci/`: Contains all CI & other development scripts to help standardize the development workflow + for Linux. +- `config/`: Contains orchestrator workflow definitions configuration. The `workflow_config.json` + file will be overwritten by a volume mount when deploying via docker. +- `scripts/`: Setup, start en stop scripts. +- `src/`: Source code for omotes-rest. +- `unit_test/`: All unit tests for omotes-rest. +- `.dockerignore`: Contains all files and directories which should not be available while building + the docker image. +- `.env.template`: Template `.env` file to run the orchestrator locally outside of docker. +- `.gitignore`: Contains all files and directories which are not kept in Git source control. +- `dev-requirements.txt`: Pinned versions of all development and non-development dependencies. +- `Dockerfile`: The build instructions for building the docker image. +- `dev.Dockerfile`: Used when running or testing with local code from the `omotes-system` + repository. +- `pyproject.toml`: The Python project (meta) information. +- `requirements.txt`: Pinned versions of all dependencies needed to run the orchestrator. + +# Development workflow + +The scripts under `ci/` are used to standardize the development proces. + +- `create_venv`: Creates a local virtual environment (`.venv/`) in which all dependencies may be + installed. +- `install_dependencies`: Installs all development and non-development dependencies in the local + virtual environment. +- `lint`: Run the `flake8` to check for linting issues. +- `test_unit`: Run all unit tests under `unit_test/` using `pytest`. +- `typecheck`: Run `mypy` to check the type annotations and look for typing issues. +- `update_dependencies`: Update `dev-requirements.txt` and `requirements.txt` based on the + dependencies specified in `pyproject.toml` + +A typical development workflow would be: + +1. create and configure `.env` from `.env.template` +2. run `create_venv` +3. run `install_dependencies`. +4. develop or update the codebase according to the requirements... +5. run `lint`, `test_unit`, and `typecheck` to check for code quality issues. + +All these scripts are expected to run from the root of the repository. + +## How to work with alembic to make database revisions + +First set up the development environment with `create_venv` and `install_dependencies`. Then you +can make the necessary changes to `src/omotes-rest/db_models/`. Finally, a new SQL schema +revision may be generated using `alembic` by running: +```bash +./scripts/db_models_generate_new_revision.sh "revision message" +``` -### Flask-smorest +All database revisions will be automatically applied when omotes-rest is started. -Flask-smorest is but one option for REST APIs in Flask. It seems currently the most up to date option, and added bonuses -are that it is relatively lightweight, building on top of existing paradigms from Flask (such as Blueprints) and -building on top of marshmallow for schema validation. +## Direct Alembic control -Flask-Restless just generates REST-like API's for your database models. This is not really REST and can be quite -fragile. +In case more control is necessary, you can run the necessary alembic commands directly after +activating the virtual environment. -Flask-RESTPlus is pretty nice and is very similar to Flask-smorest. It is heavier than Flask-smorest though, inventing -more of their own paradigms on top of Flask. It is hardly maintained though. +First, change directory: `cd src/` -There is also Flask-RESTX, which is a fork of Flask-RESTPlus so is very similar. We've used it together with -flask-accepts, marshmallow, and marshmallow-dataclass for pretty nice results. However, Flask-smorest does not need any -of that because it just directly builds on top of marshmallow, so the end result is cleaner. \ No newline at end of file +- Make a revision: `alembic revision --autogenerate -m ""` +- Perform all revisions: `alembic upgrade head` +- Downgrade to a revision: `alembic downgrade ` (revision 'base' to + undo everything.) diff --git a/dev-requirements.txt b/dev-requirements.txt index 8aa9db0..e306e43 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -12,7 +12,7 @@ aiormq==6.8.1 # via # -c requirements.txt # aio-pika -alembic==1.13.2 +alembic==1.13.3 # via # -c requirements.txt # omotes-rest (pyproject.toml) @@ -65,7 +65,7 @@ colorama==0.4.6 # via # -c requirements.txt # omotes-rest (pyproject.toml) -coverage[toml]==7.6.1 +coverage[toml]==7.6.3 # via pytest-cov flake8==6.1.0 # via @@ -129,7 +129,7 @@ mako==1.3.5 # via # -c requirements.txt # alembic -markupsafe==2.1.5 +markupsafe==3.0.1 # via # -c requirements.txt # jinja2 @@ -168,7 +168,7 @@ mypy-extensions==1.0.0 # black # mypy # typing-inspect -omotes-sdk-protocol==0.1.5 +omotes-sdk-protocol==0.1.6 # via # -c requirements.txt # omotes-sdk-python @@ -197,10 +197,14 @@ platformdirs==4.3.6 # via black pluggy==1.5.0 # via pytest -prompt-toolkit==3.0.47 +prompt-toolkit==3.0.48 # via # -c requirements.txt # click-repl +propcache==0.2.0 + # via + # -c requirements.txt + # yarl protobuf==4.25.5 # via # -c requirements.txt @@ -215,7 +219,7 @@ pydocstyle==6.3.0 # via flake8-docstrings pyflakes==3.1.0 # via flake8 -pyproject-hooks==1.1.0 +pyproject-hooks==1.2.0 # via build pytest==7.3.2 # via @@ -252,7 +256,7 @@ structlog==23.1.0 # via # -c requirements.txt # omotes-rest (pyproject.toml) -tomli==2.0.1 +tomli==2.0.2 # via black types-flask-cors==5.0.0.20240902 # via omotes-rest (pyproject.toml) @@ -270,7 +274,7 @@ typing-inspect==0.9.0 # via # -c requirements.txt # marshmallow-dataclass -tzdata==2024.1 +tzdata==2024.2 # via # -c requirements.txt # celery @@ -294,7 +298,7 @@ werkzeug==3.0.4 # -c requirements.txt # flask # flask-smorest -yarl==1.11.1 +yarl==1.15.2 # via # -c requirements.txt # aio-pika diff --git a/esdl-mapeditor-config/README.md b/esdl-mapeditor-config/README.md deleted file mode 100644 index bee0536..0000000 --- a/esdl-mapeditor-config/README.md +++ /dev/null @@ -1,6 +0,0 @@ -# MapEditor config files for NieuweWarmteNu! - -This folder contains a few python files and javascript files that are used to integrate within the MapEditor to access -the Omotes REST service. -Specifically esdl_config.py that defines an example integration in python. - diff --git a/esdl-mapeditor-config/esdl_config.py b/esdl-mapeditor-config/esdl_config.py deleted file mode 100644 index 95f207b..0000000 --- a/esdl-mapeditor-config/esdl_config.py +++ /dev/null @@ -1,927 +0,0 @@ -# This work is based on original code developed and copyrighted by TNO 2020. -# Subsequent contributions are licensed to you by the developers of such code and are -# made available to the Project under one or several contributor license agreements. -# -# This work is licensed to you under the Apache License, Version 2.0. -# You may obtain a copy of the license at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Contributors: -# TNO - Initial implementation -# Manager: -# TNO -import os - -from settings import heatnetwork_dispatcher_config - -EPS_WEB_HOST = os.getenv("EPS_WEB_HOST", "http://epsweb:3401") -ESDL_AGGREGATOR_HOST = os.getenv("ESDL_AGGREGATOR_HOST", "http://esdl-aggregator:3490") -HEATNETWORK_DISPATCHER_URL = heatnetwork_dispatcher_config["host"] + ":" + str(heatnetwork_dispatcher_config["port"]) - -esdl_config = { - "control_strategies": [ - {"name": "DrivenByDemand", "applies_to": "Conversion", "connect_to": "OutPort"}, - {"name": "DrivenBySupply", "applies_to": "Conversion", "connect_to": "InPort"}, - { - "name": "StorageStrategy", - "applies_to": "Storage", - "parameters": [ - {"name": "marginalChargeCosts", "type": "SingleValue"}, - {"name": "marginalDischargeCosts", "type": "SingleValue"}, - ], - }, - ], - "predefined_quantity_and_units": [ - { - "id": "eb07bccb-203f-407e-af98-e687656a221d", - "description": "Energy in GJ", - "physicalQuantity": "ENERGY", - "multiplier": "GIGA", - "unit": "JOULE", - }, - { - "id": "cc224fa0-4c45-46c0-9c6c-2dba44aaaacc", - "description": "Energy in TJ", - "physicalQuantity": "ENERGY", - "multiplier": "TERRA", - "unit": "JOULE", - }, - { - "id": "e9405fc8-5e57-4df5-8584-4babee7cdf1a", - "description": "Power in kW", - "physicalQuantity": "POWER", - "multiplier": "KILO", - "unit": "WATT", - }, - { - "id": "e9405fc8-5e57-4df5-8584-4babee7cdf1b", - "description": "Power in MW", - "physicalQuantity": "POWER", - "multiplier": "MEGA", - "unit": "WATT", - }, - { - "id": "e9405fc8-5e57-4df5-8584-4babee7cdf1c", - "description": "Power in VA", - "physicalQuantity": "POWER", - "unit": "VOLT_AMPERE", - }, - { - "id": "6279c72a-228b-4c2c-8924-6b794c81778c", - "description": "Reactive power in VAR", - "physicalQuantity": "POWER", - "unit": "VOLT_AMPERE_REACTIVE", - }, - ], - "predefined_esdl_services": [ - { - "id": "18d106cf-2af1-407d-8697-0dae23a0ac3e", - "name": "Get PICO wind potential", - "explanation": "This queries the Geodan wind potential service for a certain area", - "url": "https://pico.geodan.nl/pico/api/v1///windturbinegebied", - "http_method": "get", - "headers": { - "Accept": "application/esdl+xml", - "User-Agent": "ESDL Mapeditor/0.1", - }, - "type": "geo_query", - "result": [{"code": 200, "action": "esdl"}], - "geographical_scope": { - "url_area_scope": "", - "url_area_id": "", - "area_scopes": [ - {"scope": "PROVINCE", "url_value": "provincies"}, - {"scope": "REGION", "url_value": "resgebieden"}, - {"scope": "MUNICIPALITY", "url_value": "gemeenten"}, - ], - }, - "query_parameters": [ - { - "name": "Distance to buildings", - "description": "Minimum distance to the built environment (in meters)", - "parameter_name": "bebouwingsafstand", - "type": "integer", - }, - { - "name": "Restriction", - "description": "", - "parameter_name": "restrictie", - "type": "multi-selection", - "possible_values": [ - "natuur", - "vliegveld", - "infrastructuur", - "agrarisch", - "turbines", - ], - }, - { - "name": "Preference", - "description": "", - "parameter_name": "preferentie", - "type": "multi-selection", - "possible_values": [ - "natuur", - "vliegveld", - "infrastructuur", - "agrarisch", - "turbines", - ], - }, - { - "name": "Include geometry in ESDL", - "description": "", - "parameter_name": "geometrie", - "type": "boolean", - }, - ], - }, - { - "id": "50fa716f-f3b0-464c-bf9f-1acffb24f76a", - "name": "Get PICO solar field potential", - "explanation": "This queries the Geodan solar field potential service for a certain area", - "url": "https://pico.geodan.nl/pico/api/v1///zonneveldgebied", - "http_method": "get", - "headers": { - "Accept": "application/esdl+xml", - "User-Agent": "ESDL Mapeditor/0.1", - }, - "type": "geo_query", - "result": [{"code": 200, "action": "esdl"}], - "geographical_scope": { - "url_area_scope": "", - "url_area_id": "", - "area_scopes": [ - {"scope": "PROVINCE", "url_value": "provincies"}, - {"scope": "REGION", "url_value": "resgebieden"}, - {"scope": "MUNICIPALITY", "url_value": "gemeenten"}, - ], - }, - "query_parameters": [ - { - "name": "Distance to buildings (m)", - "description": "Minimum distance to the built environment (in meters)", - "parameter_name": "bebouwingsafstand", - "type": "integer", - }, - { - "name": "Restriction", - "description": "", - "parameter_name": "restrictie", - "type": "multi-selection", - "possible_values": [ - "natuur", - "vliegveld", - "infrastructuur", - "agrarisch", - "turbines", - ], - }, - { - "name": "Preference", - "description": "", - "parameter_name": "preferentie", - "type": "multi-selection", - "possible_values": [ - "natuur", - "vliegveld", - "infrastructuur", - "agrarisch", - "turbines", - ], - }, - { - "name": "Include geometry in ESDL", - "description": "", - "parameter_name": "geometrie", - "type": "boolean", - }, - ], - }, - { - "id": "c1c209e9-67ff-4201-81f6-0dd7a185ff06", - "name": "Get PICO rooftop solar potential", - "explanation": "This queries the Geodan rooftop solar potential service for a certain area", - "url": "https://pico.geodan.nl/pico/api/v1///zonopdak?geometrie=false", - "http_method": "get", - "headers": { - "Accept": "application/esdl+xml", - "User-Agent": "ESDL Mapeditor/0.1", - }, - "type": "geo_query", - "result": [{"code": 200, "action": "esdl"}], - "geographical_scope": { - "url_area_scope": "", - "url_area_id": "", - "area_scopes": [ - {"scope": "PROVINCE", "url_value": "provincies"}, - {"scope": "REGION", "url_value": "resgebieden"}, - {"scope": "MUNICIPALITY", "url_value": "gemeenten"}, - ], - }, - "query_parameters": [], - }, - { - "id": "42c584b1-43c1-4369-9001-c89ba80d8370", - "name": "Get PICO Startanalyse results", - "explanation": "This queries the Geodan start analyse service for a certain area", - "url": "https://pico.geodan.nl/pico/api/v1///startanalyse", - "http_method": "get", - "headers": { - "Accept": "application/esdl+xml", - "User-Agent": "ESDL Mapeditor/0.1", - }, - "type": "geo_query", - "result": [{"code": 200, "action": "esdl"}], - "geographical_scope": { - "url_area_scope": "", - "url_area_id": "", - "area_scopes": [ - {"scope": "MUNICIPALITY", "url_value": "gemeenten"}, - {"scope": "NEIGHBOURHOOD", "url_value": "buurt"}, - ], - }, - "query_parameters": [ - { - "name": "Strategie", - "description": "", - "parameter_name": "strategie", - "type": "selection", - "possible_values": [ - "S1a", - "S1b", - "S2a", - "S2b", - "S2c", - "S2d", - "S3a", - "S3b", - "S3c", - "S3d", - "S4a", - "S5a", - ], - } - ], - }, - { - "id": "7f8722a9-669c-499d-8d75-4a1960e0429f", - "name": "Create ETM scenario", - "explanation": "This service sends the ESDL information to the ETM and tries to generate an ETM scenario out of it.", - "url": "http://beta-esdl.energytransitionmodel.com/api/v1/EnergySystem/", - # "url": "http://localhost:5000/api/v1/EnergySystem/", - "http_method": "post", - "headers": {"Content-Type": "application/x-www-form-urlencoded"}, - "type": "send_esdl", - "body": "url_encoded", - "send_email_in_post_body_parameter": "account", - "query_parameters": [ - { - "name": "Environment", - "description": "", - "parameter_name": "environment", - "location": "body", - "type": "selection", - "possible_values": ["pro", "beta"], - } - ], - "result": [{"code": 200, "action": "print"}], - }, - { - "id": "912c4a2b-8eee-46f7-a225-87c5f85e645f", - "name": "ESDL Validator", - "explanation": "This service allows you validate an ESDL against a certain schema", - "type": "vueworkflow", - "workflow": [ - { - "name": "Select schema", - "description": "", - "type": "select-query", - "multiple": False, - "source": { - "url": "http://10.30.2.1:3011/schema", - "http_method": "get", - "label_fields": ["name"], - "value_field": "id", - }, - "target_variable": "schema", - "next_step": 1, - }, - { - "name": "Schema validation", - "description": "", - "type": "service", - "state_params": {"schemas": "schema.id"}, - "service": { - "id": "64c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Validate ESDL against scehema", - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml", - }, - "url": "http://10.30.2.1:3011/validationToMessages/", - "http_method": "post", - "type": "send_esdl", - "query_parameters": [ - { - "name": "Schemas", - "description": "ID of the schema to validate this ESDL against", - "parameter_name": "schemas", - } - ], - "body": "", - "result": [{"code": 200, "action": "asset_feedback"}], - "with_jwt_token": False, - "state_params": True, - }, - }, - ], - }, - { - "id": "123451a2-c92a-46ed-a7e4-993197712345", - "name": "Start Heat Network calculation", - "description": "Submit a heat network to the calculation service", - "headers": { - "User-Agent": "ESDL Mapeditor/1.0", - "Content-Type": "application/json" - }, - "url": f"{HEATNETWORK_DISPATCHER_URL}/job/", - "http_method": "post", - "type": "send_esdl", - "send_email_in_post_body_parameter": "user_name", - "show_query_params": True, - "query_parameters": [ - { - "type": "string", - "name": "Scenario name", - "description": "Give this scenario a unique name", - "parameter_name": "job_name", - "location": "body" - }, - { - "type": "selection", - "name": "Workflow type", - "description": "Select the calculation type to execute", - "parameter_name": "work_flow_type", - "location": "body", - "possible_values": [ - "GROWTH_OPTIMIZER" - ] - } - ], - "body_config": { - "type": "json", - "parameters": [ - { - "type": "esdl", - "encoding": "base64_encoded", - "parameter": "input_esdl" - } - ] - }, - "body": "base64_encoded", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Calculation successfully submitted." - }, - { - "code": 404, - "action": "show_message", - "message": "Error in submitting calculation." - } - ], - "with_jwt_token": True, - "state_params": False - }, - { - "id": "x12c4a2b-8eee-46f7-a225-87c5f85e645f", - "name": "Heat Network calculation results", - "explanation": "This service allows you to view results from heat network simulations and optimizations", - "type": "vueworkflow", - "workflow": [ - { - "name": "Results overview", - "description": "", - "type": "table-query", - "source": { - "url": HEATNETWORK_DISPATCHER_URL + "/job/user/{username}", - "http_method": "get", - "request_params": {"username": "session.email"}, - "columns": [ - { - "dataIndex": "job_name", - "title": "Name", - "ellipsis": True - }, - { - "dataIndex": "status", - "title": "Status", - "align": "center", - "slots": { - "customRender": "tags" - } - }, - { - "dataIndex": "stopped_at", - "title": "Finished at", - "formatter": "formatDate", - "defaultSortOrder": "descend" - } - ], - "actions": [ - { - "title": "Load result", - "next_step": 2 - }, - { - "title": "Show log", - "next_step": 3 - }, - { - "title": "Delete", - "next_step": 4 - } - ], - "value_field": "job_id" - }, - "target_variable": "job", - "with_jwt_token": True, - "next_step": 0 - }, - { - "name": "Result selected", - "state_params": { - "job_id": "job.job_id" - }, - "description": "How would you like to proceed?", - "type": "choice", - "options": [ - { - "name": "Load ESDL result", - "description": "Load the results as ESDL on the map.", - "type": "primary", - "next_step": 2 - }, - { - "name": "View log output", - "description": "View the log output of this execution.", - "type": "default", - "next_step": 3 - } - ] - }, - { - "name": "Load ESDL", - "description": "Loads the result in the ESDL MapEditor", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x1c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Load ESDL results", - "description": "The results should have been loaded in the MapEditor", - "button_label": "Go back to results", - "auto": True, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": HEATNETWORK_DISPATCHER_URL + "/job//result", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "esdl", - "encoding": "base64_encoded", - "json_field": "output_esdl" - } - ], - "with_jwt_token": True, - "state_params": True, - "next_step": 0 - } - }, - { - "name": "Show log file", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x5c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Request the log file", - "description": "The logfile is shown below:", - "button_label": "Go back", - "auto": True, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": HEATNETWORK_DISPATCHER_URL + "/job//logs", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Show log", - "json_field": "logs" - } - ], - "with_jwt_token": True, - "state_params": True, - "next_step": 0 - } - }, - { - "name": "Delete result", - "description": "Delete the selected result", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x9c9d1a2-c92a-46ed-a7e4-9931971cbb2e", - "name": "Delete the result", - "description": "Press the button below to delete the result", - "button_label": "Delete result", - "headers": { - "User-Agent": "ESDL Mapeditor/1.0", - "Content-Type": "application/json" - }, - "url": HEATNETWORK_DISPATCHER_URL + "/job/", - "http_method": "delete", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Result succesfully removed" - } - ], - "with_jwt_token": True, - "state_params": True, - "next_step": 0 - } - } - ] - } - - ], - "role_based_esdl_services": { - "eps": [ - { - "id": "9951c271-f9b6-4c4e-873f-b309dff19e03", - "name": "Energy Potential Scan", - "explanation": "This workflow allows you to run the EPS web service and view the EPS results.", - "type": "vueworkflow", - "workflow": [ - { - # 0 - "name": "Begin", - "description": "How would you like to proceed?", - "type": "choice", - "options": [ - { - "name": "Create a new EPS project", - "next_step": 1, - "type": "primary", - }, - { - "name": "Adjust EPS input", - "next_step": 13, - "type": "default", - }, - {"name": "Run EPS", "next_step": 4, "type": "primary"}, - { - "name": "Inspect EPS results", - "next_step": 6, - "type": "primary", - }, - { - "name": "Aggregate ESDL buildings for ESSIM", - "next_step": 12, - "type": "default", - }, - ], - }, - { - # 1 - "name": "Create Project", - "description": "", - "type": "custom", - "component": "eps-create-project", - "url": f"{EPS_WEB_HOST}/api/projects/", - "next_step": 0, - }, - { - # 2 - "name": "Select existing project", - "description": "Please select the project for which you would like to upload a project file.", - "type": "select-query", - "multiple": False, - "source": { - "url": f"{EPS_WEB_HOST}/api/projects/", - "http_method": "get", - "choices_attr": "projects", - "label_fields": ["name"], - "value_field": "id", - }, - "target_variable": "project_id", - "next_step": 3, - }, - { - # 3 - "name": "Upload project file", - "description": "Note: When uploading a project file with the same name as a previous project " - "file, the previous file will be overwritten!", - "type": "upload_file", - "target": { - "url": f"{EPS_WEB_HOST}/api/uploads/", - "request_params": {"project_id": "project.id"}, - "response_params": {"name": "file_name"}, - }, - "next_step": 0, - }, - { - # 4 - "name": "Select existing project", - "description": "", - "type": "select-query", - "multiple": False, - "source": { - "url": f"{EPS_WEB_HOST}/api/projects/", - "http_method": "get", - "choices_attr": "projects", - "label_fields": ["name"], - "value_field": "id", - }, - "target_variable": "project", - "next_step": 5, - }, - { - # 5 - "name": "Run the EPS", - "description": "", - "type": "custom", - "component": "eps-service", - "url": f"{EPS_WEB_HOST}/api/eps/", - "target": { - "request_params": {"project_id": "project.id"}, - "user_response_spec": { - "0": {"message": "Failed starting the EPS."}, - "200": { - "message": "EPS started successfully! It can take up to 45 minutes for the EPS to complete. When it is complete, the results can be found under 'Inspect results'." - }, - "429": { - "message": "It is currently busy on the server. We cannot start an EPS execution " - "at this time. Please try again at a later time. " - }, - }, - }, - }, - { - # 6 - "name": "EPS execution", - "description": "", - "label": "Select EPS execution to inspect:", - "type": "select-query", - "multiple": False, - "source": { - "url": f"{EPS_WEB_HOST}/api/eps/", - "http_method": "get", - "choices_attr": "executions", - "label_fields": ["project", "id", "success"], - "value_field": "id", - }, - "target_variable": "execution", - "next_step": 7, - }, - { - # 7 - "name": "Execution selected", - "description": "How would you like to proceed?", - "type": "choice", - "options": [ - { - "name": "Load EPS result", - "description": "Load the results as ESDL on the map. Your current layers will be overwritten!", - "type": "primary", - "enable_if_state": "execution.success", - "next_step": 10, - }, - { - "name": "Inspect EPS results", - "type": "default", - "enable_if_state": "execution.success", - "next_step": 11, - }, - { - "name": "Download project file", - "type": "default", - "next_step": 9, - }, - # { - # "name": "View progress", - # "type": "default", - # "disable_if_state": "execution.finished_on", - # "next_step": 8, - # }, - ], - }, - { - # 8 - "name": "EPS execution progress", - "description": "", - "type": "progress", - "refresh": 30, - "source": { - "url": f"{EPS_WEB_HOST}/api/eps/progress", - "request_params": {"execution_id": "execution.id"}, - "progress_field": "latest_progress", - "message_field": "latest_message", - }, - }, - { - # 9 - "name": "Download project file", - "description": "The project file that was used as input to perform this EPS analysis can be " - "downloaded here.", - "type": "download_file", - "source": { - "url": EPS_WEB_HOST + "/api/eps/{execution_id}/input", - "request_params": {"execution_id": "execution.id"}, - }, - }, - { - # 10 - "name": "Load EPS results", - "description": "Please wait a moment while we load an ESDL with the EPS results. When the EPS " - "is loaded, please continue.", - "type": "service", - "state_params": {"execution_id": "execution.id"}, - "service": { - "id": "9bd2f969-f240-4b26-ace5-2e03fbc04b12", - "name": "Visualize EPS", - "headers": { - "Accept": "application/esdl+xml", - "User-Agent": "ESDL Mapeditor/0.1", - }, - "url": f"{EPS_WEB_HOST}/api/eps//esdl", - "auto": True, - "clearLayers": True, - "http_method": "get", - "type": "", - "result": [{"code": 200, "action": "esdl"}], - "query_parameters": [ - { - "name": "File Name", - "location": "url", - "parameter_name": "execution_id", - } - ], - "with_jwt_token": True, - "state_params": True, - } - }, - { - # 11 - "name": "Inspect EPS results", - "description": "", - "type": "custom", - "component": "eps-inspect-result", - "custom_data": { - "url": EPS_WEB_HOST - + "/api/eps/{execution_id}/pand/{pand_bagid}" - }, - }, - { - # 12 - "name": "Aggregate ESDL buildings for ESSIM", - "description": "The ESDL generated by the EPS contains every building individually. This " - "causes unnecessarily long processing time if buildings have the same " - "profiles. With this service, similar buildings are combined into aggregated " - "buildings, which will make it faster to run ESSIM and easier to interpret " - "results. Note: This will create a new ESDL layer.", - "type": "service", - "service": { - "id": "9bd2f969-f240-4b26-ace5-2e03fbc04b13", - "name": "Aggregate ESDL buildings for ESSIM", - "headers": { - "Content-Type": "application/json", - "User-Agent": "ESDL Mapeditor/0.1", - }, - "type": "send_esdl_json", - "body": "base64_encoded", - "url": f"{ESDL_AGGREGATOR_HOST}/aggregate", - "http_method": "post", - "result": [{"code": 200, "action": "esdl"}], - "with_jwt_token": True, - }, - "next_step": 0, - }, - { - # 13 - "name": "Adjust EPS input", - "description": "", - "type": "choice", - "options": [ - { - "name": "Download project file", - "description": "If you want to inspect the KvK data or modify input data, you can download a project file.", - "next_step": 14, - "type": "primary", - }, - { - "name": "Upload project file", - "description": "A project file can be uploaded to serve as input for a new EPS run.", - "next_step": 2, - "type": "primary", - }, - ], - }, - { - # 14 - "name": "Select existing project", - "description": "Please select the project for which you would like to download a project file.", - "type": "select-query", - "multiple": False, - "source": { - "url": f"{EPS_WEB_HOST}/api/projects/", - "http_method": "get", - "choices_attr": "projects", - "label_fields": ["name"], - "value_field": "id", - }, - "target_variable": "project", - "next_step": 15, - }, - { - # 15 - "name": "Select project file", - "description": "Please select the project file you would like to download.", - "type": "select-query", - "multiple": False, - "source": { - "url": EPS_WEB_HOST + "/api/projects/{project_id}/files", - "request_params": {"project_id": "project.id"}, - "http_method": "get", - "choices_attr": "file_names", - "label_fields": ["name"], - "value_field": "id", - }, - "target_variable": "file_name", - "next_step": 16, - }, - { - # 16 - "name": "Download project file", - "description": "By clicking the button below you can download the selected project file.", - "type": "download_file", - "source": { - "url": EPS_WEB_HOST - + "/api/projects/{project_id}/files/{file_name}", - "request_params": { - "project_id": "project.id", - "file_name": "file_name.id", - }, - }, - "next_step": 0, - }, - ], - } - ] - }, -} diff --git a/esdl-mapeditor-config/nwn_dispatcher_config.template.json b/esdl-mapeditor-config/nwn_dispatcher_config.template.json deleted file mode 100644 index 14cb440..0000000 --- a/esdl-mapeditor-config/nwn_dispatcher_config.template.json +++ /dev/null @@ -1,264 +0,0 @@ -[ - { - "id": "123451a2-c92a-46ed-a7e4-993197712345", - "name": "Start Heat Network calculation", - "description": "Submit a heat network to the calculation service", - "headers": { - "User-Agent": "ESDL Mapeditor/1.0", - "Content-Type": "application/json" - }, - "url": "HEATNETWORK_DISPATCHER_URL/job/", - "http_method": "post", - "type": "send_esdl", - "send_email_in_post_body_parameter": "user_name", - "show_query_params": true, - "query_parameters": [ - { - "type": "string", - "name": "Scenario name", - "description": "Give this scenario a unique name", - "parameter_name": "job_name", - "location": "body" - }, - { - "type": "selection", - "name": "Workflow type", - "description": "Select the calculation type to execute", - "parameter_name": "work_flow_type", - "location": "body", - "possible_values": [ - "GROWTH_OPTIMIZER" - ] - } - ], - "body_config": { - "type": "json", - "parameters": [ - { - "type": "esdl", - "encoding": "base64_encoded", - "parameter": "input_esdl" - } - ] - }, - "body": "base64_encoded", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Calculation successfully submitted." - }, - { - "code": 404, - "action": "show_message", - "message": "Error in submitting calculation." - } - ], - "with_jwt_token": true, - "state_params": false - }, - { - "id": "x12c4a2b-8eee-46f7-a225-87c5f85e645f", - "name": "Heat Network calculation results", - "explanation": "This service allows you to view results from heat network simulations and optimizations", - "type": "vueworkflow", - "workflow": [ - { - "name": "Results overview", - "description": "", - "type": "table-query", - "source": { - "url": "HEATNETWORK_DISPATCHER_URL/job/user/{username}", - "http_method": "get", - "request_params": { - "username": "session.email" - }, - "columns": [ - { - "dataIndex": "job_name", - "title": "Name", - "ellipsis": true - }, - { - "dataIndex": "status", - "title": "Status", - "align": "center", - "slots": { - "customRender": "tags" - } - }, - { - "dataIndex": "stopped_at", - "title": "Finished at", - "formatter": "formatDate", - "defaultSortOrder": "descend" - } - ], - "actions": [ - { - "title": "Load result", - "next_step": 2 - }, - { - "title": "Show log", - "next_step": 3 - }, - { - "title": "Delete", - "next_step": 4 - } - ], - "value_field": "job_id" - }, - "target_variable": "job", - "with_jwt_token": true, - "next_step": 0 - }, - { - "name": "Result selected", - "state_params": { - "job_id": "job.job_id" - }, - "description": "How would you like to proceed?", - "type": "choice", - "options": [ - { - "name": "Load ESDL result", - "description": "Load the results as ESDL on the map.", - "type": "primary", - "next_step": 2 - }, - { - "name": "View log output", - "description": "View the log output of this execution.", - "type": "default", - "next_step": 3 - } - ] - }, - { - "name": "Load ESDL", - "description": "Loads the result in the ESDL MapEditor", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x1c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Load ESDL results", - "description": "The results should have been loaded in the MapEditor", - "button_label": "Go back to results", - "auto": true, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "HEATNETWORK_DISPATCHER_URL/job//result", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "esdl", - "encoding": "base64_encoded", - "json_field": "output_esdl" - } - ], - "with_jwt_token": true, - "state_params": true, - "next_step": 0 - } - }, - { - "name": "Show log file", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x5c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Request the log file", - "description": "The logfile is shown below:", - "button_label": "Go back", - "auto": true, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "HEATNETWORK_DISPATCHER_URL/job//logs", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Show log", - "json_field": "logs" - } - ], - "with_jwt_token": true, - "state_params": true, - "next_step": 0 - } - }, - { - "name": "Delete result", - "description": "Delete the selected result", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x9c9d1a2-c92a-46ed-a7e4-9931971cbb2e", - "name": "Delete the result", - "description": "Press the button below to delete the result", - "button_label": "Delete result", - "headers": { - "User-Agent": "ESDL Mapeditor/1.0", - "Content-Type": "application/json" - }, - "url": "HEATNETWORK_DISPATCHER_URL/job/", - "http_method": "delete", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Result succesfully removed" - } - ], - "with_jwt_token": true, - "state_params": true, - "next_step": 0 - } - } - ] - } -] \ No newline at end of file diff --git a/esdl-mapeditor-config/nwn_dispatcher_service.json b/esdl-mapeditor-config/nwn_dispatcher_service.json deleted file mode 100644 index 40ac024..0000000 --- a/esdl-mapeditor-config/nwn_dispatcher_service.json +++ /dev/null @@ -1,188 +0,0 @@ -[ - { - "id": "123451a2-c92a-46ed-a7e4-993197712345", - "name": "Start Heat Network calculation", - "description": "Submit a heat network to the calculation service", - "headers": { - "User-Agent": "ESDL Mapeditor/1.0", - "Content-Type": "application/json" - }, - "url": "http://localhost:9200/job", - "http_method": "post", - "type": "send_esdl", - "send_email_in_post_body_parameter": "user_name", - "show_query_params": true, - "query_parameters": [ - { - "type": "string", - "name": "Scenario name", - "description": "Give this scenario a unique name", - "parameter_name": "job_name", - "location": "body" - }, - { - "type": "selection", - "name": "Workflow type", - "description": "Select the calculation type to execute", - "parameter_name": "work_flow_type", - "location": "body", - "possible_values": [ - "GROWTH_OPTIMIZER" - ] - } - ], - "body_config": { - "type": "json", - "parameters": [ - { - "type": "esdl", - "encoding": "base64_encoded", - "parameter": "input_esdl" - } - ] - }, - "body": "base64_encoded", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Calculation successfully submitted." - }, - { - "code": 404, - "action": "show_message", - "message": "Error in submitting calculation." - } - ], - "with_jwt_token": false, - "state_params": false - }, - { - "id": "a12c4a2b-8eee-46f7-a225-87c5f85e645f", - "name": "View Heat Network optimization results", - "explanation": "This service allows you to view results from heat network simulations and optimizations", - "type": "vueworkflow", - "workflow": [ - { - "name": "Select result", - "description": "", - "type": "select-query", - "multiple": false, - "source": { - "url": "http://localhost:9200/job/", - "http_method": "get", - "label_fields": [ - "job_name", - "status", - "work_flow_type", - "stopped_at" - ], - "value_field": "job_id" - }, - "target_variable": "job", - "next_step": 1 - }, - { - "name": "Result selected", - "state_params": { - "job_id": "job.job_id" - }, - "description": "How would you like to proceed?", - "type": "choice", - "options": [ - { - "name": "Load ESDL result", - "description": "Load the results as ESDL on the map.", - "type": "primary", - "next_step": 2 - }, - { - "name": "View log output", - "description": "View the log output of this execution.", - "type": "default", - "next_step": 3 - } - ] - }, - { - "name": "Load ESDL", - "description": "Loads the result in the ESDL MapEditor", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "b4c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Load ESDL results", - "auto": true, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "http://localhost:9200/job//result", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "esdl", - "encoding": "base64_encoded", - "json_field": "output_esdl" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - }, - { - "name": "Show log file", - "description": "Show the log file", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "c4c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Request the log file", - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "http://localhost:9200/job//logs", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Show log", - "json_field": "logs" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - } - ] - } -] \ No newline at end of file diff --git a/esdl-mapeditor-config/nwn_dispatcher_service_table.json b/esdl-mapeditor-config/nwn_dispatcher_service_table.json deleted file mode 100644 index ff39439..0000000 --- a/esdl-mapeditor-config/nwn_dispatcher_service_table.json +++ /dev/null @@ -1,209 +0,0 @@ -[ - { - "id": "123451a2-c92a-46ed-a7e4-993197712345", - "name": "Start Heat Network calculation", - "description": "Submit a heat network to the calculation service", - "headers": { - "User-Agent": "ESDL Mapeditor/1.0", - "Content-Type": "application/json" - }, - "url": "http://localhost:9200/job", - "http_method": "post", - "type": "send_esdl", - "send_email_in_post_body_parameter": "user_name", - "show_query_params": true, - "query_parameters": [ - { - "type": "string", - "name": "Scenario name", - "description": "Give this scenario a unique name", - "parameter_name": "job_name", - "location": "body" - }, - { - "type": "selection", - "name": "Workflow type", - "description": "Select the calculation type to execute", - "parameter_name": "work_flow_type", - "location": "body", - "possible_values": [ - "GROWTH_OPTIMIZER" - ] - } - ], - "body_config": { - "type": "json", - "parameters": [ - { - "type": "esdl", - "encoding": "base64_encoded", - "parameter": "input_esdl" - } - ] - }, - "body": "base64_encoded", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Calculation successfully submitted." - }, - { - "code": 404, - "action": "show_message", - "message": "Error in submitting calculation." - } - ], - "with_jwt_token": false, - "state_params": false - }, - { - "id": "a12c4a2b-8eee-46f7-a225-87c5f85e645f", - "name": "View Heat Network optimization results", - "explanation": "This service allows you to view results from heat network simulations and optimizations", - "type": "vueworkflow", - "workflow": [ - { - "name": "Results overview", - "description": "", - "type": "table-query", - "source": { - "url": "http://localhost:9200/job/", - "http_method": "get", - "columns": [ - { - "dataIndex": "job_name", - "title": "Name", - "key": "job_name" - }, - { - "dataIndex": "status", - "title": "Status", - "key": "status" - }, - { - "dataIndex": "stopped_at", - "title": "Finished at", - "key": "stopped_at", - "formatter": "formatDate" - } - ], - "actions": [ - { - "title": "Load result", - "next_step": 2 - }, - { - "title": "Show log", - "next_step": 3 - } - ], - "value_field": "job_id" - }, - "target_variable": "job", - "next_step": 1 - }, - { - "name": "Result selected", - "state_params": { - "job_id": "job.job_id" - }, - "description": "How would you like to proceed?", - "type": "choice", - "options": [ - { - "name": "Load ESDL result", - "description": "Load the results as ESDL on the map.", - "type": "primary", - "next_step": 2 - }, - { - "name": "View log output", - "description": "View the log output of this execution.", - "type": "default", - "next_step": 3 - } - ] - }, - { - "name": "Load ESDL", - "description": "Loads the result in the ESDL MapEditor", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "b4c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Load ESDL results", - "auto": true, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "http://localhost:9200/job//result", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "esdl", - "encoding": "base64_encoded", - "json_field": "output_esdl" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - }, - { - "name": "Show log file", - "description": "Show the log file", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "c4c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Request the log file", - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "http://localhost:9200/job//logs", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Show log", - "json_field": "logs" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - } - ] - } -] \ No newline at end of file diff --git a/esdl-mapeditor-config/nwn_dispatcher_service_table_both.json b/esdl-mapeditor-config/nwn_dispatcher_service_table_both.json deleted file mode 100644 index 25091e7..0000000 --- a/esdl-mapeditor-config/nwn_dispatcher_service_table_both.json +++ /dev/null @@ -1,389 +0,0 @@ -[ - { - "id": "123451a2-c92a-46ed-a7e4-993197712345", - "name": "Start Heat Network calculation", - "explanation": "Submit a heat network to the calculation service", - "headers": { - "User-Agent": "ESDL Mapeditor/1.0", - "Content-Type": "application/json" - }, - "url": "http://localhost:9200/job/", - "http_method": "post", - "type": "send_esdl", - "send_email_in_post_body_parameter": "user_name", - "show_query_params": true, - "query_parameters": [ - { - "type": "string", - "name": "Scenario name", - "description": "Give this scenario a unique name", - "parameter_name": "job_name", - "location": "body" - }, - { - "type": "selection", - "name": "Workflow type", - "description": "Select the calculation type to execute", - "parameter_name": "work_flow_type", - "location": "body", - "possible_values": [ - "GROWTH_OPTIMIZER" - ] - } - ], - "body_config": { - "type": "json", - "parameters": [ - { - "type": "esdl", - "encoding": "base64_encoded", - "parameter": "input_esdl" - } - ] - }, - "body": "base64_encoded", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Calculation successfully submitted." - }, - { - "code": 404, - "action": "show_message", - "message": "Error in submitting calculation." - } - ], - "with_jwt_token": false, - "state_params": false - }, - { - "id": "a12c4a2b-8eee-46f7-a225-87c5f85e645f", - "name": "View Heat Network optimization results", - "explanation": "This service allows you to view results from heat network simulations and optimizations", - "type": "vueworkflow", - "workflow": [ - { - "name": "Select result", - "description": "", - "type": "select-query", - "multiple": false, - "source": { - "url": "http://localhost:9200/job/", - "http_method": "get", - "label_fields": [ - "job_name", - "status", - "work_flow_type", - "stopped_at" - ], - "value_field": "job_id" - }, - "target_variable": "job", - "next_step": 1 - }, - { - "name": "Result selected", - "state_params": { - "job_id": "job.job_id" - }, - "description": "How would you like to proceed?", - "type": "choice", - "options": [ - { - "name": "Load ESDL result", - "description": "Load the results as ESDL on the map.", - "type": "primary", - "next_step": 2 - }, - { - "name": "View log output", - "description": "View the log output of this execution.", - "type": "default", - "next_step": 3 - } - ] - }, - { - "name": "Load ESDL", - "description": "Loads the result in the ESDL MapEditor", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "b4c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Load ESDL results", - "auto": true, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "http://localhost:9200/job//result", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "esdl", - "encoding": "base64_encoded", - "json_field": "output_esdl" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - }, - { - "name": "Show log file", - "description": "Show the log file", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "c4c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Request the log file", - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "http://localhost:9200/job//logs", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Show log", - "json_field": "logs" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - } - ] - }, - { - "id": "x12c4a2b-8eee-46f7-a225-87c5f85e645f", - "name": "View Heat Network optimization results (table)", - "explanation": "This service allows you to view results from heat network simulations and optimizations", - "type": "vueworkflow", - "workflow": [ - { - "name": "Results overview", - "description": "", - "type": "table-query", - "source": { - "url": "http://localhost:9200/job/user/{username}", - "http_method": "get", - "request_params": {"username": "session.email"}, - "columns": [ - { - "dataIndex": "job_name", - "title": "Name", - "ellipsis": true - }, - { - "dataIndex": "status", - "title": "Status", - "align": "center", - "slots": { - "customRender": "tags" - } - }, - { - "dataIndex": "stopped_at", - "title": "Finished at", - "formatter": "formatDate", - "defaultSortOrder": "descend" - } - ], - "actions": [ - { - "title": "Load result", - "next_step": 2 - }, - { - "title": "Show log", - "next_step": 3 - }, - { - "title": "Delete", - "next_step": 4 - } - ], - "value_field": "job_id" - }, - "target_variable": "job", - "next_step": 0 - }, - { - "name": "Result selected", - "state_params": { - "job_id": "job.job_id" - }, - "description": "How would you like to proceed?", - "type": "choice", - "options": [ - { - "name": "Load ESDL result", - "description": "Load the results as ESDL on the map.", - "type": "primary", - "next_step": 2 - }, - { - "name": "View log output", - "description": "View the log output of this execution.", - "type": "default", - "next_step": 3 - } - ] - }, - { - "name": "Load ESDL", - "description": "Loads the result in the ESDL MapEditor", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x1c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Load ESDL results", - "description": "The results should have been loaded in the MapEditor", - "button_label": "Go back to results", - "auto": true, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "http://localhost:9200/job//result", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "esdl", - "encoding": "base64_encoded", - "json_field": "output_esdl" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - }, - { - "name": "Show log file", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x5c9d1a2-c92a-46ed-a7e4-9931971cbb27", - "name": "Request the log file", - "description": "The logfile is shown below:", - "button_label": "Go back", - "auto": true, - "headers": { - "User-Agent": "ESDL Mapeditor/0.1", - "Content-Type": "application/xml" - }, - "url": "http://localhost:9200/job//logs", - "http_method": "get", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Show log", - "json_field": "logs" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - }, - { - "name": "Delete result", - "description": "Delete the selected result", - "type": "service", - "state_params": { - "job_id": "job.job_id" - }, - "service": { - "id": "x9c9d1a2-c92a-46ed-a7e4-9931971cbb2e", - "name": "Delete the result", - "description": "Press the button below to delete the result", - "button_label": "Delete result", - "headers": { - "User-Agent": "ESDL Mapeditor/1.0", - "Content-Type": "application/json" - }, - "url": "http://localhost:9200/job/", - "http_method": "delete", - "type": "", - "query_parameters": [ - { - "name": "Job id", - "description": "Job id", - "parameter_name": "job_id", - "location": "url" - } - ], - "body": "", - "result": [ - { - "code": 200, - "action": "show_message", - "message": "Result succesfully removed" - } - ], - "with_jwt_token": false, - "state_params": true, - "next_step": 0 - } - } - ] - } -] \ No newline at end of file diff --git a/esdl-mapeditor-config/settings.py b/esdl-mapeditor-config/settings.py deleted file mode 100644 index ee51ab5..0000000 --- a/esdl-mapeditor-config/settings.py +++ /dev/null @@ -1,128 +0,0 @@ -# This work is based on original code developed and copyrighted by TNO 2020. -# Subsequent contributions are licensed to you by the developers of such code and are -# made available to the Project under one or several contributor license agreements. -# -# This work is licensed to you under the Apache License, Version 2.0. -# You may obtain a copy of the license at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Contributors: -# TNO - Initial implementation -# Manager: -# TNO - -import os -from dotenv import load_dotenv - -load_dotenv() -print('Starting application with {} settings'.format(os.environ.get("SETTINGS_TITLE", "NO"))) - -# Flask settings -FLASK_SERVER_HOST = '0.0.0.0' -FLASK_SERVER_PORT = 8111 -_flask_debug = os.environ.get('FLASK_DEBUG', '') # Do not use debug mode in production -FLASK_DEBUG = (_flask_debug.upper() == 'TRUE' or _flask_debug == '1') - -USER_LOGGING_ENABLED = True -# ASYNC_MODE = 'gevent_uwsgi' -ASYNC_MODE = None # default for non uWGSI deployments - -OIDC_CLIENT_SECRETS = os.environ.get('OIDC_CLIENT_SECRETS', None) # 'credentials/client_secrets_opensource.json' -_use_gevent = os.environ.get('MAPEDITOR_USE_GEVENT', '') -USE_GEVENT = (_use_gevent.upper() == 'TRUE' or _use_gevent == '1') - -settings_storage_config = { - "host": os.environ.get('SETTINGS_STORAGE_HOST', None), # "mongo", - "port": os.environ.get('SETTINGS_STORAGE_PORT', "27017"), - "database": "esdl_mapeditor_settings" -} - -user_logging_config = { - "host": os.environ.get('USER_LOGGING_HOST', None), # "influxdb", - "port": os.environ.get('USER_LOGGING_PORT', "8086"), - "database": "user_logging" -} - -boundaries_config = { - "host": os.environ.get('BOUNDARY_SERVICE_HOST', None), # "boundary-service", - "port": os.environ.get('BOUNDARY_SERVICE_PORT', None), # "4002", - "path_names": "/names", - "path_boundaries": "/boundaries" -} - -profile_database_config = { - "protocol": "http", - "host": os.environ.get('PROFILE_DATABASE_HOST', None), # "influxdb", - "port": os.environ.get('PROFILE_DATABASE_PORT', "8086"), - "database": "energy_profiles", - "filters": "", - "upload_user": "admin", - "upload_password": "admin" -} - -panel_service_config = { - "external_url": os.environ.get('PANEL_SERVICE_EXTERNAL_URL', None), # "http://localhost:3400", - "internal_url": os.environ.get('PANEL_SERVICE_INTERNAL_URL', None), # "http://panel-service:5000" - "profile_database_protocol": "http", - "profile_database_host": os.environ.get('PANEL_SERVICE_PROFILE_DB_HOST', None), - "profile_database_port": os.environ.get('PANEL_SERVICE_PROFILE_DB_PORT', "8086"), - "profile_database_upload_user": "admin", - "profile_database_upload_password": "admin" -} - -essim_config = { - "ESSIM_host": os.environ.get('ESSIM_URL', None), # "http://essim-engine:8112", - "ESSIM_host_loadflow": os.environ.get('ESSIM_LOADFLOW_URL', None), # "", - "ESSIM_path": "/essim/simulation", - "influxURL": os.environ.get('ESSIM_INFLUX_URL', None), # "http://influxdb:8086", - "grafanaURL": os.environ.get('ESSIM_GRAFANA_URL', None), # "http://grafana:3000", - "user": "essim", - "ESSIM_database_server": os.environ.get('ESSIM_DATABASE_HOST', None), # "influxdb", - "ESSIM_database_port": os.environ.get('ESSIM_DATABASE_PORT', 8086), - "start_datetime": "2015-01-01T00:00:00+0100", - "end_datetime": "2016-01-01T00:00:00+0100", - "natsURL": "nats://nats:4222" -} - -edr_config = { - "host": os.environ.get('EDR_URL', None), # "https://edr.hesi.energy", -} - -esdl_store_config = { - "hostname": os.environ.get('ESDL_STORE_URL', None) # None -} -mondaine_hub_config = { - "hostname": os.environ.get('MONDAINE_HUB_URL', None) # None -} -esdl_drive_config = { - "hostname": os.environ.get('ESDL_DRIVE_URL', None) # "http://esdl-drive:9080" -} - -ibis_config = { - "host": os.environ.get('IBIS_SERVICE_HOST', None), - "port": "4500", - "path_list": "/api/v1/BusinessParks/", - "path_contour": "/api/v1/BusinessParks/contour/" -} - -bag_config = { - "host": os.environ.get('BAG_SERVICE_HOST', None), - "port": "4012", - "path_contour": "/polygon" -} - -statistics_settings_config = { - "host": os.environ.get('STATISTICS_SERVICE_HOST', None), - "port": "6003", - "path": "/api/statistics/calculate" -} - -ielgas_config = { - "host": os.environ.get('IELGAS_DATABASE_HOST', None) -} - -heatnetwork_dispatcher_config = { - "host": os.environ.get('DISPATCHER_HOST', 'http://localhost'), - "port": os.environ.get('DISPATCHER_PORT', 9200) -} \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index bb88d61..f6e004f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -8,7 +8,7 @@ aio-pika==9.4.3 # via omotes-sdk-python aiormq==6.8.1 # via aio-pika -alembic==1.13.2 +alembic==1.13.3 # via omotes-rest (pyproject.toml) amqp==5.2.0 # via kombu @@ -61,7 +61,7 @@ kombu==5.4.2 # via celery mako==1.3.5 # via alembic -markupsafe==2.1.5 +markupsafe==3.0.1 # via # jinja2 # mako @@ -82,7 +82,7 @@ multidict==6.1.0 # via yarl mypy-extensions==1.0.0 # via typing-inspect -omotes-sdk-protocol==0.1.5 +omotes-sdk-protocol==0.1.6 # via omotes-sdk-python omotes-sdk-python==3.1.2 # via omotes-rest (pyproject.toml) @@ -96,8 +96,10 @@ pamqp==3.3.0 # via # aiormq # omotes-sdk-python -prompt-toolkit==3.0.47 +prompt-toolkit==3.0.48 # via click-repl +propcache==0.2.0 + # via yarl protobuf==4.25.5 # via omotes-sdk-protocol psycopg2-binary==2.9.9 @@ -124,7 +126,7 @@ typing-extensions==4.11.0 # typing-inspect typing-inspect==0.9.0 # via marshmallow-dataclass -tzdata==2024.1 +tzdata==2024.2 # via # celery # kombu @@ -141,7 +143,7 @@ werkzeug==3.0.4 # via # flask # flask-smorest -yarl==1.11.1 +yarl==1.15.2 # via # aio-pika # aiormq diff --git a/scripts/create_dev_environment.sh b/scripts/create_dev_environment.sh deleted file mode 100755 index abe9961..0000000 --- a/scripts/create_dev_environment.sh +++ /dev/null @@ -1,5 +0,0 @@ -#!/bin/bash - -python3.10 -m venv ./.venv/ -. ./.venv/bin/activate -pip install pip-tools diff --git a/scripts/install_dependencies.sh b/scripts/db_models_generate_new_revision.sh similarity index 72% rename from scripts/install_dependencies.sh rename to scripts/db_models_generate_new_revision.sh index 1dbaade..daa7d58 100755 --- a/scripts/install_dependencies.sh +++ b/scripts/db_models_generate_new_revision.sh @@ -5,4 +5,5 @@ if [[ "$OSTYPE" != "win32" && "$OSTYPE" != "msys" ]]; then . .venv/bin/activate fi -pip-sync ./dev-requirements.txt ./requirements.txt +cd src/ +alembic revision --autogenerate -m "$1" \ No newline at end of file diff --git a/scripts/update_dependencies.sh b/scripts/update_dependencies.sh deleted file mode 100755 index c173025..0000000 --- a/scripts/update_dependencies.sh +++ /dev/null @@ -1,8 +0,0 @@ -#!/bin/bash - -if [[ "$OSTYPE" != "win32" && "$OSTYPE" != "msys" ]]; then - . .venv/bin/activate -fi - -pip-compile -U --output-file=requirements.txt pyproject.toml -pip-compile -U --extra=dev -c requirements.txt --output-file=dev-requirements.txt pyproject.toml