Refactor osparc stack env-var handling

[mrnicegyu11]

• @pcrespov : Dump currently used env-vars for local oSparc [list of unset & total list] .
  • json-schemas of all service settings are produced as artifacts system-test-swarm-deploy_services_settings_schemas.zip on every build. SEE gh-actions in master.
• @mrnicegyu11: Add osparc.local full variable configuration (including what is now known as optional / default values) for all deployments. repo.config of a deployment overwrites the (osparc.local) defaults.
• @pcrespov : Change: All oSparc services fail if variables are unset
  • service fails to start if faulty config
  • adds a warning if settings field defaults to null (prevents situations in which typos in env-vars lead to null plugin settings that lead to shutting down plugins)
• @mrnicegyu11 Remove env_file: - ../.env from services/docker-compose.yml in osparc-simcore and capture every single necessary env-var in the environment: section

Tasks

Preview Give feedback

1. 🔨 Is3635/service settings jsonschemas #3703
   t:maintenance +1
   
2. Maintenance: fix/add settings command in some of the services's CLI #3924
   5 of 5
   a:infra+ops a:services-library t:enhancement t:maintenance +4
   

   
3. Ensure Simcore Webserver is given only necessary env variables osparc-ops-environments#268
   p:mid-prio t:enhancement +2
   

[pcrespov]

@mrnicegyu11 , my part is in principle done here. Let me know if you need extra from me.

[mrnicegyu11]

thanks a lot

[mrnicegyu11]

See ITISFoundation/osparc-ops-environments#177

[pcrespov]

[GitHK]

I have some questions as a developer. When dealing with env vars, I usually end up with in the following situations.

1. I define and env var in the settings of a service for internal usage. Does not require OPS value. I'd say that I will not expose/define it somewhere else. We have lots of these already
2. I define an env var in the settings of a service and it is mandatory. This means that it requires a value from OPS.
   • Q1 where do I define it exacly?
   • Q2 where do I put my default for development?
3. I define an env var in the settings of a service but it has a default in the code.
   • Q3 in theory I don't have to do anything else, do I still have to define it?
   • Q4 where do I put my default for development?

@pcrespov @sanderegg @YuryHrytsuk @mrnicegyu11 @matusdrobuliak66 thanks for you help in clearing this up.

[mrnicegyu11]

re 1: ideally, no env-var should have defaults. So, to do it properly in this scenario, you should always NOT allow defaults and actually set the desired value as (what you call) "OPS value", always, even if it is considered internal usage.
re 2: This is the normal case. You put it in according to the docs made by @YuryHrytsuk . Your development default for now goes in the .env-devel of osparc-simcore
re 3: This should never happen, default values are not good practice.

@YuryHrytsuk please comment what I missed :--)

[sanderegg]

@GitHK
My understanding:

1. not sure what is the point of having ENV variables for internal usage. what does this mean? why not just create some constant in the code then?

1. you need to define it in the osparc-simcore/services/docker-compose.yml file (such as MY_VAR=${MY_VAR})
2. you need to define it in .env-devel so that developers can work, and that osparc-simcore is **self-consistent**
3. you need to define that value in gitlab osparc-ops-environment-configuration (or whatever its name becomes @mrnicegyu11 @YuryHrytsuk ), in all the subfolders for all the deployments

3. see 2. even if you define a default, please define it otherwise

[mrnicegyu11]

This is done