✨ dynamic-scheduler pushes service state to the frontend (⚠️ devops)

[GitHK]

⚠️ devops

Merger the following after releasing to:

• master
• staging
• production

What do these changes do?

Service status is monitored by dynamic-scheduler. Whenever the status changes it is pushed via websocket to the fronted as a message called serviceStatus.

• both "legacy" and "new style" dynamic services are monitored and their state is propagated to the front-end
• this monitoring process is independent from director-v2's service tracking and does not interfere!
• this same monitoring process will also be the basis for all actions that the dynamic-scheduler will take based on the change of the state of the dynamic services.

Other changes:

• ♻️ get_dict_from_status shared between services is used to serialise the status to the FE
• ✨ added new Redis database for tracking the service status
• ♻️ RedisClientSDK now supports decode_response, which disables automatic encoding and decoding of bytes to strings, thus supporting raw bytes to be stored and recovered.

TODOs:

• test legacy service propagates to FE
• Updated specs for redis-commander osparc-ops-environments#761 should be merged before merging this PR

Related issue/s

• requires ⬆️ upgrading faststream repo wide #5898
• requires ♻️ refactor stop service command ⚠️🚨 #5924
• requires ♻️ refactor RedisClientsManager and RedisClientSDK #5888
• closes Service state driven by the backend osparc-issues#1175

How to test

Dev-ops checklist

• No ENV changes or I properly updated ENV (read the instruction)

[codecov]

Codecov Report

Attention: Patch coverage is 94.88372% with 22 lines in your changes missing coverage. Please review.

Project coverage is 88.2%. Comparing base (cafbf96) to head (a47f788).
Report is 575 commits behind head on master.

Files with missing lines	Patch %	Lines
...dynamic_scheduler/services/service_tracker/_api.py	88.5%	13 Missing ⚠️
...s/service-library/src/servicelib/services_utils.py	0.0%	4 Missing ⚠️
...e_dynamic_scheduler/services/notifier/_notifier.py	90.3%	3 Missing ⚠️
...er/services/status_monitor/_deferred_get_status.py	97.9%	1 Missing ⚠️
...amic_scheduler/services/status_monitor/_monitor.py	98.1%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #5892      +/-   ##
=========================================
+ Coverage    84.5%   88.2%    +3.6%     
=========================================
  Files          10    1520    +1510     
  Lines         214   62612   +62398     
  Branches       25    2023    +1998     
=========================================
+ Hits          181   55272   +55091     
- Misses         23    7028    +7005     
- Partials       10     312     +302     

Flag	Coverage Δ
integrationtests	64.7% <50.0%> (?)
unittests	86.2% <94.8%> (+1.6%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
..._library/api_schemas_dynamic_scheduler/socketio.py	100.0% <100.0%> (ø)
...ls_library/api_schemas_webserver/projects_nodes.py	97.0% <ø> (ø)
...odels-library/src/models_library/services_enums.py	78.7% <100.0%> (ø)
...c/servicelib/deferred_tasks/_redis_task_tracker.py	100.0% <100.0%> (ø)
...ges/settings-library/src/settings_library/redis.py	95.8% <100.0%> (ø)
...ervice_dynamic_scheduler/api/rest/_dependencies.py	100.0% <100.0%> (ø)
...core_service_dynamic_scheduler/api/rest/_health.py	100.0% <100.0%> (ø)
...ore_service_dynamic_scheduler/api/rpc/_services.py	100.0% <100.0%> (ø)
...core_service_dynamic_scheduler/core/application.py	100.0% <100.0%> (ø)
...ice_dynamic_scheduler/services/deferred_manager.py	100.0% <100.0%> (ø)
... and 16 more

... and 1445 files with indirect coverage changes

[sanderegg]

all good. a few comments left. we can discuss tomorrow if you want.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud