⚗️ ♻️ improving e2e tests (adding structure for additional CI tests)

[matusdrobuliak66]

What do these changes do?

• As part of the system tests, we are already running e2e-playwright tests.
• Currently (thanks to Sylvain's great work), these tests are executed in the CI whenever someone creates a pull request. They are implemented as follows:
  • make install-ci-up-simcore. This command:
    • Starts the local registry.
    • Pulls and pushes the sleeper image to the local registry.
    • Runs the entire SimCore stack.
    • Grants service access rights in the database.
  • Then we run make test-sleepers
• My current improvements are:
  • Additionally, we run make test-platform.
  • Here, we can add additional small frontend tests that we want to include as part of the CI process.
    • NOTE: I will modify these tests, when osparc-testing-id will be added on the new UI elements (folders and workspaces) -> UPDATE: tests modified based on these IDs 🎨 [e2e][Frontend] Folders and Workspaces related test-ids #6766
  • The test is designed to reuse the same browser context, so you only need to log in/out once, which speeds things up.
  • My idea is that, with this approach, it will force us to prepare both setup and teardown in the code (similar to unit tests). One of the next steps could be to add an SQL file to inject data for these tests (NOTE: this file will need to be maintained).

Disclaimer: This is just an experiment. If we find that it is not useful, we do not need to continue. However, I would like to give it a try.

Related issue/s

How to test

Dev-ops checklist

• No ENV changes or I properly updated ENV (read the instruction)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.53%. Comparing base (793318d) to head (7566445).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6765      +/-   ##
==========================================
+ Coverage   88.11%   88.53%   +0.42%     
==========================================
  Files        1512     1542      +30     
  Lines       60601    62208    +1607     
  Branches     2125     2125              
==========================================
+ Hits        53398    55078    +1680     
+ Misses       6875     6802      -73     
  Partials      328      328              

Flag	Coverage Δ
integrationtests	64.87% <ø> (+11.35%)	⬆️
unittests	86.46% <ø> (+0.12%)	⬆️

Components	Coverage Δ
api	∅ <ø> (∅)
pkg_aws_library	93.38% <ø> (ø)
pkg_dask_task_models_library	96.87% <ø> (ø)
pkg_models_library	91.93% <ø> (ø)
pkg_notifications_library	83.79% <ø> (ø)
pkg_postgres_database	87.33% <ø> (ø)
pkg_service_integration	71.44% <ø> (ø)
pkg_service_library	76.48% <ø> (ø)
pkg_settings_library	91.42% <ø> (ø)
pkg_simcore_sdk	85.27% <ø> (ø)
agent	96.98% <ø> (ø)
api_server	89.88% <ø> (ø)
autoscaling	95.22% <ø> (ø)
catalog	89.42% <ø> (ø)
clusters_keeper	98.72% <ø> (ø)
dask_sidecar	91.32% <ø> (ø)
datcore_adapter	94.05% <ø> (ø)
director	75.99% <ø> (ø)
director_v2	90.91% <ø> (ø)
dynamic_scheduler	96.59% <ø> (ø)
dynamic_sidecar	89.78% <ø> (ø)
efs_guardian	90.01% <ø> (ø)
invitations	93.51% <ø> (ø)
osparc_gateway_server	85.15% <ø> (ø)
payments	92.87% <ø> (ø)
resource_usage_tracker	90.80% <ø> (+0.14%)	⬆️
storage	89.76% <ø> (∅)
webclient	∅ <ø> (∅)
webserver	88.75% <ø> (+1.16%)	⬆️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 793318d...7566445. Read the comment docs.

---- 🚨 Try these New Features:

• JS Bundle Analysis - Avoid shipping oversized bundles

[sanderegg]

thanks!

[bisgaard-itis]

Cool! Nice initiative! 👍🏻

[GitHK]

👍

[pcrespov]

I generally agree with the idea, but here are a few suggestions to consider:

1. API Calls for Configuration Checks: Instead of testing configurations through the front-end, you might want to use straightforward API calls like /config, /statics.json, /status, or /diagnostics. These endpoints typically provide metadata and deployment state information directly.

2. Dynamic Badge API: Consider adding an API endpoint to generate a dynamic badge, as demonstrated here. This could be used to create a Markdown table with badges, effectively serving as a dashboard for all deployments.

3. Use Docker Volume Mounts: I’d suggest relying on Docker volume mounts from existing deployments rather than maintaining SQL scripts to inject data. We’ve tried the SQL script approach in the past (some of which may still be in the repo), but the maintenance overhead became significant, especially with frequent database schema changes.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud