⬆️ Upgrade Web Server service (Pydantic v2) #6584

giancarloromeo · 2024-10-23T11:51:51Z

What do these changes do?

Upgrade the Web Server service to Pydantic v2.

Related issue/s

How to test

make devenv
source .venv/bin/activate
cd services/web/server
make install-dev
make mypy
make test-dev-unit

Dev-ops checklist

No ENV changes or I properly updated ENV (read the instruction)

codecov · 2024-10-23T13:50:06Z

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 68.83%. Comparing base (24218c1) to head (c7d5f0c).
Report is 2 commits behind head on pydantic_v2_migration_do_not_squash_updates.

Additional details and impacted files

@@                               Coverage Diff                               @@
##           pydantic_v2_migration_do_not_squash_updates    #6584      +/-   ##
===============================================================================
- Coverage                                        71.63%   68.83%   -2.81%     
===============================================================================
  Files                                             1100      954     -146     
  Lines                                            49250    42328    -6922     
  Branches                                          1390     1300      -90     
===============================================================================
- Hits                                             35281    29136    -6145     
+ Misses                                           13736    12968     -768     
+ Partials                                           233      224       -9

Flag	Coverage Δ		*Carryforward flag
integrationtests	`65.02% <ø> (+0.24%)`	⬆️	Carriedforward from 399abc8
unittests	`77.96% <0.00%> (-4.98%)`	⬇️

*This pull request uses carry forward flags. Click here to find out more.

Components	Coverage Δ
api	`∅ <ø> (∅)`
pkg_aws_library	`93.32% <ø> (ø)`
pkg_dask_task_models_library	`96.61% <ø> (ø)`
pkg_models_library	`∅ <ø> (∅)`
pkg_notifications_library	`83.79% <ø> (ø)`
pkg_postgres_database	`∅ <ø> (∅)`
pkg_service_integration	`70.00% <ø> (ø)`
pkg_service_library	`76.51% <0.00%> (ø)`
pkg_settings_library	`∅ <ø> (∅)`
pkg_simcore_sdk	`77.41% <ø> (ø)`
agent	`97.22% <ø> (ø)`
api_server	`∅ <ø> (∅)`
autoscaling	`∅ <ø> (∅)`
catalog	`∅ <ø> (∅)`
clusters_keeper	`∅ <ø> (∅)`
dask_sidecar	`∅ <ø> (∅)`
datcore_adapter	`93.17% <ø> (ø)`
director	`58.38% <ø> (ø)`
director_v2	`76.21% <ø> (ø)`
dynamic_scheduler	`∅ <ø> (∅)`
dynamic_sidecar	`59.64% <ø> (ø)`
efs_guardian	`90.10% <ø> (ø)`
invitations	`93.49% <ø> (ø)`
osparc_gateway_server	`79.52% <ø> (ø)`
payments	`∅ <ø> (∅)`
resource_usage_tracker	`87.51% <ø> (ø)`
storage	`∅ <ø> (∅)`
webclient	`∅ <ø> (∅)`
webserver	`59.88% <ø> (+0.18%)`	⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 24218c1...c7d5f0c. Read the comment docs.

…1/upgrade-web-server-service

…rloromeo/osparc-simcore into is4481/upgrade-web-server-service

sanderegg

great job! thanks!

packages/common-library/src/common_library/pydantic_validators.py

packages/models-library/src/models_library/api_schemas_directorv2/clusters.py

packages/models-library/src/models_library/api_schemas_webserver/auth.py

packages/models-library/src/models_library/api_schemas_webserver/clusters.py

packages/models-library/src/models_library/clusters.py

packages/models-library/src/models_library/resource_tracker.py

sanderegg · 2024-11-05T07:25:20Z

packages/models-library/src/models_library/rest_pagination_utils.py

    last_page = ceil(total / limit) - 1

+    data = [item.model_dump() if hasattr(item, "model_dump") else item for item in chunk]
+
    return PageDict(
        _meta=PageMetaInfoLimitOffset(
-            total=total, count=len(chunk), limit=limit, offset=offset
+            total=total, count=len(data), limit=limit, offset=offset
        ),


not sure I understand that change. chunk is a list, so why does it need a change?

packages/pytest-simcore/src/pytest_simcore/hypothesis_type_strategies.py

services/web/server/src/simcore_service_webserver/api_keys/_db.py

…rv2/clusters.py Co-authored-by: Sylvain <[email protected]>

GitHK

I have some concerns that might cause issues. Please see them below.

packages/common-library/src/common_library/pydantic_validators.py

packages/models-library/src/models_library/api_schemas_directorv2/clusters.py

packages/models-library/src/models_library/api_schemas_webserver/projects.py

services/web/server/src/simcore_service_webserver/dynamic_scheduler/settings.py

services/web/server/src/simcore_service_webserver/payments/_methods_db.py

services/web/server/src/simcore_service_webserver/studies_dispatcher/_catalog.py

services/web/server/src/simcore_service_webserver/users/_schemas.py

packages/models-library/src/models_library/api_schemas_directorv2/clusters.py

matusdrobuliak66

Thanks!!

packages/models-library/src/models_library/api_schemas_webserver/auth.py

services/web/server/src/simcore_service_webserver/announcements/_models.py

pcrespov · 2024-11-05T12:37:39Z

services/web/server/src/simcore_service_webserver/application_settings.py

@@ -54,7 +63,7 @@ class ApplicationSettings(BaseCustomSettings, MixinLoggingSettings):
    # CODE STATICS ---------------------------------------------------------
    API_VERSION: str = API_VERSION
    APP_NAME: str = APP_NAME
-    API_VTAG: VersionTag = parse_obj_as(VersionTag, API_VTAG)
+    API_VTAG: VersionTag = TypeAdapter(VersionTag).validate_python(API_VTAG)


MINOR: in next PRs we can refactor all ApplicationSettings to inherit from settings_library.application.BaseApplicationSettings. That base class includes all env vars expected from the Dockerfile

services/web/server/src/simcore_service_webserver/application_settings.py

services/web/server/src/simcore_service_webserver/cli.py

packages/common-library/src/common_library/pydantic_validators.py

packages/postgres-database/src/simcore_postgres_database/models/products.py

services/web/server/src/simcore_service_webserver/director_v2/_models.py

services/web/server/src/simcore_service_webserver/dynamic_scheduler/settings.py

services/web/server/src/simcore_service_webserver/projects/_tags_api.py

services/web/server/src/simcore_service_webserver/security/_authz_db.py

services/web/server/src/simcore_service_webserver/studies_dispatcher/_catalog.py

services/web/server/src/simcore_service_webserver/users/_schemas.py

services/web/server/src/simcore_service_webserver/utils_aiohttp.py

sonarqubecloud · 2024-11-06T08:03:11Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

upgrade reqs

956c3b7

giancarloromeo changed the base branch from master to pydantic_v2_migration_do_not_squash_updates October 23, 2024 11:52

giancarloromeo self-assigned this Oct 23, 2024

giancarloromeo added a:webserver issue related to the webserver service t:maintenance Some planned maintenance work labels Oct 23, 2024

giancarloromeo added this to the MartinKippenberger milestone Oct 23, 2024

giancarloromeo added 4 commits October 23, 2024 15:31

run bump-pydantic

45119e5

add common-library dependency

9979a3f

fix import

29ab6c2

fix import

a75e832

giancarloromeo added 13 commits October 23, 2024 15:55

continue fixing

dc1f60d

fix arbitrary type field

fdb7238

continue fixing

c1db89a

continue fixing

9586ea8

continue fixing

e7ba88a

fix settings validator

c5dd024

continue fixing

1e97ead

remove deprecated

82ba4bd

fix model_config attr

be74c41

remove deprecated

8015258

continue fixing

44d97de

continue fixing

cf01290

continue fixing

2b12826

giancarloromeo mentioned this pull request Oct 24, 2024

Major upgrade to pydantic 2.0 #4481

Closed

giancarloromeo added 5 commits October 24, 2024 14:51

fix deprecated dict()

a50f41a

fix deprecated json()

6b9d7fa

Merge branch 'pydantic_v2_migration_do_not_squash_updates' into is448…

557f51d

…1/upgrade-web-server-service

fix type

335ff84

Merge branch 'is4481/upgrade-web-server-service' of github.com:gianca…

c7958b0

…rloromeo/osparc-simcore into is4481/upgrade-web-server-service

giancarloromeo requested review from GitHK and matusdrobuliak66 as code owners November 4, 2024 23:19

giancarloromeo requested a review from bisgaard-itis November 4, 2024 23:19

sanderegg approved these changes Nov 5, 2024

View reviewed changes

Update packages/models-library/src/models_library/api_schemas_directo…

ded3b25

…rv2/clusters.py Co-authored-by: Sylvain <[email protected]>

GitHK reviewed Nov 5, 2024

View reviewed changes

packages/models-library/src/models_library/api_schemas_directorv2/clusters.py Show resolved Hide resolved

matusdrobuliak66 approved these changes Nov 5, 2024

View reviewed changes

giancarloromeo added 6 commits November 5, 2024 10:26

remove pattern

bbe8ea2

revert nullable

509a7ac

remove constraint

f6e2128

fix model_dump_json option

0034952

fix fields

9db58f7

fix

d292788

pcrespov reviewed Nov 5, 2024

View reviewed changes

giancarloromeo added 3 commits November 5, 2024 13:58

remove frozen

59a64dc

fix jsondump

9a1bc00

minor fix

f24cd19

pcrespov reviewed Nov 5, 2024

View reviewed changes

giancarloromeo added 8 commits November 5, 2024 16:21

fix comments

4634acb

fix nullable

c74b2b2

fix enum

7b7d020

fix schema

274e2cb

revert none

8aac73d

remove ignore

399abc8

revert gid cast

e1b8401

fix comment

c7d5f0c

giancarloromeo merged commit 8402270 into ITISFoundation:pydantic_v2_migration_do_not_squash_updates Nov 6, 2024
52 of 74 checks passed

giancarloromeo deleted the is4481/upgrade-web-server-service branch November 6, 2024 08:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⬆️ Upgrade Web Server service (Pydantic v2) #6584

⬆️ Upgrade Web Server service (Pydantic v2) #6584

giancarloromeo commented Oct 23, 2024 •

edited

Loading

codecov bot commented Oct 23, 2024 •

edited

Loading

sanderegg left a comment

sanderegg Nov 5, 2024

GitHK left a comment

matusdrobuliak66 left a comment

pcrespov Nov 5, 2024

sonarqubecloud bot commented Nov 6, 2024

⬆️ Upgrade Web Server service (Pydantic v2) #6584

⬆️ Upgrade Web Server service (Pydantic v2) #6584

Conversation

giancarloromeo commented Oct 23, 2024 • edited Loading

What do these changes do?

Related issue/s

How to test

Dev-ops checklist

codecov bot commented Oct 23, 2024 • edited Loading

Codecov Report

sanderegg left a comment

Choose a reason for hiding this comment

sanderegg Nov 5, 2024

Choose a reason for hiding this comment

GitHK left a comment

Choose a reason for hiding this comment

matusdrobuliak66 left a comment

Choose a reason for hiding this comment

pcrespov Nov 5, 2024

Choose a reason for hiding this comment

sonarqubecloud bot commented Nov 6, 2024

Quality Gate passed

giancarloromeo commented Oct 23, 2024 •

edited

Loading

codecov bot commented Oct 23, 2024 •

edited

Loading