Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/114 add django setup configuration #115

Merged
merged 16 commits into from
Dec 3, 2024

Conversation

Coperh
Copy link
Contributor

@Coperh Coperh commented Jul 9, 2024

fixes: #114

Implements django setup configuration in OIDC itself

@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch 10 times, most recently from fc242eb to c9f80ed Compare July 10, 2024 13:18
@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch 2 times, most recently from c60c2e3 to 6a447fc Compare July 26, 2024 13:54
@Coperh Coperh marked this pull request as ready for review July 26, 2024 13:57
@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch from 6e001b1 to aca7802 Compare July 30, 2024 12:29
@Coperh
Copy link
Contributor Author

Coperh commented Jul 30, 2024

Mostly copied AdminOIDCConfigurationSettings from open inwoner and the documentation. Made the endpoints required since it fails to save without them. Converted the unittest to pytest

Example of it being used here open-zaak/open-zaak#1728

@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch from 20f406b to aca9492 Compare August 2, 2024 07:34
@Coperh
Copy link
Contributor Author

Coperh commented Aug 2, 2024

I am also unsure about the package/file names

@Coperh Coperh removed the request for review from pi-sigma August 2, 2024 07:55
Copy link
Member

@sergei-maertens sergei-maertens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a preliminary glance, haven't looked into much details yet, but I have some concerns - perhaps about the way django-setup-configuration itself works:

amount of settings

there are a lot of settings being defined purely to facilitate bootstrapping configuration. Some of these setting names also have a big potential to cause confusion with settings that actually have some meaning at runtime. I don't think this will improve the developer experience

settings datatypes

I noticed in OIP that settings are used so that envvars can be used to specify them, but the datatypes of these settings (like lists of strings, periods/comma's having special meaning...) but also dict/mappings complicates passing them via the env

you basically have a Helm chart or Ansible playbook where the datatype is properly defined in YAML, this is then templated out to a string so it can be passed as an envvar, after which the application deserializes it again from a string into the expected datatype

I would much prefer one of the options below (with the first having my preference):

  • load a YAML/TOML/JSON file that contains the data/config to initialize. you then only need to point to which file to load, plus it will scale in the future when we don't have solo models anymore but instead regular models. it's basically a django-agnostic fixture format
  • read the values directly from the env and remove the need to define them as settings. the library knows how to interpret each envvar and parse it into the expected data-structure

IMO the former approach makes documenting the format a lot easier, you can simply provide an example template, you can add comments if you use YAML/TOML and the processing step of that file is able to populate missing information (the endpoints!) from the discovery endpoint if that's all that is specified.

.github/workflows/ci.yml Show resolved Hide resolved
CHANGELOG.rst Outdated Show resolved Hide resolved
mozilla_django_oidc_db/setupconfig/boostrap.py Outdated Show resolved Hide resolved
testapp/settings.py Show resolved Hide resolved
testapp/settings.py Outdated Show resolved Hide resolved
mozilla_django_oidc_db/setupconfig/boostrap.py Outdated Show resolved Hide resolved
mozilla_django_oidc_db/setupconfig/boostrap.py Outdated Show resolved Hide resolved
mozilla_django_oidc_db/setupconfig/boostrap.py Outdated Show resolved Hide resolved
tests/setupconfig/test_auth.py Outdated Show resolved Hide resolved
@alextreme
Copy link
Member

FYI for Conor, Paul is working on a related issue: https://taiga.maykinmedia.nl/project/open-inwoner/issue/2563

@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch 2 times, most recently from 92960f5 to 68d3a61 Compare August 14, 2024 15:53
@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch 2 times, most recently from 3bc045b to 93f6c66 Compare August 23, 2024 10:47
@Coperh Coperh requested a review from sergei-maertens August 23, 2024 15:06

if groups := all_settings.get("default_groups"):
all_settings["default_groups"] = create_missing_groups(
groups, all_settings["sync_groups_glob_pattern"]
Copy link
Contributor Author

@Coperh Coperh Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it use the glob pattern here? I do not think it did in the original Open Inwoner step but does in the middleware

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I should probably make a test for this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read this correctly it will always create missing groups, regardless of the value of sync_groups? I would either remove sync_groups from the model and always set it to True then, or add it as a conditional here to make sure it works the same as in the admin

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an oversite and I assume it should use the sync_group value

Copy link
Contributor

@swrichards swrichards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Useful to see the API being battle tested. I'll update the docs at least regarding defaults and testing best practices based on this. Otherwise LGTM.

docs/setup_configuration.rst Outdated Show resolved Hide resolved
docs/setup_configuration.rst Outdated Show resolved Hide resolved
docs/setup_configuration.rst Show resolved Hide resolved
mozilla_django_oidc_db/setup_configuration/models.py Outdated Show resolved Hide resolved
tests/setupconfig/test_steps.py Outdated Show resolved Hide resolved
tests/setupconfig/test_steps.py Show resolved Hide resolved
Coperh and others added 2 commits November 29, 2024 11:03
* Pin django-setup-configuration >= 0.4.0
* Use Pydantic model defaults, remove defaults from step
* Remove now unneeded OIDCSetupConfigForm
* Set non-default values for test_configure_use_defaults
* Update Documentation
@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch from 623d37d to 1d0bdbf Compare November 29, 2024 10:44
* Set non default values in model in test_configure_use_defaults
* Remove model fixtures in favour of file yml fixtures
* Move from build_step_config_from_sources in fixture to execute_single_step in test
* Remove model validaiton tests
@Coperh
Copy link
Contributor Author

Coperh commented Nov 29, 2024

@stevenbal @SonnyBA This is ready to re-review if either of you have time.

Copy link
Contributor

@swrichards swrichards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically done I think, just a few questions/comments mainly about the use of defaults.

mozilla_django_oidc_db/setup_configuration/models.py Outdated Show resolved Hide resolved
mozilla_django_oidc_db/setup_configuration/models.py Outdated Show resolved Hide resolved
mozilla_django_oidc_db/setup_configuration/models.py Outdated Show resolved Hide resolved
tests/setupconfig/test_steps.py Outdated Show resolved Hide resolved
docs/setup_configuration.rst Outdated Show resolved Hide resolved
@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch from 4e51bd9 to ecad85b Compare December 2, 2024 13:17
docs/quickstart.rst Outdated Show resolved Hide resolved
tests/setupconfig/test_steps.py Outdated Show resolved Hide resolved
Comment on lines +67 to +68
Optional Fields:
""""""""""""""""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@swrichards perhaps this could be an alternative to the generate documentation functionality?

mozilla_django_oidc_db/constants.py Show resolved Hide resolved

if groups := all_settings.get("default_groups"):
all_settings["default_groups"] = create_missing_groups(
groups, all_settings["sync_groups_glob_pattern"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read this correctly it will always create missing groups, regardless of the value of sync_groups? I would either remove sync_groups from the model and always set it to True then, or add it as a conditional here to make sure it works the same as in the admin

@swrichards
Copy link
Contributor

@swrichards perhaps this could be an alternative to the generate documentation functionality?

@stevenbal It's definitely a good stopgap until the documentation generation is fixed, though I think we should add the auto-generation eventually (but later, because it's non-blocking for the MVP). Doing the extra work here and now makes sense because this is a library with consumers, but I think for the downstream projects we can afford to wait until the documentation generator is back up.

* Explicitly use model values
* Rename create_missing_groups to get_groups_by_name
* Move sync_groups condition to get_groups_by_name
* Fix sync_group doing nothing in step
* Test sync_group & sync_groups_glob_pattern
* Test idempotent and overwrite
@Coperh Coperh force-pushed the feature/114-add-django-setup-configuration branch from a2c8e7d to d6c8846 Compare December 2, 2024 18:47
@Coperh Coperh requested a review from stevenbal December 2, 2024 18:49
@stevenbal stevenbal merged commit 2382725 into master Dec 3, 2024
11 checks passed
@stevenbal stevenbal deleted the feature/114-add-django-setup-configuration branch December 3, 2024 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants