Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use fastjsonschema to validate user configuration #6110

Closed
wants to merge 5 commits into from

Conversation

danielhollas
Copy link
Collaborator

@danielhollas danielhollas commented Sep 4, 2023

We use jsonschema to validate user configuration upon each verdi invocation. The problem is that importing jsonschema package is very slow.

In this PR we use the fastjsonschema instead. This package uses code generation for generating fast python code based on a given schema.
To fully take advantage of this and to avoid the compilation step, the code is pre-compiled and checked-in. The disadvantage is that when the schema changes, the code needs to be re-generated as well.

@sphuber
Copy link
Contributor

sphuber commented Sep 5, 2023

Thanks @danielhollas I want to put a hold on this one, because I was working on replacing jsonschema with pydantic. I was just waiting for the ecosystem to update to pydantic v2 which is significantly faster and has been out for a while now but not all downstream dependencies have updated yet. It is quite backward incompatible so migration takes a bit for some packages.

We use jsonschema to validate user configuration
upon each verdi invocation. The problem is that importing
`jsonschema` package is very slow.

In this PR we use the fastjsonschema instead. This
package uses code generation for generating fast python
code based on a given schema.
To fully take advantage of this and to avoid the compilation step,
the code is pre-compiled and checked-in. The disadvantage is that
when the schema changes, the code needs to be re-generated as well.
@danielhollas
Copy link
Collaborator Author

Closed in favor of #6117

@danielhollas danielhollas deleted the fastjsonschema branch October 23, 2023 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants