Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for custom flow decorators to prefect deploy #14782

Merged
merged 5 commits into from
Aug 1, 2024

Conversation

desertaxle
Copy link
Member

This PR alters how the client loads a flow from source code when dependencies are missing. Previously, the client would attempt to retrieve flow metadata from the @flow decorator and generate a parameter schema for the decorated function by traversing the abstract syntax tree (AST). This approach prevents the use of custom decorators that wrap the @flow decorator.

This PR instead adds a safe_load_flow_from_entrypoint function which is capable of loading the full flow after making modifications to the AST. safe_load_flow_from_entrypoint will use safe_load_namespace to avoid raising exceptions when dependencies are not present on the current machine when loading a flow. If the flow depends on types or objects from missing dependencies, the AST will be updated to remove the annotations, causing loading to fail. This can generate a different parameter schema if the necessary dependencies are present or not.

safe_load_flow_from_entrypoint is used as a fallback within load_flow_from_entrypoint, so the behavior should not be altered in most cases.

I recognize this implementation is complex and understand this PR might be rejected. If it is accepted, or you decide to go an alternative route without AST parsing, much of the old AST parsing implementation can be deprecated and/or removed in subsequent PRs.

Changes for this PR were originally introduced to main in #14694.

Checklist

  • This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • If this pull request adds new functionality, it includes unit tests that cover the changes
  • If this pull request removes docs files, it includes redirect settings in mint.json.
  • If this pull request adds functions or classes, it includes helpful docstrings.

@desertaxle desertaxle added the fix A fix for a bug in an existing feature label Jul 29, 2024
@desertaxle desertaxle marked this pull request as ready for review July 30, 2024 02:24
@desertaxle desertaxle requested a review from a team as a code owner July 30, 2024 02:24
Copy link
Collaborator

@zzstoatzz zzstoatzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@desertaxle desertaxle merged commit 15274df into 2.x Aug 1, 2024
55 checks passed
@desertaxle desertaxle deleted the 2.x-simplify-flow-parameter-extraction branch August 1, 2024 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix A fix for a bug in an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants