Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generating task_key fails on function objects, as objects have no __code__ method #15058

Closed
pgulley opened this issue Aug 23, 2024 · 3 comments · Fixed by #15190
Closed

Generating task_key fails on function objects, as objects have no __code__ method #15058

pgulley opened this issue Aug 23, 2024 · 3 comments · Fixed by #15190
Labels
bug Something isn't working

Comments

@pgulley
Copy link

pgulley commented Aug 23, 2024

Bug summary

This is pretty niche- in my project , I assemble prefect flows dynamically based on some user provided configuration- this works via a common object which provides a call method, which individual tasks all subclass from- these are registered with flow(). A master "Pipeline" object follows the same pattern to actually call these flows in order to produce the This worked fine previously, but upon upgrading to prefect 2.20.3, I'm encountering a new error:

File "/Users/pgulley/Projects/system-metrics/venv/lib/python3.12/site-packages/prefect/tasks.py", line 145, in _generate_task_key h[:NUM_CHARS_DYNAMIC_KEY] if (h := hash_objects(fn.__code__)) else "unknown" ^^^^^^^^^^^ AttributeError: type object 'Pipeline' has no attribute '__code__'. Did you mean: '__doc__'?
This is new as of this update- so I'm not sure what's changed, but it's clear that it has to do with this funny, maybe off-label approach I've taken (as objects don't have a code method like functions do)

(Related, I think, to this issue: #12340)

Version info (prefect version output)

Prefect version yeilds: 
Version:             2.14.20
API version:         0.8.4
Python version:      3.11.9
Git commit:          8ceb0962
Built:               Thu, Feb 1, 2024 6:30 PM
OS/Arch:             darwin/arm64
Profile:             default
Server type:         cloud

But in my venv I have prefect 2.20.3 running

Additional context

No response

@pgulley pgulley added the bug Something isn't working label Aug 23, 2024
@pgulley
Copy link
Author

pgulley commented Aug 23, 2024

I've been able to circumvent by adding a code constant to the pipeline object, but this feels like a bodge

@cicdw
Copy link
Member

cicdw commented Aug 23, 2024

Thanks for the issue @pgulley and for the research you did for it! You're correct that it's definitely related to that issue - I think it was this PR specifically that triggered the problem for you.

I'm glad you have a workaround for now, but we'll also definitely get a fix for this. In fact I have an idea: can you confirm that the following attribute access works for you:

pipeline_instance = Pipeline(**kwargs)
pipeline_instance.__call__.__code__

I believe class call methods have the code object that we use for functions, so a more nuanced check of whether the fn is a true function or a callable should allow us to access it without error.

@pgulley
Copy link
Author

pgulley commented Aug 26, 2024

Great! Thanks for the prompt response!
Yes- that code you gave works- the call method does have the code object you need!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants