Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[externals] Modify context construction to support direct invocation for SubprocessExecutionResource #15984

Merged
merged 1 commit into from
Aug 21, 2023

Conversation

smackesey
Copy link
Collaborator

Summary & Motivation

This required a few minor supporting changes to OpExecutionContext and BoundOpExecutionContext.

How I Tested These Changes

New unit test.

@@ -54,8 +55,8 @@ def build_external_execution_context(
_convert_time_window(partition_time_window) if partition_time_window else None
),
run_id=context.run_id,
job_name=context.job_def.name,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is a BoundOpExecutionContext getting here in the first place? Shouldn't it always be bound by the time it gets to user space?

Copy link
Collaborator Author

@smackesey smackesey Aug 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confused by your question, it sounds like it should say "how is an UnboundOpExecutionContext...

BoundOpExecutionContext is the name of the passed class as context when directly invoking, it has nothing to do with the externals work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I got confused and meant UnboundOpExecutionContext because it is confusing that we have both BoundOpExecutionContext and OpExecutionContext as non-abstract classes

Base automatically changed from sean/external-tail-file to master August 21, 2023 21:28
@smackesey smackesey force-pushed the sean/externals-support-direct-invocation branch from dcdbddf to bdc8bea Compare August 21, 2023 22:03
@smackesey smackesey force-pushed the sean/externals-support-direct-invocation branch from bdc8bea to 1eb11d3 Compare August 21, 2023 22:15
@smackesey smackesey marked this pull request as ready for review August 21, 2023 22:20
@smackesey smackesey requested a review from schrockn August 21, 2023 22:20
@smackesey smackesey merged commit e8c4a7a into master Aug 21, 2023
@smackesey smackesey deleted the sean/externals-support-direct-invocation branch August 21, 2023 22:52
sirawats pushed a commit to sirawats/dagster that referenced this pull request Aug 24, 2023
…for SubprocessExecutionResource (dagster-io#15984)

## Summary & Motivation

This required a few minor supporting changes to `OpExecutionContext` and
`BoundOpExecutionContext`.

## How I Tested These Changes

New unit test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants