Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[prototype] sub-context for each upstream dependency #19283

Closed

Conversation

jamiedemaria
Copy link
Contributor

@jamiedemaria jamiedemaria commented Jan 18, 2024

Summary & Motivation

This PR is a prototype for a potential API for partition methods on the AssetExecutionContext. This version adds a DepContext that contains various methods that are scoped to a particular upstream dependency, including partition methods

No Hooli or DOP PR for this one since those repos just use partition methods for the currently materializing asset. They would look identical to the PRs for API 1 (Hooli PR - dagster-io/hooli-data-eng-pipelines#59 DOP PR - https://github.com/dagster-io/internal/pull/7903)

How I Tested These Changes

@jamiedemaria
Copy link
Contributor Author

jamiedemaria commented Jan 18, 2024

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

This stack of pull requests is managed by Graphite.

@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 1fee854 to 558590d Compare January 19, 2024 16:38
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from 675fc6d to 725f11d Compare January 19, 2024 16:38
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 558590d to bcc9cab Compare January 19, 2024 19:09
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from 725f11d to bc1343d Compare January 19, 2024 19:09
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from bcc9cab to af681dc Compare January 19, 2024 20:13
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from bc1343d to db5c97f Compare January 19, 2024 20:13
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from af681dc to 3016287 Compare January 23, 2024 00:11
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from db5c97f to 79c9359 Compare January 23, 2024 00:11
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 3016287 to 04fd627 Compare January 26, 2024 19:10
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from 79c9359 to 5711594 Compare January 26, 2024 19:11
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 04fd627 to 0e1fc3e Compare January 26, 2024 20:05
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from 5711594 to f62c24d Compare January 26, 2024 20:05
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 0e1fc3e to dc6edf0 Compare January 29, 2024 15:39
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from f62c24d to fc4cb27 Compare January 29, 2024 15:39
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from dc6edf0 to ccc4260 Compare January 29, 2024 16:30
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from fc4cb27 to c4d000e Compare January 29, 2024 16:30
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from ccc4260 to f416c05 Compare January 29, 2024 17:26
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from c4d000e to d8cf9ea Compare January 29, 2024 17:26
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from f416c05 to 51d46aa Compare January 29, 2024 18:16
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from d8cf9ea to d185017 Compare January 29, 2024 18:16
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 51d46aa to 2670364 Compare January 29, 2024 19:44
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from d185017 to 532ca5f Compare January 29, 2024 19:44
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 2670364 to 2660075 Compare January 29, 2024 20:03
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from 532ca5f to fc4ed8c Compare January 29, 2024 20:03
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from 2660075 to a5828e1 Compare January 30, 2024 17:23
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from fc4ed8c to 79d3238 Compare January 30, 2024 17:23
@jamiedemaria jamiedemaria force-pushed the jamie/partition-example-base branch from a5828e1 to f5be566 Compare January 30, 2024 20:19
@jamiedemaria jamiedemaria force-pushed the jamie/partition-api-4 branch from 79d3238 to d23210e Compare January 30, 2024 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant