You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A meeting decision was made to shift over the scope of DAGs from specific sub-tasks to full end-to-end study DAGs. To deal with compartmentalizing sub-tasks for testability/code readability we decided on having a bunch of factory modules which construct and return an airflow TaskGroup
The current datasource DAG needs to be re-written as a factory module so that a taskgroup is returned. The DAG module should only be responsible for chaining task groups.
The text was updated successfully, but these errors were encountered:
jerdra
changed the title
[ENH] - Create factory modules reading Datman config
[ENH] - Convert to using factory modules in lieu of factory methods in DAG file
Jan 6, 2022
jerdra
changed the title
[ENH] - Convert to using factory modules in lieu of factory methods in DAG file
[ENH] - Convert to using factory modules in lieu of a factory method in the DAG file
Jan 6, 2022
A meeting decision was made to shift over the scope of DAGs from specific sub-tasks to full end-to-end study DAGs. To deal with compartmentalizing sub-tasks for testability/code readability we decided on having a bunch of factory modules which construct and return an airflow TaskGroup
https://www.astronomer.io/guides/task-groups
The current datasource DAG needs to be re-written as a factory module so that a taskgroup is returned. The DAG module should only be responsible for chaining task groups.
The text was updated successfully, but these errors were encountered: