From 8cb7c4e90b648aa3957e2df8efcad3c3ff20652c Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Thu, 15 Sep 2022 16:48:17 -0700 Subject: [PATCH 1/9] Create 20220915airfloworchestration --- proposals/20220915airfloworchestration | 40 ++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 proposals/20220915airfloworchestration diff --git a/proposals/20220915airfloworchestration b/proposals/20220915airfloworchestration new file mode 100644 index 00000000..58bd8e8e --- /dev/null +++ b/proposals/20220915airfloworchestration @@ -0,0 +1,40 @@ +SIG TFX-Addons +Project Proposal +Instructions +Copy this template, naming it with the date in yyyymmdd format followed by a short name, and fill in the blanks below. When you're ready for review, place it in /proposals and create a pull request. + +Your name: Varun Murthy + +Your email: murthyvs@google.com + +Your company/organization: Google Core ML TensorFlow Extended (TFX) + +Project name: Airflow + +Project Description +Moving Airflow to add-ons + +Project Category +Choose 1: Component | Example | Other + +Project Use-Case(s) +Airflow is an OSS orchestration tool to run MLOps DAGs. + +Project Implementation +We will be moving native Airflow orchestration support to add-ons this year and aim to deprecate it slowly from our internal source. + +Project Dependencies +tfx/dsl/components/base +tfx/orchestration +tfx/types + +Project Team +Google Core ML TensorFlow Extended + +Note +Please be aware of the processes and requirements which are outlined here: + +SIG-TFX-Addons +Contributing Guidelines +TensorFlow Code of Conduct +Footer From a994a009e2a76c9d7509f56a844aea9112129d1f Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Thu, 15 Sep 2022 16:48:35 -0700 Subject: [PATCH 2/9] Rename 20220915airfloworchestration to 20220915airflow_orchestration --- ...20220915airfloworchestration => 20220915airflow_orchestration} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{20220915airfloworchestration => 20220915airflow_orchestration} (100%) diff --git a/proposals/20220915airfloworchestration b/proposals/20220915airflow_orchestration similarity index 100% rename from proposals/20220915airfloworchestration rename to proposals/20220915airflow_orchestration From 58f7f2cd85bb76f86370d4c0cc3b04286a58be24 Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Mon, 19 Sep 2022 01:02:30 -0700 Subject: [PATCH 3/9] Create 20220919-airflow_orchestration --- proposals/20220919-airflow_orchestration | 40 ++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 proposals/20220919-airflow_orchestration diff --git a/proposals/20220919-airflow_orchestration b/proposals/20220919-airflow_orchestration new file mode 100644 index 00000000..85fe3e29 --- /dev/null +++ b/proposals/20220919-airflow_orchestration @@ -0,0 +1,40 @@ +#### SIG TFX-Addons +# Project Proposal +## Instructions +Copy this template, naming it with the date in `yyyymmdd` format followed by a short name, and fill in the blanks below. +When you're ready for review, place it in `/proposals` and create a pull request. + +--- + +**Your name:** Varun Murthy + +**Your email:** murthyvs@google.com + +**Your company/organization:** Google Core ML + +**Project name:** Airflow Orchestration + +## Project Description +Moving Airflow to TFX add-ons due to lack of native support. + +## Project Category +Orchestrator + +## Project Use-Case(s) +We are moving Airflow from tfx/orchestration to tfx-addons. Native support for Airflow won't be provided in the near future. + +## Project Implementation +Move Airflow from tfx/orchestration to tfx-addons. + +## Project Dependencies +Same as tfx/orchestration/airflow + +## Project Team +Google Core ML TFX Team (MTV + Seoul) + +# Note +Please be aware of the processes and requirements which are outlined here: + +* [SIG-TFX-Addons](https://github.com/tensorflow/tfx-addons) +* [Contributing Guidelines](https://github.com/tensorflow/tfx-addons/blob/main/CONTRIBUTING.md) +* [TensorFlow Code of Conduct](https://github.com/tensorflow/tfx-addons/blob/main/CODE_OF_CONDUCT.md) From 16bad938190f37a09aba18d9364c5c20b31377a0 Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Mon, 19 Sep 2022 01:02:44 -0700 Subject: [PATCH 4/9] Delete 20220915airflow_orchestration --- proposals/20220915airflow_orchestration | 40 ------------------------- 1 file changed, 40 deletions(-) delete mode 100644 proposals/20220915airflow_orchestration diff --git a/proposals/20220915airflow_orchestration b/proposals/20220915airflow_orchestration deleted file mode 100644 index 58bd8e8e..00000000 --- a/proposals/20220915airflow_orchestration +++ /dev/null @@ -1,40 +0,0 @@ -SIG TFX-Addons -Project Proposal -Instructions -Copy this template, naming it with the date in yyyymmdd format followed by a short name, and fill in the blanks below. When you're ready for review, place it in /proposals and create a pull request. - -Your name: Varun Murthy - -Your email: murthyvs@google.com - -Your company/organization: Google Core ML TensorFlow Extended (TFX) - -Project name: Airflow - -Project Description -Moving Airflow to add-ons - -Project Category -Choose 1: Component | Example | Other - -Project Use-Case(s) -Airflow is an OSS orchestration tool to run MLOps DAGs. - -Project Implementation -We will be moving native Airflow orchestration support to add-ons this year and aim to deprecate it slowly from our internal source. - -Project Dependencies -tfx/dsl/components/base -tfx/orchestration -tfx/types - -Project Team -Google Core ML TensorFlow Extended - -Note -Please be aware of the processes and requirements which are outlined here: - -SIG-TFX-Addons -Contributing Guidelines -TensorFlow Code of Conduct -Footer From aaf70b3689fb6001ad3a923d2c60174caeeae12f Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Mon, 19 Sep 2022 01:03:19 -0700 Subject: [PATCH 5/9] Rename 20220919-airflow_orchestration to 20220919-airflow_orchestration.md --- ...19-airflow_orchestration => 20220919-airflow_orchestration.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{20220919-airflow_orchestration => 20220919-airflow_orchestration.md} (100%) diff --git a/proposals/20220919-airflow_orchestration b/proposals/20220919-airflow_orchestration.md similarity index 100% rename from proposals/20220919-airflow_orchestration rename to proposals/20220919-airflow_orchestration.md From 5a0d181beb07c04ee85e12124a2395c985d18501 Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Mon, 19 Sep 2022 01:03:54 -0700 Subject: [PATCH 6/9] Update 20220919-airflow_orchestration.md --- proposals/20220919-airflow_orchestration.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/proposals/20220919-airflow_orchestration.md b/proposals/20220919-airflow_orchestration.md index 85fe3e29..ce4abc68 100644 --- a/proposals/20220919-airflow_orchestration.md +++ b/proposals/20220919-airflow_orchestration.md @@ -1,10 +1,5 @@ #### SIG TFX-Addons # Project Proposal -## Instructions -Copy this template, naming it with the date in `yyyymmdd` format followed by a short name, and fill in the blanks below. -When you're ready for review, place it in `/proposals` and create a pull request. - ---- **Your name:** Varun Murthy From fdfa5f151af2679f1bcd158272e9f3e04d74d15d Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Mon, 19 Sep 2022 01:04:35 -0700 Subject: [PATCH 7/9] Update 20220919-airflow_orchestration.md --- proposals/20220919-airflow_orchestration.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/20220919-airflow_orchestration.md b/proposals/20220919-airflow_orchestration.md index ce4abc68..c22709ad 100644 --- a/proposals/20220919-airflow_orchestration.md +++ b/proposals/20220919-airflow_orchestration.md @@ -5,12 +5,12 @@ **Your email:** murthyvs@google.com -**Your company/organization:** Google Core ML +**Your company/organization:** Google Core ML TFX Team (MTV + Seoul) **Project name:** Airflow Orchestration ## Project Description -Moving Airflow to TFX add-ons due to lack of native support. +Moving Airflow to TFX add-ons due to decreasing native support. ## Project Category Orchestrator @@ -22,7 +22,7 @@ We are moving Airflow from tfx/orchestration to tfx-addons. Native support for A Move Airflow from tfx/orchestration to tfx-addons. ## Project Dependencies -Same as tfx/orchestration/airflow +Same as tfx/orchestration/airflow. ## Project Team Google Core ML TFX Team (MTV + Seoul) From d9a2374ccb133f153f1f116849f1c9e3977bafbc Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Wed, 21 Sep 2022 12:35:57 -0700 Subject: [PATCH 8/9] Update 20220919-airflow_orchestration.md --- proposals/20220919-airflow_orchestration.md | 24 +++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/proposals/20220919-airflow_orchestration.md b/proposals/20220919-airflow_orchestration.md index c22709ad..e8288e7c 100644 --- a/proposals/20220919-airflow_orchestration.md +++ b/proposals/20220919-airflow_orchestration.md @@ -19,13 +19,29 @@ Orchestrator We are moving Airflow from tfx/orchestration to tfx-addons. Native support for Airflow won't be provided in the near future. ## Project Implementation -Move Airflow from tfx/orchestration to tfx-addons. - +1. Copy tfx/orchestration/airflow to tfx-addons/airflow. +2. Mark Airflow as deprecated in tfx/ and indicate that support will be dropped in 1-2 releases. +3. Update TFX tutorials on www.tensorflow.org to indicate deprecated and moving to TFXA + ## Project Dependencies -Same as tfx/orchestration/airflow. +1. tfx/dsl/components/base:base_node +2. tfx/dsl/components/base:base_component +3. tfx/dsl/components/base:base_executor +4. tfx/dsl/components/base:executor_spec +5. tfx/orchestration:data_types +6. tfx/orchestration:metadata +7. tfx/orchestration/config:base_component_config +8. tfx/orchestration/launcher:base_component_launcher +9. tfx/orchestration:pipeline +10. tfx/orchestration:tfx_runner +11. tfx/orchestration/config:config_utils +12. tfx/orchestration/config:pipeline_config +13. tfx/utils:json_utils +14. tfx/utils:telemetry_utils ## Project Team -Google Core ML TFX Team (MTV + Seoul) +Varun Murthy (murthyvs@google.com) +Google Core ML TFX Team (tfx-team@google.com) # Note Please be aware of the processes and requirements which are outlined here: From 2a389b96a4d830a66a2dece6187f25d655ebcc5e Mon Sep 17 00:00:00 2001 From: murthyvs <113647067+murthyvs@users.noreply.github.com> Date: Wed, 21 Sep 2022 14:10:38 -0700 Subject: [PATCH 9/9] Update 20220919-airflow_orchestration.md --- proposals/20220919-airflow_orchestration.md | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-) diff --git a/proposals/20220919-airflow_orchestration.md b/proposals/20220919-airflow_orchestration.md index e8288e7c..2b240ddb 100644 --- a/proposals/20220919-airflow_orchestration.md +++ b/proposals/20220919-airflow_orchestration.md @@ -24,24 +24,10 @@ We are moving Airflow from tfx/orchestration to tfx-addons. Native support for A 3. Update TFX tutorials on www.tensorflow.org to indicate deprecated and moving to TFXA ## Project Dependencies -1. tfx/dsl/components/base:base_node -2. tfx/dsl/components/base:base_component -3. tfx/dsl/components/base:base_executor -4. tfx/dsl/components/base:executor_spec -5. tfx/orchestration:data_types -6. tfx/orchestration:metadata -7. tfx/orchestration/config:base_component_config -8. tfx/orchestration/launcher:base_component_launcher -9. tfx/orchestration:pipeline -10. tfx/orchestration:tfx_runner -11. tfx/orchestration/config:config_utils -12. tfx/orchestration/config:pipeline_config -13. tfx/utils:json_utils -14. tfx/utils:telemetry_utils +TensorFlow Extended (1.9.1) ## Project Team -Varun Murthy (murthyvs@google.com) -Google Core ML TFX Team (tfx-team@google.com) +murthyvs: Varun Murthy (murthyvs@google.com) # Note Please be aware of the processes and requirements which are outlined here: