Reorganization of OpenFL Experimental Directory Structure (#1169)

* reorganize directory structure Signed-off-by: refai06 <[email protected]> * reorganize tutorial dir structure Signed-off-by: refai06 <[email protected]> * reorganize workspace dir structure Signed-off-by: refai06 <[email protected]> * reorganize tests dir structure Signed-off-by: refai06 <[email protected]> * reorganize workspace directory Signed-off-by: refai06 <[email protected]> * reorganize dir Signed-off-by: refai06 <[email protected]> * revert testcase update Signed-off-by: refai06 <[email protected]> * update directory structure Signed-off-by: refai06 <[email protected]> * revert copyright changes Signed-off-by: refai06 <[email protected]> * update requirement Signed-off-by: refai06 <[email protected]> * directory restructure changes Signed-off-by: refai06 <[email protected]> * update directory restructure Signed-off-by: refai06 <[email protected]> --------- Signed-off-by: refai06 <[email protected]>
securefederatedai · Nov 28, 2024 · a26bdb2 · a26bdb2
1 parent 00f2755
commit a26bdb2
Show file tree

Hide file tree

Showing 347 changed files with 5,135 additions and 1,891 deletions.
diff --git a/.github/workflows/experimental_workflow_tests.yml b/.github/workflows/experimental_workflow_tests.yml
@@ -24,63 +24,63 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install -r tests/github/experimental/requirements_experimental_localruntime_tests.txt
+        pip install -r tests/github/experimental/workflow/LocalRuntime/requirements_experimental_localruntime_tests.txt
         pip install .
     - name: Workflow - Exclude Test (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_exclude.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_exclude.py
     - name: Workflow - Exclude Test (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_exclude.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_exclude.py ray
     - name: Workflow - Include Test (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_include.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_include.py
     - name: Workflow - Include Test (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_include.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_include.py ray
     - name: Workflow - Include / Exclude Test (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_include_exclude.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_include_exclude.py
     - name: Workflow - Include / Exclude Test (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_include_exclude.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_include_exclude.py ray
 
     - name: Workflow - Internal Loop Test (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_internalloop.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_internalloop.py
     - name: Workflow - Internal Loop Test (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_internalloop.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_internalloop.py ray
     - name: Workflow - Private Attributes (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_privateattributes.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_privateattributes.py
     - name: Workflow - Private Attributes (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_privateattributes.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_privateattributes.py ray
     - name: Workflow - References (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_reference.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_reference.py
     - name: Workflow - References (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_reference.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_reference.py ray
     - name: Workflow - References with Exclude (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_reference_with_exclude.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_reference_with_exclude.py
     - name: Workflow - References with Exclude (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_reference_with_exclude.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_reference_with_exclude.py ray
     - name: Workflow - References with Include (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_reference_with_include.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_reference_with_include.py
     - name: Workflow - References with Include (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_reference_with_include.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_reference_with_include.py ray
     - name: Workflow - Collaborator Subset (Single Process Backend)
       run: |
-        python tests/github/experimental/testflow_subset_of_collaborators.py
+        python tests/github/experimental/workflow/LocalRuntime/testflow_subset_of_collaborators.py
     - name: Workflow - Collaborator Subset (Ray Backend)
       run: |
-        python tests/github/experimental/testflow_subset_of_collaborators.py ray
+        python tests/github/experimental/workflow/LocalRuntime/testflow_subset_of_collaborators.py ray
     - name: Test Experimental Aggregator Based Workflow API
       run: |
-        python -m tests.github.experimental.workspace.test_experimental_agg_based_workflow --custom_template tests/github/experimental/workspace/testcase_datastore_cli --fed_workspace aggregator --col col1 --col col2  --rounds-to-train 1
+        python -m tests.github.experimental.workflow.AggregatorBasedWorkflow.test_experimental_agg_based_workflow --custom_template tests/github/experimental/workflow/AggregatorBasedWorkflow/testcase_datastore_cli --fed_workspace aggregator --col col1 --col col2  --rounds-to-train 1
diff --git a/.github/workflows/workflow_interface_101_mnist.yml b/.github/workflows/workflow_interface_101_mnist.yml
@@ -36,11 +36,11 @@ jobs:
 
     - name: Run Notebook
       run: |
-        jupyter nbconvert --execute --to notebook ./openfl-tutorials/experimental/101_MNIST.ipynb
+        jupyter nbconvert --execute --to notebook ./openfl-tutorials/experimental/workflow/101_MNIST.ipynb
         echo "Notebook run completed"
 
     - name: Tar files
-      run: tar -cvf notebook.tar ./openfl-tutorials/experimental/101_MNIST.nbconvert.ipynb
+      run: tar -cvf notebook.tar ./openfl-tutorials/experimental/workflow/101_MNIST.nbconvert.ipynb
 
     - name: Upload Artifacts
       uses: actions/upload-artifact@v4

diff --git a/README.md b/README.md
@@ -40,7 +40,7 @@ OpenFL supports two APIs to set up a Federated Learning experiment:
 Define an experiment and distribute it manually. All participants can verify model code and [FL plan](https://openfl.readthedocs.io/en/latest/about/features_index/taskrunner.html#federated-learning-plan-fl-plan-settings) prior to execution. The federation is terminated when the experiment is finished. This API is meant for enterprise-grade FL experiments, including support for mTLS-based communication channels and TEE-ready nodes (based on Intel® SGX).
 
 - [Workflow API](https://openfl.readthedocs.io/en/latest/about/features_index/workflowinterface.html) ([*experimental*](https://openfl.readthedocs.io/en/latest/developer_guide/experimental_features.html)):
-Create complex experiments that extend beyond traditional horizontal federated learning. This API enables an experiment to be simulated locally, then seamlessly scaled to a federated setting. See the [experimental tutorials](https://github.com/securefederatedai/openfl/blob/develop/openfl-tutorials/experimental/) to learn how to coordinate [aggregator validation after collaborator model training](https://github.com/securefederatedai/openfl/tree/develop/openfl-tutorials/experimental/102_Aggregator_Validation.ipynb), [perform global differentially private federated learning](https://github.com/psfoley/openfl/tree/experimental-workflow-interface/openfl-tutorials/experimental/Global_DP), measure the amount of private information embedded in a model after collaborator training with [privacy meter](https://github.com/securefederatedai/openfl/blob/develop/openfl-tutorials/experimental/Privacy_Meter/readme.md), or [add a watermark to a federated model](https://github.com/securefederatedai/openfl/blob/develop/openfl-tutorials/experimental/301_MNIST_Watermarking.ipynb).
+Create complex experiments that extend beyond traditional horizontal federated learning. This API enables an experiment to be simulated locally, then seamlessly scaled to a federated setting. See the [experimental tutorials](https://github.com/securefederatedai/openfl/blob/develop/openfl-tutorials/experimental/workflow/) to learn how to coordinate [aggregator validation after collaborator model training](https://github.com/securefederatedai/openfl/tree/develop/openfl-tutorials/experimental/workflow/102_Aggregator_Validation.ipynb), [perform global differentially private federated learning](https://github.com/psfoley/openfl/tree/experimental-workflow-interface/openfl-tutorials/experimental/workflow/Global_DP), measure the amount of private information embedded in a model after collaborator training with [privacy meter](https://github.com/securefederatedai/openfl/blob/develop/openfl-tutorials/experimental/workflow/Privacy_Meter/readme.md), or [add a watermark to a federated model](https://github.com/securefederatedai/openfl/blob/develop/openfl-tutorials/experimental/workflow/301_MNIST_Watermarking.ipynb).
 
 The quickest way to test OpenFL is to follow the [online documentation](https://openfl.readthedocs.io/en/latest/index.html) to launch your first federation.<br/>
 Read the [blog post](https://medium.com/openfl/from-centralized-machine-learning-to-federated-learning-with-openfl-b3e61da52432) explaining steps to train a model with OpenFL. <br/>

diff --git a/docs/about/features_index/privacy_meter.rst b/docs/about/features_index/privacy_meter.rst
@@ -22,7 +22,7 @@ In this threat model, each party can audit the privacy loss of the local and glo
 
 Workflow
 -----------------------------------------------
-We provide a demo code in `cifar10_PM.py <https://github.com/securefederatedai/openfl/blob/develop/openfl-tutorials/experimental/Privacy_Meter/cifar10_PM.py>`_. Here, we briefly describe its workflow.
+We provide a demo code in `cifar10_PM.py <https://github.com/securefederatedai/openfl/blob/develop/openfl-tutorials/experimental/workflow/Privacy_Meter/cifar10_PM.py>`_. Here, we briefly describe its workflow.
 In each round of FL, parties train, starting with the current global model as initialization, using their local dataset. Then, the current global model and updated local model will be passed to the privacy auditing module (See `audit` function in `cifar10_PM.py`) to produce a privacy loss report. The local model update will then be shared to the server and all such updates aggregated to form the next global model. Though this is a simulation so that no network sharing of models is involved, these reports could be used in a fully distributed setting to trigger actions when the loss is too high. These actions could include not sharing local updates to the aggregator, not 
 allowing the FL system to release the model to other outside entities, or potentially re-running local training in a differentially private mode and re-auditing in an attempt to reduce the leakage before sharing occurs.
 
@@ -36,4 +36,4 @@ To measure the success of the attack (privacy loss), we generate the ROC of the
 
 Examples
 -----------------------------------------------
-`Here <https://github.com/securefederatedai/openfl/tree/f1657abe88632d542504d6d71ca961de9333913f/openfl-tutorials/experimental/Privacy_Meter>`_, we give a few commands and the results for each of them. 
+`Here <https://github.com/securefederatedai/openfl/tree/f1657abe88632d542504d6d71ca961de9333913f/openfl-tutorials/experimental/workflow/Privacy_Meter>`_, we give a few commands and the results for each of them.