diff --git a/docs/design/package_structure_overview.dot b/docs/design/package_structure_overview.dot index 6c5eba6345..9e6144708c 100644 --- a/docs/design/package_structure_overview.dot +++ b/docs/design/package_structure_overview.dot @@ -5,7 +5,7 @@ digraph { subgraph cluster_users { research [href="https://github.com/google-research/federated/blob/master" target="_parent", fillcolor="#E2F3EB", color="#57BB8A"] - examples [href="https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples" target="_parent", fillcolor="#E2F3EB", color="#57BB8A"] + examples [href="https://github.com/tensorflow/federated/blob/main/examples" target="_parent", fillcolor="#E2F3EB", color="#57BB8A"] tests [href="https://github.com/tensorflow/federated/blob/main/tensorflow_federated/python/tests" target="_parent", fillcolor="#E2F3EB", color="#57BB8A"] } diff --git a/docs/program/federated_program.md b/docs/program/federated_program.md index b8a9d6866e..7aa107f70d 100644 --- a/docs/program/federated_program.md +++ b/docs/program/federated_program.md @@ -7,7 +7,7 @@ especially its type system. For more information about federated program, see: * [API Documentation](https://www.tensorflow.org/federated/api_docs/python/tff/program) -* [Examples](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/program) +* [Examples](https://github.com/tensorflow/federated/blob/main/examples/program) * [Developer Guide](https://github.com/tensorflow/federated/blob/main/docs/program/guide.md) [TOC] diff --git a/docs/program/guide.md b/docs/program/guide.md index 6f0604fcd7..f50c10e07b 100644 --- a/docs/program/guide.md +++ b/docs/program/guide.md @@ -16,7 +16,7 @@ This section defines guidelines for how should be authored. See the example -[program_logic.py](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/program/program_logic.py) +[program_logic.py](https://github.com/tensorflow/federated/blob/main/examples/program/program_logic.py) for more information. ### Document Type Signatures @@ -213,7 +213,7 @@ def program_logic(...) -> None: ### Tests **Do** provide unit tests for the program logic (e.g. -[program_logic_test.py](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/program/program_logic_test.py)). +[program_logic_test.py](https://github.com/tensorflow/federated/blob/main/examples/program/program_logic_test.py)). ## Program @@ -222,14 +222,14 @@ This section defines guidelines for how a should be authored. See the example -[program.py](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/program/program.py) +[program.py](https://github.com/tensorflow/federated/blob/main/examples/program/program.py) for more information. ### Document the Program **Do** document the details of the program to the customer in the docstring of the module (e.g. -[program.py](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/program/program.py)): +[program.py](https://github.com/tensorflow/federated/blob/main/examples/program/program.py)): * How to manually run the program. * What platform, computations, and data sources are used in the program. diff --git a/docs/tff_for_research.md b/docs/tff_for_research.md index 2b0dffefec..f5b557b249 100644 --- a/docs/tff_for_research.md +++ b/docs/tff_for_research.md @@ -21,7 +21,7 @@ types of logic. encapsulate logic that runs in a single location (e.g., on clients or on a server). This code is typically written and tested without any `tff.*` references, and can be re-used outside of TFF. For example, the - [client training loop in Federated Averaging](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L184-L222) + [client training loop in Federated Averaging](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L184-L222) is implemented at this level. 1. TensorFlow Federated orchestration logic, which binds together the @@ -29,12 +29,12 @@ types of logic. and then orchestrating them using abstractions like `tff.federated_broadcast` and `tff.federated_mean` inside a `tff.federated_computation`. See, for example, this - [orchestration for Federated Averaging](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py#L112-L140). + [orchestration for Federated Averaging](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tff.py#L112-L140). 1. An outer driver script that simulates the control logic of a production FL system, selecting simulated clients from a dataset and then executing federated computations defined in 2. on those clients. For example, - [a Federated EMNIST experiment driver](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py). + [a Federated EMNIST experiment driver](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py). ## Federated learning datasets @@ -114,17 +114,17 @@ TFF, depending on the desired level of customization. A minimal stand-alone implementation of the [Federated Averaging](https://arxiv.org/abs/1602.05629) algorithm is provided -[here](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg). +[here](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg). The code includes -[TF functions](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py) +[TF functions](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py) for local computation, -[TFF computations](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py) +[TFF computations](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tff.py) for orchestration, and a -[driver script](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py) +[driver script](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py) on the EMNIST dataset as an example. These files can easily be adapted for customized applciations and algorithmic changes following detailed instructions in the -[README](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/README.md). +[README](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/README.md). A more general implementation of Federated Averaging can be found [here](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/python/learning/algorithms/fed_avg.py). @@ -218,13 +218,13 @@ One approach is to let each client fine-tune a single global model (trained using federated learning) with their local data. This approach has connections to meta-learning, see, e.g., [this paper](https://arxiv.org/abs/1909.12488). An example of this approach is given in -[`emnist_p13n_main.py`](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/personalization/emnist_p13n_main.py). +[`emnist_p13n_main.py`](https://github.com/tensorflow/federated/blob/main/examples/personalization/emnist_p13n_main.py). To explore and compare different personalization strategies, you can: * Define a personalization strategy by implementing a `tf.function` that starts from an initial model, trains and evaluates a personalized model using each client's local datasets. An example is given by - [`build_personalize_fn`](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/personalization/p13n_utils.py). + [`build_personalize_fn`](https://github.com/tensorflow/federated/blob/main/examples/personalization/p13n_utils.py). * Define an `OrderedDict` that maps strategy names to the corresponding personalization strategies, and use it as the `personalize_fn_dict` argument diff --git a/docs/tutorials/custom_federated_algorithm_with_tff_optimizers.ipynb b/docs/tutorials/custom_federated_algorithm_with_tff_optimizers.ipynb index 910cf36e88..5f306fdd59 100644 --- a/docs/tutorials/custom_federated_algorithm_with_tff_optimizers.ipynb +++ b/docs/tutorials/custom_federated_algorithm_with_tff_optimizers.ipynb @@ -61,7 +61,7 @@ "source": [ "# Use TFF optimizers in custom iterative process\n", "\n", - "This is an alternative to the [Build Your Own Federated Learning Algorithm](building_your_own_federated_learning_algorithm.ipynb) tutorial and the [simple_fedavg](https://github.com/tensorflow/federated/tree/main/tensorflow_federated/examples/simple_fedavg) example to build a custom iterative process for the [federated averaging](https://arxiv.org/abs/1602.05629) algorithm. This tutorial will use [TFF optimizers](https://github.com/tensorflow/federated/tree/main/tensorflow_federated/python/learning/optimizers) instead of Keras optimizers.\n", + "This is an alternative to the [Build Your Own Federated Learning Algorithm](building_your_own_federated_learning_algorithm.ipynb) tutorial and the [simple_fedavg](https://github.com/tensorflow/federated/tree/main/examples/simple_fedavg) example to build a custom iterative process for the [federated averaging](https://arxiv.org/abs/1602.05629) algorithm. This tutorial will use [TFF optimizers](https://github.com/tensorflow/federated/tree/main/tensorflow_federated/python/learning/optimizers) instead of Keras optimizers.\n", "The TFF optimizer abstraction is desgined to be state-in-state-out to be easier to be incorporated in a TFF iterative process. The `tff.learning` APIs also accept TFF optimizers as input argument." ] }, @@ -124,7 +124,7 @@ }, "source": [ "## Preparing data and model\n", - "The EMNIST data processing and model are very similar to the [simple_fedavg](https://github.com/tensorflow/federated/tree/main/tensorflow_federated/examples/simple_fedavg) example." + "The EMNIST data processing and model are very similar to the [simple_fedavg](https://github.com/tensorflow/federated/tree/main/examples/simple_fedavg) example." ] }, { diff --git a/docs/tutorials/random_noise_generation.ipynb b/docs/tutorials/random_noise_generation.ipynb index e3f9ec5a6a..3244761306 100644 --- a/docs/tutorials/random_noise_generation.ipynb +++ b/docs/tutorials/random_noise_generation.ipynb @@ -575,7 +575,7 @@ "id": "x9kc6G0RIATV" }, "source": [ - "Now let us use the helper class and `tf.random.stateless_normal` to generate (nested structure of) random noise in TFF. The following code snippet looks a lot like a TFF iterative process, see [simple_fedavg](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py) as an example of expressing federated learning algorithm as TFF iterative process. The pseudo seed state here for random noise generation is `tf.Tensor` that can be easily transported in TFF and TF functions." + "Now let us use the helper class and `tf.random.stateless_normal` to generate (nested structure of) random noise in TFF. The following code snippet looks a lot like a TFF iterative process, see [simple_fedavg](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tff.py) as an example of expressing federated learning algorithm as TFF iterative process. The pseudo seed state here for random noise generation is `tf.Tensor` that can be easily transported in TFF and TF functions." ] }, { diff --git a/docs/tutorials/simulations_with_accelerators.ipynb b/docs/tutorials/simulations_with_accelerators.ipynb index c0bc329ca5..1c6f9a2476 100644 --- a/docs/tutorials/simulations_with_accelerators.ipynb +++ b/docs/tutorials/simulations_with_accelerators.ipynb @@ -227,7 +227,7 @@ "id": "QJZh5NoiG-hR" }, "source": [ - "We define a function preprocessing the EMNIST example following the [simple_fedavg](https://github.com/tensorflow/federated/tree/main/tensorflow_federated/examples/simple_fedavg) example. Note that the argument `client_epochs_per_round` controls the number of local epochs on clients in federated learning. " + "We define a function preprocessing the EMNIST example following the [simple_fedavg](https://github.com/tensorflow/federated/tree/main/examples/simple_fedavg) example. Note that the argument `client_epochs_per_round` controls the number of local epochs on clients in federated learning. " ] }, { @@ -342,7 +342,7 @@ "id": "8OGBU1SLpyiR" }, "source": [ - "Now let us define the training loop for EMNIST. Note that `use_experimental_simulation_loop=True` in `tff.learning.algorithms.build_weighted_fed_avg` is suggested for performant TFF simulation, and required to take advantage of multi-GPUs on a single machine. See [simple_fedavg](https://github.com/tensorflow/federated/tree/main/tensorflow_federated/examples/simple_fedavg) example for how to define customized federated learning algorithm that has high performance on GPUs, one of the key features is to explicitly use `for ... iter(dataset)` for training loops. " + "Now let us define the training loop for EMNIST. Note that `use_experimental_simulation_loop=True` in `tff.learning.algorithms.build_weighted_fed_avg` is suggested for performant TFF simulation, and required to take advantage of multi-GPUs on a single machine. See [simple_fedavg](https://github.com/tensorflow/federated/tree/main/examples/simple_fedavg) example for how to define customized federated learning algorithm that has high performance on GPUs, one of the key features is to explicitly use `for ... iter(dataset)` for training loops. " ] }, { diff --git a/tensorflow_federated/examples/README.md b/examples/README.md similarity index 100% rename from tensorflow_federated/examples/README.md rename to examples/README.md diff --git a/tensorflow_federated/examples/custom_data_backend/BUILD b/examples/custom_data_backend/BUILD similarity index 100% rename from tensorflow_federated/examples/custom_data_backend/BUILD rename to examples/custom_data_backend/BUILD diff --git a/tensorflow_federated/examples/custom_data_backend/data_backend_example.cc b/examples/custom_data_backend/data_backend_example.cc similarity index 100% rename from tensorflow_federated/examples/custom_data_backend/data_backend_example.cc rename to examples/custom_data_backend/data_backend_example.cc diff --git a/tensorflow_federated/examples/custom_data_backend/data_backend_example.h b/examples/custom_data_backend/data_backend_example.h similarity index 100% rename from tensorflow_federated/examples/custom_data_backend/data_backend_example.h rename to examples/custom_data_backend/data_backend_example.h diff --git a/tensorflow_federated/examples/datasets/BUILD b/examples/datasets/BUILD similarity index 100% rename from tensorflow_federated/examples/datasets/BUILD rename to examples/datasets/BUILD diff --git a/tensorflow_federated/examples/datasets/load_flair.py b/examples/datasets/load_flair.py similarity index 100% rename from tensorflow_federated/examples/datasets/load_flair.py rename to examples/datasets/load_flair.py diff --git a/tensorflow_federated/examples/learning/federated_program/vizier/BUILD b/examples/learning/federated_program/vizier/BUILD similarity index 100% rename from tensorflow_federated/examples/learning/federated_program/vizier/BUILD rename to examples/learning/federated_program/vizier/BUILD diff --git a/tensorflow_federated/examples/learning/federated_program/vizier/data_sources.py b/examples/learning/federated_program/vizier/data_sources.py similarity index 100% rename from tensorflow_federated/examples/learning/federated_program/vizier/data_sources.py rename to examples/learning/federated_program/vizier/data_sources.py diff --git a/tensorflow_federated/examples/learning/federated_program/vizier/learning_process.py b/examples/learning/federated_program/vizier/learning_process.py similarity index 100% rename from tensorflow_federated/examples/learning/federated_program/vizier/learning_process.py rename to examples/learning/federated_program/vizier/learning_process.py diff --git a/tensorflow_federated/examples/learning/federated_program/vizier/program.py b/examples/learning/federated_program/vizier/program.py similarity index 95% rename from tensorflow_federated/examples/learning/federated_program/vizier/program.py rename to examples/learning/federated_program/vizier/program.py index 0a9c9c3f69..fd010db623 100644 --- a/tensorflow_federated/examples/learning/federated_program/vizier/program.py +++ b/examples/learning/federated_program/vizier/program.py @@ -36,9 +36,9 @@ from vizier import pyvizier from vizier.client import client_abc -from tensorflow_federated.examples.learning.federated_program.vizier import data_sources -from tensorflow_federated.examples.learning.federated_program.vizier import learning_process -from tensorflow_federated.examples.learning.federated_program.vizier import vizier_service +from exampleslearning.federated_program.vizier import data_sources +from exampleslearning.federated_program.vizier import learning_process +from exampleslearning.federated_program.vizier import vizier_service _TOTAL_ROUNDS = 2 diff --git a/tensorflow_federated/examples/learning/federated_program/vizier/study_spec.textproto b/examples/learning/federated_program/vizier/study_spec.textproto similarity index 100% rename from tensorflow_federated/examples/learning/federated_program/vizier/study_spec.textproto rename to examples/learning/federated_program/vizier/study_spec.textproto diff --git a/tensorflow_federated/examples/learning/federated_program/vizier/vizier_service.py b/examples/learning/federated_program/vizier/vizier_service.py similarity index 100% rename from tensorflow_federated/examples/learning/federated_program/vizier/vizier_service.py rename to examples/learning/federated_program/vizier/vizier_service.py diff --git a/tensorflow_federated/examples/learning/federated_program/vizier/vizier_service_test.py b/examples/learning/federated_program/vizier/vizier_service_test.py similarity index 94% rename from tensorflow_federated/examples/learning/federated_program/vizier/vizier_service_test.py rename to examples/learning/federated_program/vizier/vizier_service_test.py index 9892726d20..a749ddcc41 100644 --- a/tensorflow_federated/examples/learning/federated_program/vizier/vizier_service_test.py +++ b/examples/learning/federated_program/vizier/vizier_service_test.py @@ -17,7 +17,7 @@ from absl.testing import absltest -from tensorflow_federated.examples.learning.federated_program.vizier import vizier_service +from exampleslearning.federated_program.vizier import vizier_service from vizier.service import clients from vizier.service import pyvizier diff --git a/tensorflow_federated/examples/personalization/BUILD b/examples/personalization/BUILD similarity index 100% rename from tensorflow_federated/examples/personalization/BUILD rename to examples/personalization/BUILD diff --git a/tensorflow_federated/examples/personalization/README.md b/examples/personalization/README.md similarity index 92% rename from tensorflow_federated/examples/personalization/README.md rename to examples/personalization/README.md index 6c4d7ef5b6..5353badef4 100644 --- a/tensorflow_federated/examples/personalization/README.md +++ b/examples/personalization/README.md @@ -39,7 +39,7 @@ an unbatched `tf.data.Dataset` for test, and an extra `context` object, trains a personalized model, and returns the evaluation metrics. Users can define whatever personalization strategies they like. An example of fine-tuning based personalization strategy is given by `build_personalize_fn` in -[`p13n_utils`](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/personalization/p13n_utils.py). +[`p13n_utils`](https://github.com/tensorflow/federated/blob/main/examples/personalization/p13n_utils.py). The [`tff.learning.build_personalization_eval_computation`](https://www.tensorflow.org/federated/api_docs/python/tff/learning/build_personalization_eval_computation) @@ -49,7 +49,7 @@ time, starting from the same global model. Specifically, users define a and then pass it to the API as the `personalize_fn_dict` argument. In -[our experiment](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/personalization/emnist_p13n_main.py), +[our experiment](https://github.com/tensorflow/federated/blob/main/examples/personalization/emnist_p13n_main.py), we define and evaluate two fine-tuning based personalization strategies: one uses SGD and other other uses Adam optimizer. @@ -60,7 +60,7 @@ The API has an argument `max_num_samples` with a default value 100. Metrics from at most `max_num_samples` clients (the clients are sampled without replacement) will be collected and returned. In -[our experiment](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/personalization/emnist_p13n_main.py), +[our experiment](https://github.com/tensorflow/federated/blob/main/examples/personalization/emnist_p13n_main.py), we set this value to be larger than the number of clients in the federated dataset, which means that metrics from all clients will be returned. @@ -70,7 +70,7 @@ strategy names to the evaluation metrics of the corresponding personalization strategies. Each returned metric contains a list of scalars (each scalar comes from one sampled client). Metric values at the same position, e.g., `metric_1[i]`, `metric_2[i]`, ..., corresponds to the same client. In -[our experiment](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/personalization/emnist_p13n_main.py), +[our experiment](https://github.com/tensorflow/federated/blob/main/examples/personalization/emnist_p13n_main.py), the baseline evaluation function is given by `evaluate_fn` in -[`p13n_utils`](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/personalization/p13n_utils.py), +[`p13n_utils`](https://github.com/tensorflow/federated/blob/main/examples/personalization/p13n_utils.py), which is the same evaluation function used by the personalization strategies. diff --git a/tensorflow_federated/examples/personalization/emnist_p13n_main.py b/examples/personalization/emnist_p13n_main.py similarity index 98% rename from tensorflow_federated/examples/personalization/emnist_p13n_main.py rename to examples/personalization/emnist_p13n_main.py index 056d7e3cbd..116962b286 100644 --- a/tensorflow_federated/examples/personalization/emnist_p13n_main.py +++ b/examples/personalization/emnist_p13n_main.py @@ -21,8 +21,8 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.personalization.p13n_utils import build_personalize_fn -from tensorflow_federated.examples.personalization.p13n_utils import evaluate_fn +from examplespersonalization.p13n_utils import build_personalize_fn +from examplespersonalization.p13n_utils import evaluate_fn def _get_emnist_datasets() -> ( diff --git a/tensorflow_federated/examples/personalization/p13n_utils.py b/examples/personalization/p13n_utils.py similarity index 100% rename from tensorflow_federated/examples/personalization/p13n_utils.py rename to examples/personalization/p13n_utils.py diff --git a/tensorflow_federated/examples/personalization/p13n_utils_test.py b/examples/personalization/p13n_utils_test.py similarity index 97% rename from tensorflow_federated/examples/personalization/p13n_utils_test.py rename to examples/personalization/p13n_utils_test.py index 06fecd9841..bbf3a48623 100644 --- a/tensorflow_federated/examples/personalization/p13n_utils_test.py +++ b/examples/personalization/p13n_utils_test.py @@ -18,7 +18,7 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.personalization import p13n_utils +from examplespersonalization import p13n_utils _INPUT_DIM = 2 _OUTPUT_DIM = 1 diff --git a/tensorflow_federated/examples/program/BUILD b/examples/program/BUILD similarity index 100% rename from tensorflow_federated/examples/program/BUILD rename to examples/program/BUILD diff --git a/tensorflow_federated/examples/program/computations.py b/examples/program/computations.py similarity index 100% rename from tensorflow_federated/examples/program/computations.py rename to examples/program/computations.py diff --git a/tensorflow_federated/examples/program/program.py b/examples/program/program.py similarity index 97% rename from tensorflow_federated/examples/program/program.py rename to examples/program/program.py index e4e81628fe..18986b839d 100644 --- a/tensorflow_federated/examples/program/program.py +++ b/examples/program/program.py @@ -55,8 +55,8 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.program import computations -from tensorflow_federated.examples.program import program_logic +from examplesprogram import computations +from examplesprogram import program_logic _OUTPUT_DIR = flags.DEFINE_string('output_dir', None, 'The output path.') diff --git a/tensorflow_federated/examples/program/program_logic.py b/examples/program/program_logic.py similarity index 100% rename from tensorflow_federated/examples/program/program_logic.py rename to examples/program/program_logic.py diff --git a/tensorflow_federated/examples/program/program_logic_test.py b/examples/program/program_logic_test.py similarity index 99% rename from tensorflow_federated/examples/program/program_logic_test.py rename to examples/program/program_logic_test.py index 89986ce325..2cffbb3fd8 100644 --- a/tensorflow_federated/examples/program/program_logic_test.py +++ b/examples/program/program_logic_test.py @@ -24,8 +24,8 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.program import computations -from tensorflow_federated.examples.program import program_logic +from examplesprogram import computations +from examplesprogram import program_logic def _create_native_federated_context(): diff --git a/tensorflow_federated/examples/simple_fedavg/BUILD b/examples/simple_fedavg/BUILD similarity index 100% rename from tensorflow_federated/examples/simple_fedavg/BUILD rename to examples/simple_fedavg/BUILD diff --git a/tensorflow_federated/examples/simple_fedavg/README.md b/examples/simple_fedavg/README.md similarity index 70% rename from tensorflow_federated/examples/simple_fedavg/README.md rename to examples/simple_fedavg/README.md index 3b877544a4..ed7f760772 100644 --- a/tensorflow_federated/examples/simple_fedavg/README.md +++ b/examples/simple_fedavg/README.md @@ -11,9 +11,9 @@ for a more full-featured implementation. A minimal implementation of the [Federated Averaging](https://arxiv.org/abs/1602.05629) algorithm is provided -[here](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg), +[here](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg), along with an example -[federated EMNIST experiment](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py). +[federated EMNIST experiment](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py). The implementation demonstrates the three main types of logic of a typical federated learning simulation. @@ -32,59 +32,59 @@ federated learning simulation. This EMNIST example can easily be adapted for experimental changes: * In the driver file - [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py), + [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py), we can change the - [dataset](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L49-L79), + [dataset](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L49-L79), the - [neural network architecture](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L82-L122), + [neural network architecture](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L82-L122), the - [server_optimizer](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L125-L126), + [server_optimizer](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L125-L126), and the - [client_optimizer](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L129-L130) + [client_optimizer](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L129-L130) for cutomized applications. Note that we need a - [model wrapper](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L151-152), + [model wrapper](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L151-152), and build an - [iterative process](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L154-L155) + [iterative process](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L154-L155) with TFF. We define a stand-alone model wrapper for keras models in - [simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L39-L81), + [simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L39-L81), which can be substituted with `tff.learning.models.VariableModel` by calling `tff.learning.models.from_keras_model`. Note that the inner - [keras_model](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L174) + [keras_model](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L174) of `tff.learning.models.VariableModel` may not be directly accessible for evaluation. * In the TF function file - [simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py), + [simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py), we have more control over the local computations performed in optimization process. In each round, on the server side, we will update the - [ServerState](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L102-L113) + [ServerState](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L102-L113) in - [server_update](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L131-L141) + [server_update](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L131-L141) function; we then build a - [BroadcastMessage](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L116-L128) + [BroadcastMessage](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L116-L128) with the - [build_server_broadcast_message](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L165-L181) + [build_server_broadcast_message](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L165-L181) function to prepare for broadcasting from server to clients; on the client side, we perform local updates with the - [client_update](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L184-L222) + [client_update](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L184-L222) function and return - [ClientOutput](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L84-L99) + [ClientOutput](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L84-L99) to be sent back to server. Note that server_optimizer defined in - [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L125-L126) + [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L125-L126) is used in - [server_update](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L131-L141) + [server_update](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L131-L141) function; client_optimizer defined in - [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py#L129-L130) + [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py#L129-L130) is used in - [client_update](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py#L184-L222). + [client_update](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py#L184-L222). These functions are used as local computation building blocks in the overall TFF computation, which handles the broadcasting and aggregation between server and clients. * In the TFF file - [simple_fedavg_tff](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py), + [simple_fedavg_tff](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tff.py), we have control over the orchestration strategy. We take the - [weighted average](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py#L132-L133) + [weighted average](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tff.py#L132-L133) of client updates to update the model kept in server state. More detailed instruction on the usage of TFF functions `federated_broadcast`, `federated_map`, and `federated_mean` can be found in the @@ -92,33 +92,33 @@ This EMNIST example can easily be adapted for experimental changes: We expect researchers working on applications and models only need to change the driver file -[emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py), +[emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py), researchers working on optimization may implement most of the ideas by writing pure TF code in the TF file -[simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py), +[simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py), while researchers who need more control over the orchestration strategy may get familiar with TFF code in -[simple_fedavg_tff](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py). +[simple_fedavg_tff](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tff.py). We encourage readers to consider the following exercises for using this set of code for your research: 1. Try a more complicated server optimizer such as ADAM. You only need to change - [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py). + [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py). 1. Implement a model that uses L2 regularization. You will need to change the model definition in - [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py) + [emnist_fedavg_main](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/emnist_fedavg_main.py) and add Keras regularization losses in the `KerasModelWrapper` class in - [simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py). + [simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py). 1. Implement a decaying learning rate schedule on the clients based on the global round, using the `round_num` broadcasted to the clients in - [simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py). + [simple_fedavg_tf](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tf.py). 1. Implement a more complicated aggregation procedure that drops the client updates with the largest and smallest l2 norms. You will need to change - [simple_fedavg_tff](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py). + [simple_fedavg_tff](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg/simple_fedavg_tff.py). ## Citation diff --git a/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py b/examples/simple_fedavg/emnist_fedavg_main.py similarity index 98% rename from tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py rename to examples/simple_fedavg/emnist_fedavg_main.py index e1562cc607..bc07e9b7d4 100644 --- a/tensorflow_federated/examples/simple_fedavg/emnist_fedavg_main.py +++ b/examples/simple_fedavg/emnist_fedavg_main.py @@ -27,7 +27,7 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.simple_fedavg import simple_fedavg_tff +from examplessimple_fedavg import simple_fedavg_tff # Training hyperparameters flags.DEFINE_integer('total_rounds', 256, 'Number of total training rounds.') diff --git a/tensorflow_federated/examples/simple_fedavg/simple_fedavg_test.py b/examples/simple_fedavg/simple_fedavg_test.py similarity index 99% rename from tensorflow_federated/examples/simple_fedavg/simple_fedavg_test.py rename to examples/simple_fedavg/simple_fedavg_test.py index a2fadbb6bb..120ef99c1e 100644 --- a/tensorflow_federated/examples/simple_fedavg/simple_fedavg_test.py +++ b/examples/simple_fedavg/simple_fedavg_test.py @@ -23,8 +23,8 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.simple_fedavg import simple_fedavg_tf -from tensorflow_federated.examples.simple_fedavg import simple_fedavg_tff +from examplessimple_fedavg import simple_fedavg_tf +from examplessimple_fedavg import simple_fedavg_tff def _create_test_cnn_model(): diff --git a/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py b/examples/simple_fedavg/simple_fedavg_tf.py similarity index 100% rename from tensorflow_federated/examples/simple_fedavg/simple_fedavg_tf.py rename to examples/simple_fedavg/simple_fedavg_tf.py diff --git a/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py b/examples/simple_fedavg/simple_fedavg_tff.py similarity index 93% rename from tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py rename to examples/simple_fedavg/simple_fedavg_tff.py index ac9f3094b1..155a6ecff6 100644 --- a/tensorflow_federated/examples/simple_fedavg/simple_fedavg_tff.py +++ b/examples/simple_fedavg/simple_fedavg_tff.py @@ -29,12 +29,12 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.simple_fedavg.simple_fedavg_tf import batch_client_update -from tensorflow_federated.examples.simple_fedavg.simple_fedavg_tf import build_server_broadcast_message -from tensorflow_federated.examples.simple_fedavg.simple_fedavg_tf import client_update -from tensorflow_federated.examples.simple_fedavg.simple_fedavg_tf import init_client_ouput -from tensorflow_federated.examples.simple_fedavg.simple_fedavg_tf import server_update -from tensorflow_federated.examples.simple_fedavg.simple_fedavg_tf import ServerState +from examplessimple_fedavg.simple_fedavg_tf import batch_client_update +from examplessimple_fedavg.simple_fedavg_tf import build_server_broadcast_message +from examplessimple_fedavg.simple_fedavg_tf import client_update +from examplessimple_fedavg.simple_fedavg_tf import init_client_ouput +from examplessimple_fedavg.simple_fedavg_tf import server_update +from examplessimple_fedavg.simple_fedavg_tf import ServerState def _initialize_optimizer_vars( diff --git a/tensorflow_federated/examples/stateful_clients/BUILD b/examples/stateful_clients/BUILD similarity index 100% rename from tensorflow_federated/examples/stateful_clients/BUILD rename to examples/stateful_clients/BUILD diff --git a/tensorflow_federated/examples/stateful_clients/README.md b/examples/stateful_clients/README.md similarity index 96% rename from tensorflow_federated/examples/stateful_clients/README.md rename to examples/stateful_clients/README.md index bb8526c71f..a01045252e 100644 --- a/tensorflow_federated/examples/stateful_clients/README.md +++ b/examples/stateful_clients/README.md @@ -9,7 +9,7 @@ protection. This project is based on the standalone implementaion of Federated Averaging algorithm in -[`simple_fedavg`](https://github.com/tensorflow/federated/blob/main/tensorflow_federated/examples/simple_fedavg). +[`simple_fedavg`](https://github.com/tensorflow/federated/blob/main/examples/simple_fedavg). We introduce a coutner on each client, which tracks the total number of iterations for model training on the clients. For example, if client A has been sampled m times at round n, and each time local model training has been run with diff --git a/tensorflow_federated/examples/stateful_clients/emnist_fedavg_main.py b/examples/stateful_clients/emnist_fedavg_main.py similarity index 97% rename from tensorflow_federated/examples/stateful_clients/emnist_fedavg_main.py rename to examples/stateful_clients/emnist_fedavg_main.py index 8851806015..a8b204ba7a 100644 --- a/tensorflow_federated/examples/stateful_clients/emnist_fedavg_main.py +++ b/examples/stateful_clients/emnist_fedavg_main.py @@ -25,8 +25,8 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.stateful_clients import stateful_fedavg_tf -from tensorflow_federated.examples.stateful_clients import stateful_fedavg_tff +from examplesstateful_clients import stateful_fedavg_tf +from examplesstateful_clients import stateful_fedavg_tff # Training hyperparameters flags.DEFINE_integer('total_rounds', 256, 'Number of total training rounds.') diff --git a/tensorflow_federated/examples/stateful_clients/stateful_fedavg_test.py b/examples/stateful_clients/stateful_fedavg_test.py similarity index 98% rename from tensorflow_federated/examples/stateful_clients/stateful_fedavg_test.py rename to examples/stateful_clients/stateful_fedavg_test.py index a5050955f3..c26039a07e 100644 --- a/tensorflow_federated/examples/stateful_clients/stateful_fedavg_test.py +++ b/examples/stateful_clients/stateful_fedavg_test.py @@ -22,8 +22,8 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.stateful_clients import stateful_fedavg_tf -from tensorflow_federated.examples.stateful_clients import stateful_fedavg_tff +from examplesstateful_clients import stateful_fedavg_tf +from examplesstateful_clients import stateful_fedavg_tff def _create_test_cnn_model(): diff --git a/tensorflow_federated/examples/stateful_clients/stateful_fedavg_tf.py b/examples/stateful_clients/stateful_fedavg_tf.py similarity index 100% rename from tensorflow_federated/examples/stateful_clients/stateful_fedavg_tf.py rename to examples/stateful_clients/stateful_fedavg_tf.py diff --git a/tensorflow_federated/examples/stateful_clients/stateful_fedavg_tff.py b/examples/stateful_clients/stateful_fedavg_tff.py similarity index 98% rename from tensorflow_federated/examples/stateful_clients/stateful_fedavg_tff.py rename to examples/stateful_clients/stateful_fedavg_tff.py index 2db8323521..626867eff1 100644 --- a/tensorflow_federated/examples/stateful_clients/stateful_fedavg_tff.py +++ b/examples/stateful_clients/stateful_fedavg_tff.py @@ -30,7 +30,7 @@ import tensorflow as tf import tensorflow_federated as tff -from tensorflow_federated.examples.stateful_clients import stateful_fedavg_tf +from examplesstateful_clients import stateful_fedavg_tf def _initialize_optimizer_vars(model, optimizer):