Skip to content

Latest commit

 

History

History
43 lines (26 loc) · 1.49 KB

README.rst

File metadata and controls

43 lines (26 loc) · 1.49 KB

Asynchronous algorithms for DL

We implemented the following asynchronous algorithms for decentralized learning:

  • ADPSGD
  • Gossip Learning
  • Super Gossip Learning (our version of Gossip Learning)

In the tutorial folder, there are different running scripts for each algorithm.

Important: For ADPSGD experiments, the graph must be bipartite.

Before running the code

Important parameters to set in the config.ini file:

  • [SHARING] section

    sharing_package = decentralizepy.sharing.AsyncSharing.{ADPSGDSharing, GossipSharing}
sharing_class = {ADPSGDSharing, GossipSharing}

  • [ASYNC_PARAMS] section

    • training_time: the amount of time in minutes that the algorithm should run for
    • timeout: time in seconds, used only in gossip learning
    • eval_on_train_set: if set to True, the train loss is calculated every train_evaluate_after rounds.
    • eval_on_test_set: if set to True, the model is evaluated on the test set every test_after rounds. Otherwise, the model gets saved in a file.

Evaluation on test set

In the read_and_test folder there is a python script for reading the saved models and evaluating them on the test set of CIFAR10 dataset. After the execution of read_and_test.py, a json file is generated for each node containing test accuracy and test loss with respect to iterations and time.