Skip to content

Commit

Permalink
github/actions: rolling deploy
Browse files Browse the repository at this point in the history
Closes: #666
  • Loading branch information
tharvik committed May 13, 2024
1 parent 08168bd commit b7c2a68
Show file tree
Hide file tree
Showing 9 changed files with 110 additions and 202 deletions.
47 changes: 0 additions & 47 deletions .github/workflows/deploy-server.yml

This file was deleted.

50 changes: 0 additions & 50 deletions .github/workflows/github-pages.yml

This file was deleted.

5 changes: 4 additions & 1 deletion .github/workflows/lint-test-build.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
name: lint-test-build
on:
push:
branches: production
branches: [develop]
pull_request:

permissions:
contents: read

jobs:
download-datasets:
runs-on: ubuntu-latest
Expand Down
94 changes: 94 additions & 0 deletions .github/workflows/publish-deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: publish-deploy
on:
push:
branches: [develop]

jobs:
publish-npm:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
cache: npm
registry-url: "https://registry.npmjs.org"
scope: "@epfml"
- run: npm ci
- run: npm --workspace=discojs{,-node,-web} version prerelease --preid=p`date +%Y%m%d%H%M%S`
- run: npm --workspace=discojs{,-node,-web} run build
- run: npm --workspace=discojs{,-node,-web} publish --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

build-webapp:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
cache: npm
- run: npm ci
- run: npm --workspace={discojs,discojs-web,webapp} run build
- uses: actions/upload-pages-artifact@v3
with:
path: webapp/dist
deploy-pages:
needs: build-webapp
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v4

publish-github-container:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
outputs:
digest: ${{ steps.build.outputs.digest }}
steps:
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/epfml/disco
- uses: docker/build-push-action@v5
id: build
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

deploy-server:
needs: [publish-github-container]
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v4
- uses: google-github-actions/auth@v2
with:
project_id: disco-367212
workload_identity_provider: projects/905419434797/locations/global/workloadIdentityPools/github/providers/disco
- uses: google-github-actions/deploy-cloudrun@v2
with:
service: disco
region: europe-west6
image: europe-west6-docker.pkg.dev/disco-367212/github/epfml/disco@${{ needs.publish-github-container.outputs.digest }}
92 changes: 0 additions & 92 deletions .github/workflows/publish.yml

This file was deleted.

10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

DISCO leverages federated :star2: and decentralized :sparkles: learning to allow several data owners to collaboratively build machine learning models without sharing any original data.

The latest version is always running on the following link, [directly in your browser](https://epfml.github.io/disco/), for web and mobile:
The latest version is always running on the following link, [directly in your browser](https://discolab.ai/), for web and mobile:

:man_dancing: [https://epfml.github.io/disco/](https://epfml.github.io/disco/) :man_dancing:
:man_dancing: [https://discolab.ai/](https://discolab.ai/) :man_dancing:

___
:magic_wand: DEVELOPERS: Have a look at our [developer guide](DEV.md)
Expand All @@ -24,7 +24,7 @@ ___
- a central server : **federated** learning ( :star2: )
- directly between users : **decentralized** learning ( :sparkles: ) i.e. no central coordination
- Model updates are then securely aggregated into a trained model
- See more [HERE](https://epfml.github.io/disco/#/information)
- See more [HERE](https://discolab.ai/#/information)

___
:question: **DISCO TECHNOLOGY**
Expand All @@ -50,9 +50,9 @@ ___


:checkered_flag: **HOW TO USE DISCO**
- Start by exploring our example *DISCOllaboratives* in the [`Tasks` page](https://epfml.github.io/disco/#/list).
- Start by exploring our example *DISCOllaboratives* in the [`Tasks` page](https://discolab.ai/#/list).
- The example models are based on popular datasets such as [Titanic](https://www.kaggle.com/c/titanic), [MNIST](https://www.kaggle.com/c/digit-recognizer) or [CIFAR-10](https://www.kaggle.com/pankrzysiu/cifar10-python)
- It is also possible to create your own task without coding on the [custom training page](https://epfml.github.io/disco/#/create):
- It is also possible to create your own task without coding on the [custom training page](https://discolab.ai/#/create):
- Upload the initial model
- You can choose from several existing dataloaders
- Choose between federated and decentralized for your DISCO training scheme ... connect your data and... done! :bar_chart:
Expand Down
4 changes: 2 additions & 2 deletions docs/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ with both to a certain extent. If not, the following references might be useful:
## First steps

DISCO is a complex project composed of the Disco.js library (`discojs`, `discojs-node` and `discojs-node`), a front-end (`webapp`),
DISCO is a complex project composed of the Disco.js library (`discojs`, `discojs-node` and `discojs-web`), a front-end (`webapp`),
a `server` and a `cli` (e.g., for benchmarking). Depending on what your goal is, you might only use a subset of them, e.g. you won't need an in-depth understanding of the webapp and Vue.js to add a new decentralized learning feature. Instead, you will probably rely on the CLI.

1. If you are going to work, contribute and improve the project, I first recommend you get a good understand of what DISCO does: play around with the [website](https://epfml.github.io/disco/#/), train a model from the pre-defined tasks, or even create your own custom task. Feedback is always appreciated, feel free to let us know on slack/in the github issues/in person if you noticed any issues or thought of an improvement.
1. If you are going to work, contribute and improve the project, I first recommend you get a good understand of what DISCO does: play around with the [website](https://discolab.ai/#/), train a model from the pre-defined tasks, or even create your own custom task. Feedback is always appreciated, feel free to let us know on slack/in the github issues/in person if you noticed any issues or thought of an improvement.

2. Then, get a high-level understanding of the different parts of the projects in the [developer guide](../DEV.md), even if you're planning on working on a subset of the project. If you want to know more about a specific part of the project, refer to the table of contents at the end of the DEV guide.

Expand Down
8 changes: 4 additions & 4 deletions docs/TASK.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
DISCO currently allows learning of arbitrary machine learning tasks, where tasks can be defined in three possible ways:

1. **Predefined tasks**: As examples, DISCO already hosts several pre-defined popular tasks such as [Titanic](../discojs/src/tasks/titanic.ts), [CIFAR-10](../discojs/src/tasks/cifar10.ts), and [MNIST](../discojs/src/tasks/mnist.ts) among others.
2. **Task creation UI**: new tasks can be defined via the [**task creation form**](https://epfml.github.io/disco/#/create)
2. **Task creation UI**: new tasks can be defined via the [**task creation form**](https://discolab.ai/#/create)
3. **Implementing custom tasks**: tasks too specific for the UI form need to be implemented in the repository directly.

In any case, one user needs to upload the initial model that is going to be trained collaboratively.
Expand Down Expand Up @@ -38,13 +38,13 @@ tensorflowjs_converter --input_format=tf_saved_model my_tensorflow_saved_model /

## 1. Predefined tasks

Predefined tasks are example use cases available in the [DISCO website](https://epfml.github.io/disco/#/list) where users can upload their respective data and train collaboratively. For predefined tasks, the initial model to train is already defined and doesn't need to be uploaded.
Predefined tasks are example use cases available in the [DISCO website](https://discolab.ai/#/list) where users can upload their respective data and train collaboratively. For predefined tasks, the initial model to train is already defined and doesn't need to be uploaded.

## 2. Task creation UI

The [task creation form](https://epfml.github.io/disco/#/create) lets users create a custom task DISCO without programming. In this case, users can choose between the data modalities and preprocessing that are already supported (such as tabular, images, text etc) and upload an initial model.
The [task creation form](https://discolab.ai/#/create) lets users create a custom task DISCO without programming. In this case, users can choose between the data modalities and preprocessing that are already supported (such as tabular, images, text etc) and upload an initial model.

1. On the [DISCO website](https://epfml.github.io/disco/#), click on `Get Started` and then `Create`.
1. On the [DISCO website](https://discolab.ai/), click on `Get Started` and then `Create`.
2. Fill in all the relevant information for the task
3. Upload model files: 1) a TF.js architecture file in JSON format (cf. the _Uploading ML models_ section) as well as a weight file (`.bin` format), which is necessary in this case. This is the initial weights provided to new users joining your task (pre-trained or random initialization).

Expand Down
2 changes: 1 addition & 1 deletion webapp/.env.production
Original file line number Diff line number Diff line change
@@ -1 +1 @@
VITE_SERVER_URL=https://deai-313515.ew.r.appspot.com
VITE_SERVER_URL=https://disco-zbkj3i466a-oa.a.run.app

0 comments on commit b7c2a68

Please sign in to comment.