diff --git a/README.md b/README.md index 2674d52..b3b6e0f 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ A cookiecutter is a generic project template that will instantiate a new project * Flake8 * Pytest -More information on what a cookiecutter is [here.](https://cookiecutter.readthedocs.io/en/) +More information on what a cookiecutter is [here.](https://cookiecutter.readthedocs.io) Quickstart ---------- diff --git a/cookiecutter.json b/cookiecutter.json index 3e6909c..67f79ec 100644 --- a/cookiecutter.json +++ b/cookiecutter.json @@ -7,6 +7,7 @@ "project_short_description": "{{ cookiecutter.project_name }} is wonderful!", "python_version": "3.8", "dl_framework": ["pytorch", "tensorflow_cpu", "tensorflow_gpu"], + "environment": ["generic", "mila"], "pypi_username": "{{ cookiecutter.github_username }}", "version": "0.0.1", "open_source_license": ["MIT license", "BSD license", "ISC license", "Apache Software License 2.0", "GNU General Public License v3", "Not open source"] diff --git a/{{cookiecutter.project_slug}}/.github/workflows/tests.yml b/{{cookiecutter.project_slug}}/.github/workflows/tests.yml index 7f65188..6800e41 100644 --- a/{{cookiecutter.project_slug}}/.github/workflows/tests.yml +++ b/{{cookiecutter.project_slug}}/.github/workflows/tests.yml @@ -4,11 +4,11 @@ on: # but only for the main/develop branch push: branches: - - master + - main - develop pull_request: branches: - - master + - main - develop jobs: build: diff --git a/{{cookiecutter.project_slug}}/README.md b/{{cookiecutter.project_slug}}/README.md index 4180275..9a594a6 100644 --- a/{{cookiecutter.project_slug}}/README.md +++ b/{{cookiecutter.project_slug}}/README.md @@ -1,5 +1,3 @@ -[![Build Status](https://travis-ci.com/{{ cookiecutter.github_username }}/{{ cookiecutter.project_slug }}.png?branch=master)](https://travis-ci.com/{{ cookiecutter.github_username }}/{{ cookiecutter.project_slug }}) - {% set is_open_source = cookiecutter.open_source_license != 'Not open source' -%} # {{ cookiecutter.project_name }} @@ -46,9 +44,12 @@ These hooks will: Go on github and follow the instructions to create a new project. When done, do not add any file, and follow the instructions to link your local git to the remote project, which should look like this: +(PS: these instructions are reported here for your convenience. +We suggest to also look at the GitHub project page for more up-to-date info) git remote add origin git@github.com:{{ cookiecutter.github_username }}/{{ cookiecutter.project_slug }}.git - git push -u origin master + git branch -M main + git push -u origin main ### Setup Continuous Integration @@ -66,7 +67,7 @@ Check the following instructions for more details. Github actions are already configured in `.github/workflows/tests.yml`. Github actions are already enabled by default when using Github, so, when pushing to github, they will be executed automatically for pull requests to -`master` and to `develop`. +`main` and to `develop`. #### Travis @@ -120,12 +121,10 @@ Note you have two new folders now: You can run mlflow from this folder (`examples/local`) by running `mlflow ui`. -#### Run on the Mila cluster -(NOTE: this example also apply to Compute Canada - use the folders -`slurm_cc` and `slurm_cc_orion` instead of `slurm_mila` and `slurm_mila_orion`.) +#### Run on a remote cluster (with Slurm) -First, bring you project on the Mila cluster (assuming you didn't create your -project directly there). To do so, simply login on the Mila cluster and git +First, bring you project on the cluster (assuming you didn't create your +project directly there). To do so, simply login on the cluster and git clone your project: git clone git@github.com:{{ cookiecutter.github_username }}/{{ cookiecutter.project_slug }}.git @@ -135,12 +134,13 @@ Then activate your virtual env, and install the dependencies: cd {{ cookiecutter.project_slug }} pip install -e . -To run with SLURM, just: +To run with Slurm, just: - cd examples/slurm_mila + cd examples/slurm sh run.sh Check the log to see that you got an almost perfect loss (i.e., 0). +{%- if cookiecutter.environment == 'mila' %} #### Measure GPU time (and others) on the Mila cluster @@ -184,11 +184,12 @@ In a separate shell on your local computer, run the following command: where `` is your user name on the Mila cluster and `` is the name of the machine your job is currenty running on (`leto35` in our example). You can then navigate your local browser to `http://localhost:19999/` to view the ressources being used on the cluster and monitor your job. You should see something like this: ![image](https://user-images.githubusercontent.com/18450628/88088807-fe2acd80-cb58-11ea-8ab2-bd090e8a826c.png) +{%- endif %} -#### Run with Orion on the Mila cluster +#### Run with Orion on the Slurm cluster This example will run orion for 2 trials (see the orion config file). -To do so, go into `examples/slurm_mila_orion`. +To do so, go into `examples/slurm_orion`. Here you can find the orion config file (`orion_config.yaml`), as well as the config file (`config.yaml`) for your project (that contains the hyper-parameters). @@ -204,7 +205,7 @@ Inside these folders, you can find the models (the best one and the last one), t the hyper-parameters for this trial, and the log file. You can check orion status with the following commands: -(to be run from `examples/slurm_mila_orion`) +(to be run from `examples/slurm_orion`) export ORION_DB_ADDRESS='orion_db.pkl' export ORION_DB_TYPE='pickleddb' diff --git a/{{cookiecutter.project_slug}}/examples/slurm_cc/config.yaml b/{{cookiecutter.project_slug}}/examples/slurm/config.yaml similarity index 100% rename from {{cookiecutter.project_slug}}/examples/slurm_cc/config.yaml rename to {{cookiecutter.project_slug}}/examples/slurm/config.yaml diff --git a/{{cookiecutter.project_slug}}/examples/slurm_cc/run.sh b/{{cookiecutter.project_slug}}/examples/slurm/run.sh similarity index 100% rename from {{cookiecutter.project_slug}}/examples/slurm_cc/run.sh rename to {{cookiecutter.project_slug}}/examples/slurm/run.sh diff --git a/{{cookiecutter.project_slug}}/examples/slurm_mila/to_submit.sh b/{{cookiecutter.project_slug}}/examples/slurm/to_submit.sh similarity index 55% rename from {{cookiecutter.project_slug}}/examples/slurm_mila/to_submit.sh rename to {{cookiecutter.project_slug}}/examples/slurm/to_submit.sh index fed5b77..c87cc12 100644 --- a/{{cookiecutter.project_slug}}/examples/slurm_mila/to_submit.sh +++ b/{{cookiecutter.project_slug}}/examples/slurm/to_submit.sh @@ -1,5 +1,16 @@ #!/bin/bash -#SBATCH --partition=long +{%- if cookiecutter.environment == 'mila' %} +## this is for the mila cluster (uncomment it if you need it): +##SBATCH --account=rrg-bengioy-ad +## this instead for ComputCanada (uncomment it if you need it): +##SBATCH --partition=long +# to attach a tag to your run (e.g., used to track the GPU time) +# uncomment the following line and add replace `my_tag` with the proper tag: +##SBATCH --wckey=my_tag +{%- endif %} +{%- if cookiecutter.environment == 'generic' %} +## set --account=... or --partition=... as needed. +{%- endif %} #SBATCH --cpus-per-task=2 #SBATCH --gres=gpu:1 #SBATCH --mem=5G diff --git a/{{cookiecutter.project_slug}}/examples/slurm_cc/to_submit.sh b/{{cookiecutter.project_slug}}/examples/slurm_cc/to_submit.sh deleted file mode 100644 index 5e33447..0000000 --- a/{{cookiecutter.project_slug}}/examples/slurm_cc/to_submit.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/bin/bash -#SBATCH --account=rrg-bengioy-ad -#SBATCH --cpus-per-task=2 -#SBATCH --gres=gpu:1 -#SBATCH --mem=5G -#SBATCH --time=0:05:00 -#SBATCH --job-name={{ cookiecutter.project_slug }} -#SBATCH --output=logs/%x__%j.out -#SBATCH --error=logs/%x__%j.err -# remove one # if you prefer receiving emails -##SBATCH --mail-type=all -##SBATCH --mail-user={{ cookiecutter.email }} - -export MLFLOW_TRACKING_URI='mlruns' - -main --data ../data --output output --config config.yaml --tmp-folder ${SLURM_TMPDIR} --disable-progressbar diff --git a/{{cookiecutter.project_slug}}/examples/slurm_cc_orion/to_submit.sh b/{{cookiecutter.project_slug}}/examples/slurm_cc_orion/to_submit.sh deleted file mode 100644 index 209fecb..0000000 --- a/{{cookiecutter.project_slug}}/examples/slurm_cc_orion/to_submit.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/bin/bash -# __TODO__ fix options if needed -#SBATCH --job-name={{ cookiecutter.project_slug }} -#SBATCH --account=rrg-bengioy-ad -#SBATCH --cpus-per-task=2 -#SBATCH --gres=gpu:1 -#SBATCH --mem=5G -#SBATCH --time=0:05:00 -#SBATCH --output=logs/%x__%j.out -#SBATCH --error=logs/%x__%j.err -# remove one # if you prefer receiving emails -##SBATCH --mail-type=all -##SBATCH --mail-user={{ cookiecutter.email }} - -export MLFLOW_TRACKING_URI='mlruns' -export ORION_DB_ADDRESS='orion_db.pkl' -export ORION_DB_TYPE='pickleddb' - -orion -v hunt --config orion_config.yaml \ - main --data ../data --config config.yaml --disable-progressbar \ - --output '{exp.working_dir}/{exp.name}_{trial.id}/' \ - --log '{exp.working_dir}/{exp.name}_{trial.id}/exp.log' \ - --tmp-folder ${SLURM_TMPDIR} diff --git a/{{cookiecutter.project_slug}}/examples/slurm_mila/config.yaml b/{{cookiecutter.project_slug}}/examples/slurm_mila/config.yaml deleted file mode 100644 index 2e58acc..0000000 --- a/{{cookiecutter.project_slug}}/examples/slurm_mila/config.yaml +++ /dev/null @@ -1,14 +0,0 @@ -# general -batch_size: 32 -optimizer: adam -loss: L1 -patience: 5 -architecture: my_model -max_epoch: 99 -exp_name: my_exp_1 -# set to null to avoid setting a seed (can speed up GPU computation, but -# results will not be reproducible) -seed: 1234 - -# architecture -size: 10 diff --git a/{{cookiecutter.project_slug}}/examples/slurm_mila/run.sh b/{{cookiecutter.project_slug}}/examples/slurm_mila/run.sh deleted file mode 100644 index 9370362..0000000 --- a/{{cookiecutter.project_slug}}/examples/slurm_mila/run.sh +++ /dev/null @@ -1,2 +0,0 @@ -mkdir -p logs -sbatch to_submit.sh diff --git a/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/config.yaml b/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/config.yaml deleted file mode 100644 index 5c0028c..0000000 --- a/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/config.yaml +++ /dev/null @@ -1,14 +0,0 @@ -# general -batch_size: 32 -optimizer: adam -loss: L1 -patience: 5 -architecture: my_model -max_epoch: 99 -exp_name: my_exp_1 -# set to null to avoid setting a seed (can speed up GPU computation, but -# results will not be reproducible) -seed: 1234 - -# architecture -size: 'orion~uniform(1,100,discrete=True)' diff --git a/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/orion_config.yaml b/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/orion_config.yaml deleted file mode 100644 index f6bd2e1..0000000 --- a/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/orion_config.yaml +++ /dev/null @@ -1,16 +0,0 @@ -experiment: - name: - my_exp - max_trials: 2 - working_dir: - orion_working_dir - algorithms: - random: - seed: 1234 -evc: - non_monitored_arguments: - - output - - data - - tmp-folder - ignore_code_changes: - true diff --git a/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/run.sh b/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/run.sh deleted file mode 100644 index 9370362..0000000 --- a/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/run.sh +++ /dev/null @@ -1,2 +0,0 @@ -mkdir -p logs -sbatch to_submit.sh diff --git a/{{cookiecutter.project_slug}}/examples/slurm_cc_orion/config.yaml b/{{cookiecutter.project_slug}}/examples/slurm_orion/config.yaml similarity index 100% rename from {{cookiecutter.project_slug}}/examples/slurm_cc_orion/config.yaml rename to {{cookiecutter.project_slug}}/examples/slurm_orion/config.yaml diff --git a/{{cookiecutter.project_slug}}/examples/slurm_cc_orion/orion_config.yaml b/{{cookiecutter.project_slug}}/examples/slurm_orion/orion_config.yaml similarity index 100% rename from {{cookiecutter.project_slug}}/examples/slurm_cc_orion/orion_config.yaml rename to {{cookiecutter.project_slug}}/examples/slurm_orion/orion_config.yaml diff --git a/{{cookiecutter.project_slug}}/examples/slurm_cc_orion/run.sh b/{{cookiecutter.project_slug}}/examples/slurm_orion/run.sh similarity index 100% rename from {{cookiecutter.project_slug}}/examples/slurm_cc_orion/run.sh rename to {{cookiecutter.project_slug}}/examples/slurm_orion/run.sh diff --git a/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/to_submit.sh b/{{cookiecutter.project_slug}}/examples/slurm_orion/to_submit.sh similarity index 70% rename from {{cookiecutter.project_slug}}/examples/slurm_mila_orion/to_submit.sh rename to {{cookiecutter.project_slug}}/examples/slurm_orion/to_submit.sh index a6e669e..1143f83 100644 --- a/{{cookiecutter.project_slug}}/examples/slurm_mila_orion/to_submit.sh +++ b/{{cookiecutter.project_slug}}/examples/slurm_orion/to_submit.sh @@ -1,16 +1,23 @@ #!/bin/bash -# __TODO__ fix options if needed #SBATCH --job-name={{ cookiecutter.project_slug }} -#SBATCH --partition=long +{%- if cookiecutter.environment == 'mila' %} +## this is for the mila cluster (uncomment it if you need it): +##SBATCH --account=rrg-bengioy-ad +## this instead for ComputCanada (uncomment it if you need it): +##SBATCH --partition=long +# to attach a tag to your run (e.g., used to track the GPU time) +# uncomment the following line and add replace `my_tag` with the proper tag: +##SBATCH --wckey=my_tag +{%- endif %} +{%- if cookiecutter.environment == 'generic' %} +## set --account=... or --partition=... as needed. +{%- endif %} #SBATCH --cpus-per-task=2 #SBATCH --gres=gpu:1 #SBATCH --mem=5G #SBATCH --time=0:05:00 #SBATCH --output=logs/%x__%j.out #SBATCH --error=logs/%x__%j.err -# to attach a tag to your run (e.g., used to track the GPU time) -# uncomment the following line and add replace `my_tag` with the proper tag: -##SBATCH --wckey=my_tag # remove one # if you prefer receiving emails ##SBATCH --mail-type=all ##SBATCH --mail-user={{ cookiecutter.email }}