Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: move PyPI deps installation to conda #565

Merged
merged 43 commits into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
84d05e7
move pyg optional deps to conda
gcroci2 Feb 6, 2024
fd5e743
add conda installation of pip deps
gcroci2 Feb 6, 2024
181d6e2
remove pdb2sql from conda installations
gcroci2 Feb 6, 2024
cf19db6
finish adding conda installation for the required deps
gcroci2 Feb 6, 2024
6d367d7
uniform specifications in conda deps
gcroci2 Feb 8, 2024
ca672f1
insert >= for pytorch and its deps
gcroci2 Feb 8, 2024
484496f
fix pytorch version to 2.1.1
gcroci2 Feb 8, 2024
a02b4f8
insert >= for pyg and its deps
gcroci2 Feb 8, 2024
b2e4c27
try to do conda installations with yml file
gcroci2 Feb 8, 2024
e2c3f72
change action to miniconda
gcroci2 Feb 8, 2024
489d9bc
add bash property
gcroci2 Feb 8, 2024
7d4fd5d
add -el
gcroci2 Feb 8, 2024
edcf798
try to fix env activation
gcroci2 Feb 8, 2024
03461dc
try to fix again conda env activation
gcroci2 Feb 8, 2024
fa4d3f8
specify sbl channel for dssp for fixing the error
gcroci2 Feb 8, 2024
8f66c6d
remove deps from pyproject
gcroci2 Feb 13, 2024
d27cefe
fix coveralls and linting actions
gcroci2 Feb 13, 2024
e9a7ec4
reorganize env files
gcroci2 Feb 15, 2024
32825be
edit env name in package action
gcroci2 Feb 15, 2024
7a591f8
separate docker env files
gcroci2 Feb 15, 2024
19f8313
update docker file with new env files names
gcroci2 Feb 15, 2024
94b4194
update action.yml with the renamed env file
gcroci2 Feb 15, 2024
6ab611d
add pdb2sql conda installation and remove the pip installation
gcroci2 Feb 19, 2024
43fb72d
fix error in data processing tutorials notebooks
gcroci2 Feb 19, 2024
f10336a
update docs for new installation
gcroci2 Feb 19, 2024
2e621b5
merge with main
gcroci2 Feb 19, 2024
0796bc4
test for rerunning codacy
gcroci2 Feb 19, 2024
7d4247d
remove test file
gcroci2 Feb 19, 2024
d8ea706
spell out NL as The Netherlands in the affiliations
gcroci2 Feb 19, 2024
e954a15
add missing dois
gcroci2 Feb 19, 2024
c45f828
Merge pull request #571 from DeepRank/joss_acceptance
gcroci2 Feb 20, 2024
869e3fe
remove readme.dev.md
gcroci2 Feb 22, 2024
1f7ce09
bump2version patch
gcroci2 Feb 22, 2024
d0c4e36
update release date
gcroci2 Feb 22, 2024
d45d275
readd readme.dev.md
gcroci2 Feb 22, 2024
39cb127
fix link in the contributing.rst
gcroci2 Feb 22, 2024
a02013c
Merge pull request #574 from DeepRank/570_new_patch_release_gcroci2
gcroci2 Feb 22, 2024
b8948d1
Merge branch 'dev' into 559_conda_migration_gcroci2
gcroci2 Feb 22, 2024
27edea2
update deeprank2 minimum version for docker
gcroci2 Feb 22, 2024
d5ede9f
Update README.md
gcroci2 Feb 26, 2024
f755d41
uniform installation title
Feb 26, 2024
9e30a02
add python version specification
Feb 27, 2024
b4a8bc9
update the section title
gcroci2 Feb 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 3.0.0
current_version = 3.0.1

[comment]
comment = The contents of this file cannot be merged with that of setup.cfg until https://github.com/c4urself/bump2version/issues/185 is resolved
Expand Down
73 changes: 23 additions & 50 deletions .github/actions/install-python-and-package/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,70 +26,43 @@ runs:
uses: styfle/[email protected]
with:
access_token: ${{ github.token }}

- uses: actions/checkout@v3
- name: Setup conda
uses: s-weigand/setup-conda@v1

- name: Setup miniconda
uses: conda-incubator/setup-miniconda@v2
with:
update-conda: true
auto-update-conda: true
miniforge-variant: Mambaforge
channels: conda-forge
python-version: ${{ inputs.python-version }}
conda-channels: pytorch, pyg, bioconda, defaults, sbl, conda-forge
activate-environment: deeprank2
environment-file: env/deeprank2.yml
use-mamba: true

- run: |
conda --version
conda env list
shell: bash {0}
shell: bash -l {0}

- name: Python info
shell: bash -e {0}
shell: bash -l {0}
run: |
which python3
python3 --version
- name: Install dependencies on Linux
shell: bash {0}
env:
CMAKE_INSTALL_PREFIX: .local
if: runner.os == 'Linux'
run: |
# Install deeprank2 conda dependencies
## DSSP
conda install -c sbl dssp>=4.2.2.1
## MSMS
conda install -c bioconda msms>=2.6.1
## PyTorch, PyG, PyG adds
### Installing for CPU only on the CI
conda install pytorch=2.1.1 torchvision=0.16.1 torchaudio=2.1.1 cpuonly=2.0.* -c pytorch
conda install pyg=2.4.0 -c pyg
pip install torch_scatter==2.1.2 torch_sparse==0.6.18 torch_cluster==1.6.3 torch_spline_conv==1.2.2 -f https://data.pyg.org/whl/torch-2.1.0+cpu.html
- name: Install dependencies on MacOS
shell: bash {0}
env:
CMAKE_INSTALL_PREFIX: .local
if: runner.os == 'macOS'
run: |
# Install dependencies not handled by setuptools
## DSSP
conda install -c sbl dssp>=4.2.2.1
## MSMS
cd /tmp/
wget http://mgltools.scripps.edu/downloads/tars/releases/MSMSRELEASE/REL2.6.1/msms_i86Linux2_2.6.1.tar.gz
sudo mkdir /usr/local/lib/msms
cd /usr/local/lib/msms
sudo tar zxvf /tmp/msms_i86Linux2_2.6.1.tar.gz
sudo ln -s /usr/local/lib/msms/msms.i86Linux2.2.6.1 /usr/local/bin/msms
sudo ln -s /usr/local/lib/msms/pdb_to_xyzr* /usr/local/bin
## PyTorch, PyG, PyG adds
### Installing for CPU only on the CI
conda install pytorch torchvision torchaudio cpuonly -c pytorch
pip install torch_geometric
pip install torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-$(python3 -c "import torch; print(torch.__version__)")+cpu.html
# PyTables via conda only for MacOS
conda install pytables

- name: Install the GitHub repository version of the package
shell: bash {0}
shell: bash -l {0}
if: ${{ inputs.pkg-installation-type == 'repository' }}
run: pip install .'[${{ inputs.extras-require }}]'
- name: Install the latest released version of the package
shell: bash {0}
run: |
conda activate deeprank2
pip install .'[${{ inputs.extras-require }}]'

- name: Install the latest released PyPI version of the package
shell: bash -l {0}
if: ${{ inputs.pkg-installation-type == 'latest' }}
run: |
conda activate deeprank2
pip install pytest
rm -r deeprank2
pip install deeprank2
8 changes: 8 additions & 0 deletions .github/workflows/build-repo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,22 @@ jobs:
os: ["ubuntu-latest"]
python-version: ["3.10"] # ["3.10", "3.11"]

# https://github.com/marketplace/actions/setup-miniconda#use-a-default-shell
defaults:
run:
shell: bash -l {0}

steps:
- uses: actions/checkout@v3

- uses: ./.github/actions/install-python-and-package
with:
python-version: ${{ matrix.python-version }}
extras-require: test, publishing
pkg-installation-type: "repository"

- name: Run unit tests
run: pytest -v

- name: Verify that we can build the package
run: python3 -m build
5 changes: 5 additions & 0 deletions .github/workflows/coveralls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ jobs:
os: ["ubuntu-latest"]
python-version: ["3.10"]

# https://github.com/marketplace/actions/setup-miniconda#use-a-default-shell
defaults:
run:
shell: bash -l {0}

steps:
- uses: actions/checkout@v3
- uses: ./.github/actions/install-python-and-package
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ jobs:
os: ["ubuntu-latest"]
python-version: ["3.10"]

# https://github.com/marketplace/actions/setup-miniconda#use-a-default-shell
defaults:
run:
shell: bash -l {0}

steps:
- uses: actions/checkout@v3
- uses: ./.github/actions/install-python-and-package
Expand Down
18 changes: 9 additions & 9 deletions .zenodo.json
Original file line number Diff line number Diff line change
@@ -1,42 +1,42 @@
{
"creators": [
{
"affiliation": "Netherlands eScience Center, Amsterdam, NL",
"affiliation": "Netherlands eScience Center, Amsterdam, The Netherlands",
"name": "Giulia Crocioni",
"orcid": "0000-0002-0823-0121"
},
{
"affiliation": "Netherlands eScience Center, Amsterdam, NL",
"affiliation": "Netherlands eScience Center, Amsterdam, The Netherlands",
"name": "Dani L. Bodor",
"orcid": "0000-0003-2109-2349"
},
{
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL",
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands",
"name": "Coos Baakman",
"orcid": "0000-0003-4317-1566"
},
{
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL",
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands",
"name": "Farzaneh M. Parizi",
"orcid": "0000-0003-4230-7492"
},
{
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL",
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands",
"name": "Daniel T. Rademaker",
"orcid": "0000-0003-1959-1317"
},
{
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL",
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands",
"name": "Gayatri Ramakrishnan",
"orcid": "0000-0001-8203-2783"
},
{
"affiliation": "Netherlands eScience Center, Amsterdam, NL",
"affiliation": "Netherlands eScience Center, Amsterdam, The Netherlands",
"name": "Sven van der Burg",
"orcid": "0000-0003-1250-6968"
},
{
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL",
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands",
"name": "Dario F. Marzella",
"orcid": "0000-0002-0043-3055"
},
Expand All @@ -45,7 +45,7 @@
"orcid": "0000-0002-9113-0622"
},
{
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL",
"affiliation": "Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands",
"name": "Li C. Xue",
"orcid": "0000-0002-2613-538X"
}
Expand Down
22 changes: 11 additions & 11 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -12,44 +12,44 @@ authors:
- given-names: Giulia
family-names: Crocioni
email: [email protected]
affiliation: Netherlands eScience Center, Amsterdam, NL
affiliation: Netherlands eScience Center, Amsterdam, The Netherlands
orcid: "https://orcid.org/0000-0002-0823-0121"
- given-names: Dani L.
family-names: Bodor
email: [email protected]
affiliation: Netherlands eScience Center, Amsterdam, NL
affiliation: Netherlands eScience Center, Amsterdam, The Netherlands
orcid: "https://orcid.org/0000-0003-2109-2349"
- given-names: Coos
family-names: Baakman
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands
orcid: "https://orcid.org/0000-0003-4317-1566"
- given-names: Farzaneh Meimandi
family-names: Parizi
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands
orcid: "https://orcid.org/0000-0003-4230-7492"
- given-names: Daniel-T.
family-names: Rademaker
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands
orcid: "https://orcid.org/0000-0003-1959-1317"
- given-names: Gayatri
family-names: Ramakrishnan
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands
orcid: "https://orcid.org/0000-0001-8203-2783"
- given-names: Sven A.
family-names: van der Burg
affiliation: Netherlands eScience Center, Amsterdam, NL
affiliation: Netherlands eScience Center, Amsterdam, The Netherlands
orcid: "https://orcid.org/0000-0003-1250-6968"
- given-names: Dario F.
family-names: Marzella
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands
orcid: "https://orcid.org/0000-0002-0043-3055"
- given-names: João M.C.
family-names: Teixeira
affiliation: Independent Researcher
orcid: "https://orcid.org/0000-0002-9113-0622"
- given-names: Li C.
family-names: Xue
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, NL
affiliation: Center for Molecular Bioinformatics, Radboud Universitair Medisch Centrum, Nijmegen, Gelderland, The Netherlands
orcid: "https://orcid.org/0000-0002-2613-538X"
abstract: >-
DeepRank2 is an open-source deep learning
Expand All @@ -68,5 +68,5 @@ keywords:
- DeepRank
license: Apache-2.0
commit: 4e8823758ba03f824b4281f5689cb6a335ab2f6c
version: "3.0.0"
date-released: "2024-01-25"
version: "3.0.1"
date-released: "2024-02-22"
4 changes: 2 additions & 2 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ You want to make some kind of change to the code base
#. if needed, fork the repository to your own Github profile and create your own feature branch off of the latest main commit. While working on your feature branch, make sure to stay up to date with the main branch by pulling in changes, possibly from the 'upstream' repository (follow the instructions `here <https://help.github.com/articles/configuring-a-remote-for-a-fork/>`__ and `here <https://help.github.com/articles/syncing-a-fork/>`__);
#. make sure the existing tests still work by running ``python setup.py test``;
#. add your own tests (if necessary);
#. ensure the code is correctly linted (`ruff .`) and formatted (`ruff format .`);
#. see our `developer's readme <README.dev.md>`` for detailed information on our style conventions, etc.;
#. ensure the code is correctly linted (``ruff .``) and formatted (``ruff format .``);
#. see our `developer's readme <README.dev.md>`_ for detailed information on our style conventions, etc.;
#. update or expand the documentation;
#. `push <http://rogerdudler.github.io/git-guide/>`_ your feature branch to (your fork of) the DeepRank2 repository on GitHub;
#. create the pull request, e.g. following the instructions `here <https://help.github.com/articles/creating-a-pull-request/>`__.
Expand Down
30 changes: 15 additions & 15 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,26 @@ FROM --platform=linux/x86_64 condaforge/miniforge3:23.3.1-1

# Add files
ADD ./tutorials /home/deeprank2/tutorials
ADD ./env/environment.yml /home/deeprank2
ADD ./env/requirements.txt /home/deeprank2
ADD ./env/deeprank2-docker.yml /home/deeprank2
ADD ./env/requirements-docker.txt /home/deeprank2

# Install
RUN \
apt update -y &&
apt install unzip -y &&
# Install dependencies and package
apt update -y && \
apt install unzip -y && \
## GCC
apt install -y gcc &&
## Conda and pip deps
mamba env create -f /home/deeprank2/environment.yml &&
## Get the data for running the tutorials
if [ -d "/home/deeprank2/tutorials/data_raw" ]; then rm -Rf /home/deeprank2/tutorials/data_raw; fi &&
if [ -d "/home/deeprank2/tutorials/data_processed" ]; then rm -Rf /home/deeprank2/tutorials/data_processed; fi &&
wget https://zenodo.org/records/8349335/files/data_raw.zip &&
unzip data_raw.zip -d data_raw &&
apt install -y gcc && \
## Create the environment and install the dependencies
mamba env create -f /home/deeprank2/deeprank2-docker.yml && \
## Activate the environment automatically when entering the container
echo "source activate deeprank2" >~/.bashrc && \
# Get the data for running the tutorials
if [ -d "/home/deeprank2/tutorials/data_raw" ]; then rm -Rf /home/deeprank2/tutorials/data_raw; fi && \
if [ -d "/home/deeprank2/tutorials/data_processed" ]; then rm -Rf /home/deeprank2/tutorials/data_processed; fi && \
wget https://zenodo.org/records/8349335/files/data_raw.zip && \
unzip data_raw.zip -d data_raw && \
mv data_raw /home/deeprank2/tutorials

# Activate the environment
RUN echo "source activate deeprank2" >~/.bashrc
ENV PATH /opt/conda/envs/deeprank2/bin:$PATH

# Define working directory
Expand Down
2 changes: 1 addition & 1 deletion README.dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ During the development cycle, three main supporting branches are used:

## Making a release

1. Branch from `dev` and prepare the branch for the release (e.g., removing the unnecessary dev files such as the current one, fix minor bugs if necessary).
1. Branch from `dev` and prepare the branch for the release (e.g., removing the unnecessary dev files, fix minor bugs if necessary).
2. [Bump the version](https://github.com/DeepRank/deeprank2/blob/dev/README.dev.md#versioning).
3. Verify that the information in `CITATION.cff` is correct (update the release date), and that `.zenodo.json` contains equivalent data.
4. Merge the release branch into `main` (and `dev`), and [run the tests](https://github.com/DeepRank/deeprank2/blob/dev/README.dev.md#running-the-tests).
Expand Down
Loading
Loading