Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add v0.3.0 LXPLUS9 deploy requirements.txt and lockfile #121

Merged
merged 4 commits into from
Nov 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ recast run examples/rome --backend reana

[ATLAS Exotics Workshop 2018]: https://indico.cern.ch/event/710748/contributions/2982534/subcontributions/254796

#### On [LXPLUS8](https://clouddocs.web.cern.ch/clients/lxplus.html)
#### On [LXPLUS9](https://clouddocs.web.cern.ch/clients/lxplus.html)

```console
ssh lxplus8.cern.ch
ssh lxplus9.cern.ch
source ~recast/public/setup.sh
recast catalogue ls
recast run examples/rome
Expand Down
71 changes: 71 additions & 0 deletions deploy/deploy_lxplus9.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/bin/bash

# Example usage:
# $ bash deploy_lxplus9.sh 0.3.0
# or to get the latest recast-atlas release by default:
# $ bash deploy_lxplus9.sh

# Ensure that pip can't install outside a venv
export PIP_REQUIRE_VIRTUALENV=true

RECAST_ATLAS_VERSION="${1:-0.0.0}"

if [[ "${RECAST_ATLAS_VERSION}" == "0.0.0" ]]; then
# `pip index versions` will output the following warning to stderr:
# WARNING: pip index is currently an experimental command. It may be removed/changed in a future release without prior warning.
# so direct stderr to /dev/null to avoid this.
# The next line will be:
# recast-atlas (x.y.z)
# so filter out the version number and output that
RECAST_ATLAS_VERSION=$(python3 -m pip index versions recast-atlas 2> /dev/null | awk -F"[()]" 'NR==1 {print $2}')
fi

# Setup Python virtual environment and install recast-atlas and requirements
python3 -m venv ~recast/public/recast-atlas-"${RECAST_ATLAS_VERSION}"
. ~recast/public/recast-atlas-"${RECAST_ATLAS_VERSION}"/bin/activate

if [[ "${RECAST_ATLAS_VERSION}" == "0.1.8" ]]; then
# c.f. https://github.com/reanahub/reana-client/issues/558
python -m pip install --upgrade pip 'setuptools<58.0.0' wheel
else
python -m pip install --upgrade pip setuptools wheel
fi

# Install the environment from a "lockfile" (just called a requirements.txt by
# pip-tools) generated with pip-tools compile. If the lockfile is missing, make
# compile one with pip-tools.
if [ ! -f ~recast/deploy/recast-atlas-"${RECAST_ATLAS_VERSION}".lock ]; then
python -m pip install pip-tools
python -m piptools compile \
--generate-hashes \
--output-file ~recast/deploy/recast-atlas-"${RECAST_ATLAS_VERSION}".lock \
~recast/deploy/recast-atlas-"${RECAST_ATLAS_VERSION}"-requirements.txt
fi
python -m pip install \
--no-deps \
--require-hashes \
--requirement ~recast/deploy/recast-atlas-"${RECAST_ATLAS_VERSION}".lock

# Use heredoc syntax to generate public environment setup script
cat << EOF > ~recast/public/setup_"${RECAST_ATLAS_VERSION}".sh
#!/bin/bash

export RECAST_DEFAULT_RUN_BACKEND=local
export RECAST_DEFAULT_BUILD_BACKEND=kubernetes
export PACKTIVITY_CONTAINER_RUNTIME=singularity
export SINGULARITY_CACHEDIR="/tmp/\$(whoami)/singularity"
mkdir -p "\${SINGULARITY_CACHEDIR}"

# https://twitter.com/lukasheinrich_/status/1021398718996713475
export LC_ALL=en_US.utf-8
export LANG=en_US.utf-8

. ~recast/public/recast-atlas-${RECAST_ATLAS_VERSION}/bin/activate

\$(recast catalogue add /eos/project/r/recast/atlas/catalogue)
export KUBECONFIG=/eos/project/r/recast/atlas/cluster/clusterconfig
export PATH="\${PATH}:~recast/public/bin"
EOF

# Link public setup script to this version
ln --symbolic --force ~recast/public/setup_"${RECAST_ATLAS_VERSION}".sh ~recast/public/setup.sh
13 changes: 13 additions & 0 deletions deploy/recast-atlas-0.3.0-requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
recast-atlas==0.3.0
yadage-schemas==0.10.8
# local
adage==0.10.3
packtivity==0.16.2
yadage==0.20.1
# local viz
pydotplus==2.0.2
# kubernetes
kubernetes==9.0.0
# reana
reana-client==0.9.1
reana-commons==0.9.3
Loading