Skip to content

Commit

Permalink
Use cluster-agnostic environment variables in runRscript
Browse files Browse the repository at this point in the history
runRscript.sh was written for use with SGE, using SGE-specific
environment variables. Modify the script to process either SGE or
Slurm environment variables by setting cluster-agnostic variables
at the beginning of the script based on the SGE variables or the
Slurm variables, depending on are set. Then use these cluster-agnostic
variables in the remainder of the script.

Note: it would probably be better to have the Cluster subclasses
in TopmedPipeline.py set the proper variables for the script, if
we refactored the classes following #55.
  • Loading branch information
amstilp committed Mar 22, 2023
1 parent 91ed055 commit 0da5918
Showing 1 changed file with 24 additions and 9 deletions.
33 changes: 24 additions & 9 deletions runRscript.sh
Original file line number Diff line number Diff line change
@@ -1,15 +1,30 @@
#! /bin/bash

# TODO: UPDATE FOR SLURM?
# Set cross-cluster environment variables.


# Determine cluster type and set cluster-agnostic variables appropriate.
if [[ ! -z "${SGE_ROOT}" ]]
then
CLUSTER_JOB_ID=$JOB_ID
CLUSTER_TASK_ID=$SGE_TASK_ID
elif [[ ! -z "${SLURM_CLUSTER_NAME}" ]]
then
export CLUSTER_JOB_ID=$SLURM_JOB_ID
export CLUSTER_TASK_ID=$SLURM_ARRAY_TASK_ID
fi

# set MKL_NUM_THREADS to match number of available cores
export MKL_NUM_THREADS=$NSLOTS
export MKL_NUM_THREADS=$CLUSTER_SLOTS

while getopts ":cs" opt; do
case $opt in
c)
TASK="--chromosome $SGE_TASK_ID"
TASK="--chromosome $CLUSTER_TASK_ID"
;;
s)
TASK="--segment $SGE_TASK_ID"
TASK="--segment $CLUSTER_TASK_ID"
;;
\?)
echo "Invalid option: -$OPTARG" >&2
Expand All @@ -18,25 +33,25 @@ while getopts ":cs" opt; do
done
shift "$((OPTIND-1))"

if [ "$SGE_TASK_ID" == "undefined" ] || [ "$SGE_TASK_ID" == "" ]; then
if [ "$CLUSTER_TASK_ID" == "undefined" ] || [ "$CLUSTER_TASK_ID" == "" ]; then
TASK=""
fi

args=("$@") # all arguments
unset args[0] # remove first argument (R script name)
tb=`date`
echo ">>> Start job: $JOB_ID at $tb ... "
echo ">>> Start job: $CLUSTER_JOB_ID at $tb ... "
R -q --vanilla --args ${args[@]} $TASK < $1


R_exit_code="$?"
r_status=$R_exit_code
if [[ $R_exit_code -ne "0" ]]; then
echo ">>> Error: R status code $R_exit_code"
if [[ "$SGE_TASK_ID" -ne "" ]]; then
touch fail.${JOB_ID}.${SGE_TASK_ID}
if [[ "$CLUSTER_TASK_ID" -ne "" ]]; then
touch fail.${CLUSTER_JOB_ID}.${CLUSTER_TASK_ID}
else
touch fail.${JOB_ID}
touch fail.${CLUSTER_JOB_ID}
fi
if [[ ! -z "${SGE_ROOT+x}" && ! -z "${ENABLE_EQW+x}" ]]; then
r_status=100
Expand All @@ -45,5 +60,5 @@ if [[ $R_exit_code -ne "0" ]]; then
fi

te=`date`
echo ">>> End job: $JOB_ID at $te"
echo ">>> End job: $CLUSTER_JOB_ID at $te"
exit $r_status

0 comments on commit 0da5918

Please sign in to comment.