Skip to content

Commit

Permalink
Move general instructions to WIKI
Browse files Browse the repository at this point in the history
  • Loading branch information
verku committed Jun 19, 2024
1 parent a803dc4 commit f842973
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 109 deletions.
58 changes: 2 additions & 56 deletions config/slurm/README.md
Original file line number Diff line number Diff line change
@@ -1,60 +1,6 @@
# GenErode execution on SLURM clusters
# GenErode execution on Dardel (PDC/KTH)

With the switch to Snakemake version 8, GenErode can be run
the following on SLURM clusters:

1) Create the GenErode conda environment or update an earlier
version. The latest conda environment contains the Snakemake
executor plugin for slurm:

```
conda create -f environment.yaml -n generode
```

2) Copy one of the example configuration files `config/slurm/profile/config_plugin_rackham.yaml`
or `config/slurm/profile/config_plugin_dardel.yaml` to
`slurm/config.yaml`. This file specifies compute resources
for each rule or group jobs. Any rule or group job that is
not listed under `set-threads` or `set-resources` uses
default resources specified under `default-resources`. If
any rule or group jobs fail due to too little memory or run
time, their compute resources can be updated in this file.

> Note that the current configuration files were adjusted to the
HPC clusters Rackham from UPPMAX and Dardel from PDC/KTH. Details
on how to configure and run GenErode on Dardel are provided below.
Memory requirements are specified three times in these configuration
files: 1) under `set-threads` (used by Snakemake to specify threads
in rules), 2) under `set-resources` and therein under `mem_mb`,
specifying the memory in Megabytes (multiplying the number of threads
with the available memory per thread), and 3) under `set-resources`
and therein under `cpus-per-task` (the same number as specified under
`set-threads`, required for correct memory assignment on Dardel). The
configuration file for Snakemake version 7 was kept for comparison,
which was also written for Rackham/UPPMAX.

3) Start GenErode the following:

- Open a tmux or screen session
- Activate the GenErode conda environment
- Start the dry run:

```
snakemake --profile slurm -np &> YYMMDD_dry.out
```

- Start the main run:

```
snakemake --profile slurm &> YYMMDD_main.out
```

> Useful flags for running the pipeline: `--ri` to re-run
incomplete jobs and `-k` to keep going in case a job fails.

## Specific instructions for Dardel

1) Load the following modules on Dardel:
1) Load the following modules:

```
module load PDC UPPMAX bioinfo-tools conda singularity tmux
Expand Down
55 changes: 2 additions & 53 deletions utilities/mutational_load_snpeff/slurm/profile/README.md
Original file line number Diff line number Diff line change
@@ -1,57 +1,6 @@
# GenErode execution on SLURM clusters
# GenErode execution on Dardel

With the switch to Snakemake version 8, GenErode can be run
the following on SLURM clusters:

1) Create the GenErode conda environment or update an earlier
version. The latest conda environment contains the Snakemake
executor plugin for slurm:

```
conda create -f environment.yaml -n generode
```

2) Copy the example configuration file `slurm/profile/config_plugin_dardel.yaml`
to `slurm/config.yaml`. This file specifies compute resources
for each rule or group jobs. Any rule or group job that is
not listed under `set-threads` or `set-resources` uses
default resources specified under `default-resources`. If
any rule or group jobs fail due to too little memory or run
time, their compute resources can be updated in this file.

> Note that the current configuration file was adjusted to the
HPC cluster Dardel from PDC/KTH. Details on how to configure and
run GenErode on Dardel are provided below. Memory requirements are
specified three times in the configuration file: 1) under
`set-threads` (used by Snakemake to specify threads in rules), 2)
under `set-resources` and therein under `mem_mb`, specifying the
memory in Megabytes (multiplying the number of threads with the
available memory per thread), and 3) under `set-resources` and
therein under `cpus-per-task` (the same number as specified under
`set-threads`, required for correct memory assignment on Dardel).

3) Start GenErode the following:

- Open a tmux or screen session
- Activate the GenErode conda environment
- Start the dry run:

```
snakemake --profile slurm -n &> YYMMDD_dry.out
```

- Start the main run:

```
snakemake --profile slurm &> YYMMDD_main.out
```

> Useful flags for running the pipeline: `--ri` to re-run
incomplete jobs and `-k` to keep going in case a job fails.

## Specific instructions for Dardel

1) Load the following modules on Dardel:
1) Load the following modules:

```
module load PDC UPPMAX bioinfo-tools conda singularity tmux
Expand Down

0 comments on commit f842973

Please sign in to comment.