diff --git a/LICENSE b/LICENSE index 283d513..4769978 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ MIT License -Copyright (c) 2024 Fred Hutch Cancer Center +Copyright (c) 2024 Fred Hutch Data Science Lab WILDS Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/README.md b/README.md index 4b553ab..bceaf5c 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,50 @@ # wdl-unit-tests -Collection of simple WDL's for use in unit testing of WDL-centric software such as PROOF +A collection of minimal WDL (Workflow Description Language) workflows designed for testing WDL execution environments and backend configurations. Each workflow tests a specific aspect of WDL functionality while maintaining simplicity for easy debugging and verification. + +## Workflows + +### 1. HelloHostname +Basic WDL workflow that returns the hostname of the execution node. Used for testing basic WDL functionality and execution environment. +- Tests basic command execution +- Tests basic output handling +- No container or module requirements + +### 2. HelloDockerHostname +Extension of the basic hostname workflow that runs within a Docker container. Used for testing Docker integration in WDL environments. +- Tests Docker container support +- Tests container runtime environment +- Uses ubuntu:latest container image + +### 3. HelloModuleHostname +Version of the hostname workflow that uses environment modules. Used for testing module-based software management systems. +- Tests environment module support +- Tests module loading functionality +- Uses Python/3.7.4-foss-2019b-fh1 module + +## Usage +Each workflow can be run using standard WDL execution engines: + +```bash +# Using Cromwell +java -jar cromwell.jar run .wdl + +# Using miniwdl +miniwdl run .wdl +``` + +## Purpose +This test suite is designed to: +- Validate WDL execution environments +- Test different backend configurations +- Verify software deployment methods (native, container, modules) +- Provide simple debugging tools +- Serve as examples for WDL development + +## Contributing +When adding new test workflows, please follow these guidelines: +- Keep workflows minimal and focused on testing specific features +- Include comprehensive parameter metadata +- Document runtime requirements clearly +- Isolate it to its own directory + + diff --git a/helloDockerHostname/README.md b/helloDockerHostname/README.md new file mode 100644 index 0000000..3c5ca29 --- /dev/null +++ b/helloDockerHostname/README.md @@ -0,0 +1,46 @@ +# HelloDockerHostname WDL Workflow + +## Overview +A test workflow that demonstrates Docker container integration in WDL by returning the hostname of the compute node where the job is executed. This workflow extends the basic hostname test by running the command within a Docker container. + +## Workflow Components + +### Workflow: `HelloDockerHostname` +The main workflow executes a single task within a Docker container and returns its output. + +**Outputs:** +- `stdout`: File containing the hostname of the execution node + +### Task: `Hostname` +Executes the `hostname` command within an Ubuntu Docker container. + +**Runtime Requirements:** +- CPU: 1 core +- Memory: 1 GB +- Docker: `ubuntu:latest` + +**Outputs:** +- `out`: File containing the hostname of the execution node + +## Usage +```bash +# Execute with cromwell +java -jar cromwell.jar run helloDockerHostname.wdl + +# Execute with miniwdl +miniwdl run helloDockerHostname.wdl +``` + +## Purpose +This workflow serves as a test case for: +- Docker container integration +- Container runtime environment validation +- Basic resource allocation testing +- Output handling validation from containerized tasks +- Backend Docker support verification + +## Version +WDL 1.0 + +## Notes +- Comparing outputs between this and the non-Docker version can help verify proper container execution diff --git a/helloDockerHostname/helloDockerHostname.wdl b/helloDockerHostname/helloDockerHostname.wdl new file mode 100644 index 0000000..8f3ee87 --- /dev/null +++ b/helloDockerHostname/helloDockerHostname.wdl @@ -0,0 +1,40 @@ +version 1.0 +## This is a test workflow that returns the hostname of the node +## the job is submitted to as a test for Docker functionality on Gizmo. + +#### WORKFLOW DEFINITION + +workflow HelloDockerHostname { + call Hostname { + } + + output { + File stdout = Hostname.out + } + + parameter_meta { + stdout: "hostname of the node the job was submitted to" + } +} + +#### TASK DEFINITIONS + +task Hostname { + command <<< + echo $(hostname) + >>> + + output { + File out = stdout() + } + + runtime { + cpu: 1 + memory: "1 GB" + docker: "ubuntu:latest" + } + + parameter_meta { + out: "hostname of the node the job was submitted to" + } +} diff --git a/helloHostname/README.md b/helloHostname/README.md new file mode 100644 index 0000000..1a29f87 --- /dev/null +++ b/helloHostname/README.md @@ -0,0 +1,41 @@ +# HelloHostname WDL Workflow + +## Overview +A simple test workflow that demonstrates basic WDL functionality by returning the hostname of the compute node where the job is executed. This workflow is particularly useful for testing backend configurations and verifying proper job distribution. + +## Workflow Components + +### Workflow: `HelloHostname` +The main workflow executes a single task and returns its output. + +**Outputs:** +- `stdout`: File containing the hostname of the execution node + +### Task: `Hostname` +A basic task that executes the `hostname` command and captures its output. + +**Runtime Requirements:** +- CPU: 1 core +- Memory: 1 GB + +**Outputs:** +- `out`: File containing the hostname of the execution node + +## Usage +```bash +# Execute with cromwell +java -jar cromwell.jar run helloHostname.wdl + +# Execute with miniwdl +miniwdl run helloHostname.wdl +``` + +## Purpose +This workflow serves as a basic test case for: +- WDL execution environment validation +- Backend configuration verification +- Basic resource allocation testing +- Output handling validation + +## Version +WDL 1.0 diff --git a/helloHostname/helloHostname.wdl b/helloHostname/helloHostname.wdl new file mode 100644 index 0000000..b6812a3 --- /dev/null +++ b/helloHostname/helloHostname.wdl @@ -0,0 +1,39 @@ +version 1.0 +## This is a test workflow that returns the hostname of the node +## the job is submitted to as a test for the Gizmo backend. + +#### WORKFLOW DEFINITION + +workflow HelloHostname { + call Hostname { + } + + output { + File stdout = Hostname.out + } + + parameter_meta { + stdout: "hostname of the node the job was submitted to" + } +} + +#### TASK DEFINITIONS + +task Hostname { + command <<< + echo $(hostname) + >>> + + output { + File out = stdout() + } + + runtime { + cpu: 1 + memory: "1 GB" + } + + parameter_meta { + out: "hostname of the node the job was submitted to" + } +} diff --git a/helloModuleHostname/README.md b/helloModuleHostname/README.md new file mode 100644 index 0000000..a747e32 --- /dev/null +++ b/helloModuleHostname/README.md @@ -0,0 +1,47 @@ +# HelloModuleHostname WDL Workflow + +## Overview +A test workflow that demonstrates environment module integration in WDL by returning the hostname of the compute node where the job is executed. This workflow extends the basic hostname test by running the command within a specific module environment. + +## Workflow Components + +### Workflow: `HelloDockerHostname` +The main workflow executes a single task within a module-configured environment and returns its output. + +**Outputs:** +- `stdout`: File containing the hostname of the execution node + +### Task: `Hostname` +Executes the `hostname` command within an environment with Python modules loaded. + +**Runtime Requirements:** +- CPU: 1 core +- Memory: 1 GB +- Modules: Python/3.7.4-foss-2019b-fh1 + +**Outputs:** +- `out`: File containing the hostname of the execution node + +## Usage +```bash +# Execute with cromwell +java -jar cromwell.jar run helloModuleHostname.wdl + +# Execute with miniwdl +miniwdl run helloModuleHostname.wdl +``` + +## Purpose +This workflow serves as a test case for: +- Environment module integration +- Module loading validation +- Basic resource allocation testing +- Output handling validation from module-enabled tasks +- Backend module support verification + +## Version +WDL 1.0 + +## Notes +- Comparing outputs between this and the basic version can help verify proper module loading +- Useful for testing HPC environments that use the module system for software management diff --git a/helloModuleHostname/helloModuleHostname.wdl b/helloModuleHostname/helloModuleHostname.wdl new file mode 100644 index 0000000..a9bcb84 --- /dev/null +++ b/helloModuleHostname/helloModuleHostname.wdl @@ -0,0 +1,40 @@ +version 1.0 +## This is a test workflow that returns the hostname of the node +## the job is submitted to as a test for module functionality on Gizmo. + +#### WORKFLOW DEFINITION + +workflow HelloModuleHostname { + call Hostname { + } + + output { + File stdout = Hostname.out + } + + parameter_meta { + stdout: "hostname of the node the job was submitted to" + } +} + +#### TASK DEFINITIONS + +task Hostname { + command <<< + echo $(hostname) + >>> + + output { + File out = stdout() + } + + runtime { + cpu: 1 + memory: "1 GB" + modules: "Python/3.7.4-foss-2019b-fh1" + } + + parameter_meta { + out: "hostname of the node the job was submitted to" + } +}