-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Adding initial versions of unit test workflows
- Loading branch information
Showing
8 changed files
with
303 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,50 @@ | ||
# wdl-unit-tests | ||
Collection of simple WDL's for use in unit testing of WDL-centric software such as PROOF | ||
A collection of minimal WDL (Workflow Description Language) workflows designed for testing WDL execution environments and backend configurations. Each workflow tests a specific aspect of WDL functionality while maintaining simplicity for easy debugging and verification. | ||
|
||
## Workflows | ||
|
||
### 1. HelloHostname | ||
Basic WDL workflow that returns the hostname of the execution node. Used for testing basic WDL functionality and execution environment. | ||
- Tests basic command execution | ||
- Tests basic output handling | ||
- No container or module requirements | ||
|
||
### 2. HelloDockerHostname | ||
Extension of the basic hostname workflow that runs within a Docker container. Used for testing Docker integration in WDL environments. | ||
- Tests Docker container support | ||
- Tests container runtime environment | ||
- Uses ubuntu:latest container image | ||
|
||
### 3. HelloModuleHostname | ||
Version of the hostname workflow that uses environment modules. Used for testing module-based software management systems. | ||
- Tests environment module support | ||
- Tests module loading functionality | ||
- Uses Python/3.7.4-foss-2019b-fh1 module | ||
|
||
## Usage | ||
Each workflow can be run using standard WDL execution engines: | ||
|
||
```bash | ||
# Using Cromwell | ||
java -jar cromwell.jar run <workflow_name>.wdl | ||
|
||
# Using miniwdl | ||
miniwdl run <workflow_name>.wdl | ||
``` | ||
|
||
## Purpose | ||
This test suite is designed to: | ||
- Validate WDL execution environments | ||
- Test different backend configurations | ||
- Verify software deployment methods (native, container, modules) | ||
- Provide simple debugging tools | ||
- Serve as examples for WDL development | ||
|
||
## Contributing | ||
When adding new test workflows, please follow these guidelines: | ||
- Keep workflows minimal and focused on testing specific features | ||
- Include comprehensive parameter metadata | ||
- Document runtime requirements clearly | ||
- Isolate it to its own directory | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
# HelloDockerHostname WDL Workflow | ||
|
||
## Overview | ||
A test workflow that demonstrates Docker container integration in WDL by returning the hostname of the compute node where the job is executed. This workflow extends the basic hostname test by running the command within a Docker container. | ||
|
||
## Workflow Components | ||
|
||
### Workflow: `HelloDockerHostname` | ||
The main workflow executes a single task within a Docker container and returns its output. | ||
|
||
**Outputs:** | ||
- `stdout`: File containing the hostname of the execution node | ||
|
||
### Task: `Hostname` | ||
Executes the `hostname` command within an Ubuntu Docker container. | ||
|
||
**Runtime Requirements:** | ||
- CPU: 1 core | ||
- Memory: 1 GB | ||
- Docker: `ubuntu:latest` | ||
|
||
**Outputs:** | ||
- `out`: File containing the hostname of the execution node | ||
|
||
## Usage | ||
```bash | ||
# Execute with cromwell | ||
java -jar cromwell.jar run helloDockerHostname.wdl | ||
|
||
# Execute with miniwdl | ||
miniwdl run helloDockerHostname.wdl | ||
``` | ||
|
||
## Purpose | ||
This workflow serves as a test case for: | ||
- Docker container integration | ||
- Container runtime environment validation | ||
- Basic resource allocation testing | ||
- Output handling validation from containerized tasks | ||
- Backend Docker support verification | ||
|
||
## Version | ||
WDL 1.0 | ||
|
||
## Notes | ||
- Comparing outputs between this and the non-Docker version can help verify proper container execution |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
version 1.0 | ||
## This is a test workflow that returns the hostname of the node | ||
## the job is submitted to as a test for Docker functionality on Gizmo. | ||
#### WORKFLOW DEFINITION | ||
workflow HelloDockerHostname { | ||
call Hostname { | ||
} | ||
|
||
output { | ||
File stdout = Hostname.out | ||
} | ||
|
||
parameter_meta { | ||
stdout: "hostname of the node the job was submitted to" | ||
} | ||
} | ||
|
||
#### TASK DEFINITIONS | ||
task Hostname { | ||
command <<< | ||
echo $(hostname) | ||
>>> | ||
|
||
output { | ||
File out = stdout() | ||
} | ||
|
||
runtime { | ||
cpu: 1 | ||
memory: "1 GB" | ||
docker: "ubuntu:latest" | ||
} | ||
|
||
parameter_meta { | ||
out: "hostname of the node the job was submitted to" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
# HelloHostname WDL Workflow | ||
|
||
## Overview | ||
A simple test workflow that demonstrates basic WDL functionality by returning the hostname of the compute node where the job is executed. This workflow is particularly useful for testing backend configurations and verifying proper job distribution. | ||
|
||
## Workflow Components | ||
|
||
### Workflow: `HelloHostname` | ||
The main workflow executes a single task and returns its output. | ||
|
||
**Outputs:** | ||
- `stdout`: File containing the hostname of the execution node | ||
|
||
### Task: `Hostname` | ||
A basic task that executes the `hostname` command and captures its output. | ||
|
||
**Runtime Requirements:** | ||
- CPU: 1 core | ||
- Memory: 1 GB | ||
|
||
**Outputs:** | ||
- `out`: File containing the hostname of the execution node | ||
|
||
## Usage | ||
```bash | ||
# Execute with cromwell | ||
java -jar cromwell.jar run helloHostname.wdl | ||
|
||
# Execute with miniwdl | ||
miniwdl run helloHostname.wdl | ||
``` | ||
|
||
## Purpose | ||
This workflow serves as a basic test case for: | ||
- WDL execution environment validation | ||
- Backend configuration verification | ||
- Basic resource allocation testing | ||
- Output handling validation | ||
|
||
## Version | ||
WDL 1.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
version 1.0 | ||
## This is a test workflow that returns the hostname of the node | ||
## the job is submitted to as a test for the Gizmo backend. | ||
#### WORKFLOW DEFINITION | ||
workflow HelloHostname { | ||
call Hostname { | ||
} | ||
|
||
output { | ||
File stdout = Hostname.out | ||
} | ||
|
||
parameter_meta { | ||
stdout: "hostname of the node the job was submitted to" | ||
} | ||
} | ||
|
||
#### TASK DEFINITIONS | ||
task Hostname { | ||
command <<< | ||
echo $(hostname) | ||
>>> | ||
|
||
output { | ||
File out = stdout() | ||
} | ||
|
||
runtime { | ||
cpu: 1 | ||
memory: "1 GB" | ||
} | ||
|
||
parameter_meta { | ||
out: "hostname of the node the job was submitted to" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
# HelloModuleHostname WDL Workflow | ||
|
||
## Overview | ||
A test workflow that demonstrates environment module integration in WDL by returning the hostname of the compute node where the job is executed. This workflow extends the basic hostname test by running the command within a specific module environment. | ||
|
||
## Workflow Components | ||
|
||
### Workflow: `HelloDockerHostname` | ||
The main workflow executes a single task within a module-configured environment and returns its output. | ||
|
||
**Outputs:** | ||
- `stdout`: File containing the hostname of the execution node | ||
|
||
### Task: `Hostname` | ||
Executes the `hostname` command within an environment with Python modules loaded. | ||
|
||
**Runtime Requirements:** | ||
- CPU: 1 core | ||
- Memory: 1 GB | ||
- Modules: Python/3.7.4-foss-2019b-fh1 | ||
|
||
**Outputs:** | ||
- `out`: File containing the hostname of the execution node | ||
|
||
## Usage | ||
```bash | ||
# Execute with cromwell | ||
java -jar cromwell.jar run helloModuleHostname.wdl | ||
|
||
# Execute with miniwdl | ||
miniwdl run helloModuleHostname.wdl | ||
``` | ||
|
||
## Purpose | ||
This workflow serves as a test case for: | ||
- Environment module integration | ||
- Module loading validation | ||
- Basic resource allocation testing | ||
- Output handling validation from module-enabled tasks | ||
- Backend module support verification | ||
|
||
## Version | ||
WDL 1.0 | ||
|
||
## Notes | ||
- Comparing outputs between this and the basic version can help verify proper module loading | ||
- Useful for testing HPC environments that use the module system for software management |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
version 1.0 | ||
## This is a test workflow that returns the hostname of the node | ||
## the job is submitted to as a test for module functionality on Gizmo. | ||
#### WORKFLOW DEFINITION | ||
workflow HelloModuleHostname { | ||
call Hostname { | ||
} | ||
|
||
output { | ||
File stdout = Hostname.out | ||
} | ||
|
||
parameter_meta { | ||
stdout: "hostname of the node the job was submitted to" | ||
} | ||
} | ||
|
||
#### TASK DEFINITIONS | ||
task Hostname { | ||
command <<< | ||
echo $(hostname) | ||
>>> | ||
|
||
output { | ||
File out = stdout() | ||
} | ||
|
||
runtime { | ||
cpu: 1 | ||
memory: "1 GB" | ||
modules: "Python/3.7.4-foss-2019b-fh1" | ||
} | ||
|
||
parameter_meta { | ||
out: "hostname of the node the job was submitted to" | ||
} | ||
} |