Skip to content

Commit

Permalink
Adding initial versions of unit test workflows
Browse files Browse the repository at this point in the history
  • Loading branch information
tefirman committed Dec 16, 2024
1 parent da89a20 commit 144e5a2
Show file tree
Hide file tree
Showing 8 changed files with 303 additions and 2 deletions.
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2024 Fred Hutch Cancer Center
Copyright (c) 2024 Fred Hutch Data Science Lab WILDS

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
50 changes: 49 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,50 @@
# wdl-unit-tests
Collection of simple WDL's for use in unit testing of WDL-centric software such as PROOF
A collection of minimal WDL (Workflow Description Language) workflows designed for testing WDL execution environments and backend configurations. Each workflow tests a specific aspect of WDL functionality while maintaining simplicity for easy debugging and verification.

## Workflows

### 1. HelloHostname
Basic WDL workflow that returns the hostname of the execution node. Used for testing basic WDL functionality and execution environment.
- Tests basic command execution
- Tests basic output handling
- No container or module requirements

### 2. HelloDockerHostname
Extension of the basic hostname workflow that runs within a Docker container. Used for testing Docker integration in WDL environments.
- Tests Docker container support
- Tests container runtime environment
- Uses ubuntu:latest container image

### 3. HelloModuleHostname
Version of the hostname workflow that uses environment modules. Used for testing module-based software management systems.
- Tests environment module support
- Tests module loading functionality
- Uses Python/3.7.4-foss-2019b-fh1 module

## Usage
Each workflow can be run using standard WDL execution engines:

```bash
# Using Cromwell
java -jar cromwell.jar run <workflow_name>.wdl

# Using miniwdl
miniwdl run <workflow_name>.wdl
```

## Purpose
This test suite is designed to:
- Validate WDL execution environments
- Test different backend configurations
- Verify software deployment methods (native, container, modules)
- Provide simple debugging tools
- Serve as examples for WDL development

## Contributing
When adding new test workflows, please follow these guidelines:
- Keep workflows minimal and focused on testing specific features
- Include comprehensive parameter metadata
- Document runtime requirements clearly
- Isolate it to its own directory


46 changes: 46 additions & 0 deletions helloDockerHostname/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# HelloDockerHostname WDL Workflow

## Overview
A test workflow that demonstrates Docker container integration in WDL by returning the hostname of the compute node where the job is executed. This workflow extends the basic hostname test by running the command within a Docker container.

## Workflow Components

### Workflow: `HelloDockerHostname`
The main workflow executes a single task within a Docker container and returns its output.

**Outputs:**
- `stdout`: File containing the hostname of the execution node

### Task: `Hostname`
Executes the `hostname` command within an Ubuntu Docker container.

**Runtime Requirements:**
- CPU: 1 core
- Memory: 1 GB
- Docker: `ubuntu:latest`

**Outputs:**
- `out`: File containing the hostname of the execution node

## Usage
```bash
# Execute with cromwell
java -jar cromwell.jar run helloDockerHostname.wdl

# Execute with miniwdl
miniwdl run helloDockerHostname.wdl
```

## Purpose
This workflow serves as a test case for:
- Docker container integration
- Container runtime environment validation
- Basic resource allocation testing
- Output handling validation from containerized tasks
- Backend Docker support verification

## Version
WDL 1.0

## Notes
- Comparing outputs between this and the non-Docker version can help verify proper container execution
40 changes: 40 additions & 0 deletions helloDockerHostname/helloDockerHostname.wdl
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
version 1.0
## This is a test workflow that returns the hostname of the node
## the job is submitted to as a test for Docker functionality on Gizmo.
#### WORKFLOW DEFINITION
workflow HelloDockerHostname {
call Hostname {
}

output {
File stdout = Hostname.out
}

parameter_meta {
stdout: "hostname of the node the job was submitted to"
}
}

#### TASK DEFINITIONS
task Hostname {
command <<<
echo $(hostname)
>>>

output {
File out = stdout()
}

runtime {
cpu: 1
memory: "1 GB"
docker: "ubuntu:latest"
}

parameter_meta {
out: "hostname of the node the job was submitted to"
}
}
41 changes: 41 additions & 0 deletions helloHostname/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# HelloHostname WDL Workflow

## Overview
A simple test workflow that demonstrates basic WDL functionality by returning the hostname of the compute node where the job is executed. This workflow is particularly useful for testing backend configurations and verifying proper job distribution.

## Workflow Components

### Workflow: `HelloHostname`
The main workflow executes a single task and returns its output.

**Outputs:**
- `stdout`: File containing the hostname of the execution node

### Task: `Hostname`
A basic task that executes the `hostname` command and captures its output.

**Runtime Requirements:**
- CPU: 1 core
- Memory: 1 GB

**Outputs:**
- `out`: File containing the hostname of the execution node

## Usage
```bash
# Execute with cromwell
java -jar cromwell.jar run helloHostname.wdl

# Execute with miniwdl
miniwdl run helloHostname.wdl
```

## Purpose
This workflow serves as a basic test case for:
- WDL execution environment validation
- Backend configuration verification
- Basic resource allocation testing
- Output handling validation

## Version
WDL 1.0
39 changes: 39 additions & 0 deletions helloHostname/helloHostname.wdl
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
version 1.0
## This is a test workflow that returns the hostname of the node
## the job is submitted to as a test for the Gizmo backend.
#### WORKFLOW DEFINITION
workflow HelloHostname {
call Hostname {
}

output {
File stdout = Hostname.out
}

parameter_meta {
stdout: "hostname of the node the job was submitted to"
}
}

#### TASK DEFINITIONS
task Hostname {
command <<<
echo $(hostname)
>>>

output {
File out = stdout()
}

runtime {
cpu: 1
memory: "1 GB"
}

parameter_meta {
out: "hostname of the node the job was submitted to"
}
}
47 changes: 47 additions & 0 deletions helloModuleHostname/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# HelloModuleHostname WDL Workflow

## Overview
A test workflow that demonstrates environment module integration in WDL by returning the hostname of the compute node where the job is executed. This workflow extends the basic hostname test by running the command within a specific module environment.

## Workflow Components

### Workflow: `HelloDockerHostname`
The main workflow executes a single task within a module-configured environment and returns its output.

**Outputs:**
- `stdout`: File containing the hostname of the execution node

### Task: `Hostname`
Executes the `hostname` command within an environment with Python modules loaded.

**Runtime Requirements:**
- CPU: 1 core
- Memory: 1 GB
- Modules: Python/3.7.4-foss-2019b-fh1

**Outputs:**
- `out`: File containing the hostname of the execution node

## Usage
```bash
# Execute with cromwell
java -jar cromwell.jar run helloModuleHostname.wdl

# Execute with miniwdl
miniwdl run helloModuleHostname.wdl
```

## Purpose
This workflow serves as a test case for:
- Environment module integration
- Module loading validation
- Basic resource allocation testing
- Output handling validation from module-enabled tasks
- Backend module support verification

## Version
WDL 1.0

## Notes
- Comparing outputs between this and the basic version can help verify proper module loading
- Useful for testing HPC environments that use the module system for software management
40 changes: 40 additions & 0 deletions helloModuleHostname/helloModuleHostname.wdl
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
version 1.0
## This is a test workflow that returns the hostname of the node
## the job is submitted to as a test for module functionality on Gizmo.
#### WORKFLOW DEFINITION
workflow HelloModuleHostname {
call Hostname {
}

output {
File stdout = Hostname.out
}

parameter_meta {
stdout: "hostname of the node the job was submitted to"
}
}

#### TASK DEFINITIONS
task Hostname {
command <<<
echo $(hostname)
>>>

output {
File out = stdout()
}

runtime {
cpu: 1
memory: "1 GB"
modules: "Python/3.7.4-foss-2019b-fh1"
}

parameter_meta {
out: "hostname of the node the job was submitted to"
}
}

0 comments on commit 144e5a2

Please sign in to comment.