pytest-wdl

This package is a plugin for the pytest unit testing framework that enables testing of workflows written in Workflow Description Language.

Dependencies

• Python 3.6+
• Java 1.8+
• Cromwell JAR file
• Docker daemon (if your WDL tasks depend on Docker images)

Other python dependencies are installed when you install the library.

Installation

Install from PyPI

$ pip install pytest-wdl

Install from source

You can to clone the repository and install:

$ make install

Or use pip to install from github:

$ pip install git+https://github.com/elilillyco/pytest-wdl.git

Install optional dependencies

Some optional features of pytest-wdl have additional dependencies that are loaded on-demand. For example, to enable comparison of expected and actual BAM file outputs of a workflow, the pysam library is required.

The following data types require an "extras" installation:

• bam

To install the dependencies for a data type that has extra dependencies:

$ pip install pytest-wdl[<data_type>]

To do this locally, you can clone the repo and run:

$ pip install -e .[<data_type>]

To install pytest-wdl and all extras dependencies:

$ pip install pytest-wdl[all]

Usage

The pytest-wdl plugin provides a set of fixtures for use with pytest. Here is a quick example:

# test_variant_caller.py
def test_variant_caller(workflow_data, workflow_runner):
    inputs = workflow_data.get_dict("bam", "bai")
    inputs["index"] = {
        "fasta": workflow_data["index_fa"],
        "organism": "human"
    }
    expected = workflow_data.get_dict("vcf")
    workflow_runner(
        "variant_caller/variant_caller.wdl",
        "call_variants",
        inputs,
        expected
    )

This test will execute a workflow (such as the following one) with the specified inputs, and will compare the outputs to the specified expected outputs.

# variant_caller.wdl
version 1.0

struct Index {
  File fasta
  String organism
}

workflow call_variants {
  input {
    File bam
    File bai
    Index index
  }
  ...
  output {
    File vcf = variant_caller.vcf
  }
}

Input and output data are defined in a test_data.json file in the same directory as your test script:

{
  "bam": {
    "url": "http://example.com/my.bam"
  },
  "bai": {
    "url": "http://example.com/my.bam.bai"
  },
  "index_fa": {
    "name": "chr22.fasta"
  },
  "vcf": {
    "url": "http://example.com/expected.vcf.gz",
    "type": "vcf",
    "allowed_diff_lines": 2
  }
}

For details, read the docs.

Development

To develop pytest-wdl, clone the repository and install all the dependencies:

$ git clone https://github.com/EliLillyCo/pytest-wdl.git
$ pip install -r requirements.txt

To run the full build and unit tests, run:

$ make