v0.3.6

• Added parameters

  • --java-heap-server: Java heap size java -Xmx for caper server subcommand.
  • --java-heap-run: Java heap size java -Xmx for caper run subcommand.

• Bug fixes

  • --docker not correctly caught when it's already defined in WDL
  • added warning when --docker is not defined for cloud backends

v0.3.5

• Bug fix for docker
  • Cromwell 42 defaults to use delegated flag for docker in mounting volumes but this flag was introduced in docker ce 17.04. Caper will not use this flag for backward compatibility (for docker ce < 17.04).

v0.3.4

• final testing on AWS (since isolation issue for aliased tasks is fixed in cromwell-42)
  • chip-seq-pipeline worked fine
• documentation for setting up cromwell server on AWS
• fixed bug in file_exists() for AWS
  • aws s3 ls does not show exactly matched files. it also shows partially matched files.

v0.3.3

• bug fix for locked copy (deadlock for URL to cloud URIs transfer)
• suppress gsutil/aws warnings (no URLs matched one of the ...)

v0.3.2

• added user to caper list format
• users can filter workflows by submission datetime --hide-result-before ( or hide-result-before in your conf at ~/.caper/default.conf)
  • you can use the same (or shorter in terms of string length) datatime format shown in caper list.
    • this is just about simple datetime string sorting

For example,

(base) leepc12@kadru:/users/leepc12/code/test_wdl$ caper list
id      status  name    str_label       user    submission
648d61d1-a6df-4d1d-8050-2b7fab0b1d7c    On Hold None    T5      leepc12 2019-06-13T11:12:25.661Z
afd8b507-928b-4ae9-9504-a584ce84442b    On Hold None    T4      leepc12 2019-06-13T11:12:21.917Z
05fd8f2b-da72-46c8-8e88-848bc4f0f5ea    On Hold None    T3      leepc12 2019-06-13T11:12:16.349Z
aa08ca19-0ac8-4ec6-995d-53bcfaaa1038    On Hold None    T2      leepc12 2019-06-13T11:12:10.611Z
a6573ee9-5e08-432e-bf7f-5733069b7510    On Hold None    T1      leepc12 2019-06-13T11:11:51.057Z
(base) leepc12@kadru:/users/leepc12/code/test_wdl$ caper list --hide-result-before 2019-06-13T11:12:16
id      status  name    str_label       user    submission
648d61d1-a6df-4d1d-8050-2b7fab0b1d7c    On Hold None    T5      leepc12 2019-06-13T11:12:25.661Z
afd8b507-928b-4ae9-9504-a584ce84442b    On Hold None    T4      leepc12 2019-06-13T11:12:21.917Z
05fd8f2b-da72-46c8-8e88-848bc4f0f5ea    On Hold None    T3      leepc12 2019-06-13T11:12:16.349Z
(base) leepc12@kadru:/users/leepc12/code/test_wdl$ caper list --hide-result-before 2019-06-13
id      status  name    str_label       user    submission
648d61d1-a6df-4d1d-8050-2b7fab0b1d7c    On Hold None    T5      leepc12 2019-06-13T11:12:25.661Z
afd8b507-928b-4ae9-9504-a584ce84442b    On Hold None    T4      leepc12 2019-06-13T11:12:21.917Z
05fd8f2b-da72-46c8-8e88-848bc4f0f5ea    On Hold None    T3      leepc12 2019-06-13T11:12:16.349Z
aa08ca19-0ac8-4ec6-995d-53bcfaaa1038    On Hold None    T2      leepc12 2019-06-13T11:12:10.611Z
a6573ee9-5e08-432e-bf7f-5733069b7510    On Hold None    T1      leepc12 2019-06-13T11:11:51.057Z
(base) leepc12@kadru:/users/leepc12/code/test_wdl$ caper list --hide-result-before 2019-06-14
id      status  name    str_label       user    submission

v0.3.1

• Fix for SLURM Stanford Sherlock job failing issue even though Job is actually done successfully. squeue doesn't seem to work correctly.
  • Job check-alive polling (squeue -j {job_id}) somtimes fail when server is busy
    • Try squeue for every 20 second up to 3 times.
    • increase interval for polling 180 sec -> 360 sec

v0.3.0

• thread safe inter-storage file transfer
  • added a simple spin lock (with .lock file) to obtain thread safety for file transfer
    • multiple users can share the same tmp-dir, tmp-s3-bucket and tmp-gs-bucket.
  • limitation
    • possible race condition
    • skip copying only if file size of source and target files are same (does not check md5sum)
    • source files are not locked to prevent deadlock
      • make sure that source files are almost like read-only

v0.2.8

• Added --max-retries (default:1) to retry failing tasks
• Replaced deprecated actor factory for google backend
• Cromwell server submits jobs slowly (1 per 2 second) due to possible socket error for SLURM/other cluster engines.
• Bug fix for aws backend (typo in code: out_aws_bucket->out_s3_bucket)
• Upgrade cromwell 40 to 42 (to fix aliased task isolation issue on aws)

v0.2.7

• Bug fix for singularity in slurm backend (task hanging issue)

v0.2.5

• Show more information (even successfully completed tasks if --show-completed-task is given) for caper troubleshoot
  • JOB_ID: PID or JOB_ID (SLURM or SGE) of JOB
  • RUN_START: start datetime if task is run
  • RUN_END: end datetime if task is run