Retry on failure?

[dfornika]

I'm working on a galaxy wrapper for this tool. During my testing of the tool wrapper, I've noticed that downloads will sometimes fail. I'd like to have a way to retry the download if it fails.

Would it be more sensible to integrate that directly into the tool, or into the galaxy tool wrapper?

[kblin]

I guess this depends on the failure mode. The problem with HTTP downloads is that the "all's good" response from the server arrives in the header, before the actual data comes in. If anything happens to the connection afterwards, it can be difficult for the client side tool to notice.

So I guess the question is "what kind of download failures are you thinking about"?

Also, are you using any of the validation functions?

[dfornika]

I wasn't using the validation function when I posted this issue, but I've since updated the galaxy wrapper to run with --extended-validation all enabled. I also added a 5-second delay between downloads. I haven't added any mechanism to retry a failed download to the galaxy wrapper.

Those changes seem to have improved the robustness of the galaxy tool (based an a few quick tests running 6 downloads). I was getting ~3-6 successful downloads before making those changes but got 6 successful downloads on my subsequent test. (I know I should put together a larger test).

But to answer your question, here are a few kinds of download failures I can think of:

1. The network is degraded/unreliable and no sequence file data is sent.
2. The network is degraded/unreliable and corrupt or incomplete sequence file data is sent.
3. The user provides an incorrect accession number so no sequence file data is sent.