What happened?

Currently, Query on Batch waits for the full stage of workers to complete before collecting the results even in the event of a failure. Now that Query on Batch uses Job Groups, we can use the job group cancel_after_n_failures functionality to cancel remaining jobs in the stage after a certain number have failed.

Query on Batch should set cancel_after_n_failures = 1 so that the user can see the error of the failed partition without waiting for all the partitions to run (and paying for them).

Version

0.2.131

Relevant log output

No response