Atomic renames and asynchronous destructors

[jamesmcm]

We want the rename operations to be atomic, that is the S3 bucket should never be left in an inconsistent state where we have copied some keys to their new names, but not deleted the original keys.

This a problem in the case that s3rename is suddenly terminated. To alleviate this we try to wrap the Copy in a struct that triggers the corresponding Delete when dropped - so if there is a panic (or an interrupt signal is received) then we would still trigger these deletes for the completed copies. See: https://github.com/jamesmcm/s3rename/blob/master/src/wrapped_copy.rs#L36

However, we don't want these deletions to block the OS thread they run on. In principle this is okay, since the DeleteObjectRequest returns a future, however the problem is that drop() (from Drop) is not async, and so we cannot .await inside it.

This means the destructors have to be synchronous and we would be required to block the threads, greatly reducing the throughput of key renames.

To bypass this, we instead tokio::spawn() a new task from inside the destructor, and then await those tasks back in our async main function. See: https://github.com/jamesmcm/s3rename/blob/master/src/main.rs#L122

However, this doesn't resolve the original issue of atomicity, because whilst the tasks will be spawned when the destructors are called, if s3rename receives an interrupt, those tasks will not have time to complete (and will not be awaited).

Unfortunately, Asynchronous Destructors are not yet supported in Rust. But perhaps there is a better work-around for now.

Also note that if we didn't care about atomicity, we could make a much smaller number of batched DeleteObjectsRequests which deletes many keys at a time.