Better error message for s3fs-mapped files that are in glacier (C4-747), second try

[netsettler]

This is a copy of already-reviewed PR #23 (which got into an odd state).

This addresses SubmitCGAP vague error from awscli (C4-747) by making a specific error message relating to s3fs. However, to do that, it requires that the environment variables CGAP_S3FS_UPLOAD_DIR and CGAP_S3FS_UPLOAD_BUCKETS are set.

@sbreiff, best would be if we could get the setup script for s3fs into SubmitCGAP as well, so that it can arrange to set those variables. From discussion on the ticket, with env vars adjusted:

#!/bin/bash
# Expects $CGAP_S3FS_UPLOAD_DIR = ~/upload_files
# and $CGAP_S3FS_UPLOAD_BUCKETS set to a string containing one or more bucket names,
# separated by line breaks.

# Install s3fs-fuse for mounting S3 buckets
sudo amazon-linux-extras install epel -y
sudo yum install s3fs-fuse -y

# Mount buckets to $CGAP_S3FS_UPLOAD_DIR directory
mkdir $CGAP_S3FS_UPLOAD_DIR
for BUCKET in $CGAP_S3FS_UPLOAD_BUCKETS
do
    s3fs $BUCKET $CGAP_S3FS_UPLOAD_DIR -o iam_role
done

# Create virtual env for package installation
python3 -m venv ~/cgap_submission
source ~/cgap_submission/bin/activate

# Run SubmitCGAP with mounted files
pip install submit_cgap

The env vars this PR uses are slightly different, so that would have to be adjusted, too. The interaction looks like:

export CGAP_S3FS_UPLOAD_BUCKETS=elasticbeanstalk-fourfront-cgap-wfoutput
export CGAP_S3FS_UPLOAD_DIR=~/upload_files/
bash upload_files.sh
source ~/cgap_submission/bin/activate
resume-uploads 76298911-78b8-4a97-8704-37f5ce391e58 -u $UPLOAD_DIR -s http://fourfront-cgaptest.9wzadzju3p.us-east-1.elasticbeanstalk.com

The interaction looks like:

Upload 1 file? [yes/no]: yes
Uploading /home/ec2-user/upload_files/6502af34-4313-4295-bdff-12991d8fcd46/GAPFI4SUPQO9.cram to item 82e5354e-cc3f-4bf5-a0d7-3948c32df0c2 ...
Going to upload /home/ec2-user/upload_files/6502af34-4313-4295-bdff-12991d8fcd46/GAPFI4SUPQO9.cram to s3://elasticbeanstalk-fourfront-cgaptest-wfoutput/82e5354e-cc3f-4bf5-a0d7-3948c32df0c2/GAPFI1HFPW19.cram.
upload failed: upload_files/6502af34-4313-4295-bdff-12991d8fcd46/GAPFI4SUPQO9.cram to s3://elasticbeanstalk-fourfront-cgaptest-wfoutput/82e5354e-cc3f-4bf5-a0d7-3948c32df0c2/GAPFI1HFPW19.cram [Errno 5] Input/output error
The file /home/ec2-user/upload_files/6502af34-4313-4295-bdff-12991d8fcd46/GAPFI4SUPQO9.cram is mapped via S3FS to DEEP_ARCHIVE storage.
RuntimeError: Upload failed with exit code 1