Challenges Making Authenticated Requests to S3 without Pre-signed URLs

[exai-sukh]

Description:

I am attempting to implement authenticated requests to Amazon S3 without using pre-signed URLs to minimize security risks. However, I have encountered challenges with the available options.

Options Explored:

1. OAuth:

   • Reference: OAuth Support in igv.js
   • I'm uncertain about direct compatibility of OAuth with S3, and was unable to find information supporting compatibility in AWS documentation. If anyone has successfully used OAuth with S3, I would greatly appreciate any examples or insights.

2. Headers for Tracks:

   • Reference: Tracks 2.0 - Options for All Track Types
   • While using headers for single S3 URLs in tracks are functional, challenges arise when dealing with tracks containing multiple files, such as annotations with "URL" and "indexURL". Since the S3 Authentication header relies on the object's key (AWS Documentation), sharing the header for requests to both objects leads to a "SignatureDoesNotMatch" error.

Environment:

• igv.js Version: 2.15.11

[jrobinso]

We could address (2) if that would help. Others have used functional URLS (functions in place of strings in the URL fields) for this problem.

If you are not using signed URLs what are you using for authenticated access in Javascript?

[exai-sukh]

Hi @jrobinso, sorry for my late response. Could you provide more detail or an example of working around this through functional URLs? I don't understand the parameters it can accept and the return type.

For accessing S3 data in my application for other purposes, I rely on the AWS SDK, where my application obtains credentials through an AWS IAM role.

[jrobinso]

I can't design this for you, but here is an approach that should work. A function or promise can be used for any igv.js property that takes a url. The function should return a url, or a promise for a url. For example, you could specify a track as follows

{
   type: annotation,
   format: bed
   url:  getSignedURL("s3://path-to-my-bucket/test.bed")
}

where getSignedURL is a function you implement that exchanges the s3 url for a signed URL obtained using the user credentials. The function can return a promise (i.e. be async).

See "functional-url.html" in the examples folder.