Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for versioned content in VERSIONED S3 bucket #553

Open
yarikoptic opened this issue Dec 9, 2022 · 0 comments
Open

Add support for versioned content in VERSIONED S3 bucket #553

yarikoptic opened this issue Dec 9, 2022 · 0 comments

Comments

@yarikoptic
Copy link

I believe interfacing data from S3 was added to LORIS recently. Some buckets could use versioning support to provide efficient way to host multiple version of the datasets. E.g it is done so by OpenNeuro.

If LORIS could reference specific version ID for specific file -- that would open better integration of LORIS with S3 for such projects.
E.g. have a look at S3 bucket for openneuro which has multiple versions for different files, e.g.

❯ datalad ls -aL s3://openneuro.org/ds004288/README
Connecting to bucket: openneuro.org
[INFO   ] S3 session: Connecting to the bucket openneuro.org anonymously 
Bucket info:
  Versioning: S3ResponseError: 403 Forbidden
     Website: S3ResponseError: 403 Forbidden
         ACL: <Policy: openneurocommon (owner) = FULL_CONTROL, http://acs.amazonaws.com/groups/global/AllUsers = READ, rblair2 = FULL_CONTROL, http://acs.amazonaws.com/groups/global/AllUsers = READ_ACP>
ds004288/README 2022-12-08T16:32:24.000Z 1085 ver:2j0mc8EgCLUExJqdiXICySNR_EU.5dqG  acl:AccessDenied  http://openneuro.org.s3.amazonaws.com/ds004288/README?versionId=2j0mc8EgCLUExJqdiXICySNR_EU.5dqG [OK]
ds004288/README 2022-12-08T16:32:24.000Z DeleteMarker
ds004288/README 2022-12-08T16:32:21.000Z  835 ver:_7xU3APkKeX2XWZK_7CtAj6icCSQSa0a  acl:AccessDenied  http://openneuro.org.s3.amazonaws.com/ds004288/README?versionId=_7xU3APkKeX2XWZK_7CtAj6icCSQSa0a [OK]
ds004288/README 2022-12-08T16:32:20.000Z DeleteMarker
ds004288/README 2022-12-08T16:32:19.000Z  512 ver:vmWiK82MmFwbSvusTC4QAD523Ex8TOma  acl:AccessDenied  http://openneuro.org.s3.amazonaws.com/ds004288/README?versionId=vmWiK82MmFwbSvusTC4QAD523Ex8TOma [OK]
ds004288/README 2022-12-08T16:32:18.000Z DeleteMarker
...

attn @breen0074

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant