Add a resolver for source images stored in S3.

[ratliff]

I written this resolver for images stored in S3. It mimics the functionality of the SimpleHTTPResolver. Requires installation of Boto3 SDK. I bet others would find this useful.

Let me know if you have questions.

--- Mark

[jpstroop]

@ckassel I wonder if this is useful for the work you presented at DLF? Not sure if you're the right person to ask, but maybe you could pass it on?

[ckassel]

Thanks, will do!

On Mon, Nov 21, 2016 at 9:54 AM, Jon Stroop [email protected]
wrote:

@ckassel https://github.com/ckassel I wonder if this is useful for the
work you presented at DLF? Not sure if you're the right person to ask, but
maybe you could pass it on?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#257 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AC7u90kkPNibJFr5ASVYwz1WX9mmsRF3ks5rAbCsgaJpZM4K3rXG
.

Carol Kassel
Senior Manager, Digital Library Infrastructure
NYU Digital Library Technology Services
[email protected]
(212) 992-9246
dlib.nyu.edu

[jeremyf]

@ratliff Do you have metrics regarding serving files from "local" storage vs. s3 storage?

We are interested in exploring this, but concerned about s3 performance.

[ratliff]

@jeremyf I don't have any numbers, but I did have our Art Museum run load tests against the configuration and they were (and still are) happy with the response times.

[jeremyf]

@ratliff I notice there is a caching layer that you incorporated. What is your caching layer for the S3 objects? Do you store them on the app server?

[ratliff]

Yes. I cache images in Elastic File System (basically an NFS server). I have a CloudFormation template that you can use to launch the entire stack. Let me know if you want me to e-mail it to you. (Not sure that I ever checked it into a repo.)

[jeremyf]

@ratliff I would love the CloudFormation template. My email is [email protected]

[ratliff]

@jeremyf The repo containing the Cloud Formation template is here https://github.com/loris-imageserver/loris-cloudformation

[jeremyf]

@ratliff Thank you! I think I have what I need for now. Trying to get a little movement on IIIF locally.

[alexwlchan]

@ratliff We might have use for an S3 resolver at Wellcome soon, so we’d be interested in getting this merged.

This needs some tests and cleanup before it’s ready to go – are you interested in finishing this off, or would you be happy for me to tidy it up and merge it?

[flannon]

@alexchan we would be very happy if you were to tidy this up and merge it! At NYU we've been interested in having an S3 resolver, but I haven't had time to work on it yet.

[ratliff]

@alexwlchan I'm no longer actively involved. I handed what I had developed over to folks in the Princeton Univ Art Museum. So, I (and probably they) would be very happy if you would do what needs to be done to merge the code. Let me know if you need any info from me in order to accomplish that.

[alexwlchan]

@ratliff Thanks for the fast reply! I’ll see how I get on, and ask if I run into any issues.

[nmaekawa]

@alexwlchan s3 support is something harvardx is interested in too.