Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for multipart upload in s3 http engine #1824

Open
alex-katz opened this issue Oct 7, 2024 · 0 comments
Open

Support for multipart upload in s3 http engine #1824

alex-katz opened this issue Oct 7, 2024 · 0 comments

Comments

@alex-katz
Copy link

MPU in S3 http engine
Hello,
I'm looking for support for MPU in s3 http engine implementation. Based on what I can see in the traffic capture, there is currently no such mechanism. Meaning that the entire object is uploaded in a single PUT request.

Amazon suggests that all the objects which size exceeds 100Mb should not be uploaded in a single request. There is a bit more information about the mechanism: https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html.
Not sure however it should be forced for all objects of a specific size but instead better to have it configurable for the specific job.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant