Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDK computes the checksum even if one is passed by application #6744

Closed
3 of 4 tasks
trivikr opened this issue Dec 17, 2024 · 1 comment · Fixed by #6745
Closed
3 of 4 tasks

SDK computes the checksum even if one is passed by application #6744

trivikr opened this issue Dec 17, 2024 · 1 comment · Fixed by #6745
Labels
bug This issue is a bug. p3 This is a minor priority issue queued This issues is on the AWS team's backlog

Comments

@trivikr
Copy link
Member

trivikr commented Dec 17, 2024

Checkboxes for prior research

Describe the bug

SDK computes the checksum even if one is passed by the customer.

If it's the same checksum algorithm, the call is successful but there's additional time spent in computing the checksum.

If it's a different checksum algorithm, the call fails as S3 returns

Regression Issue

  • Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

All, verified in v22.11.0

Reproduction Steps

import { S3 } from "@aws-sdk/client-s3";

const client = new S3();
const Bucket = "test-flexible-checksums"; // Replace with your test bucket name.
const Body = "Hello world";
const Key = "helloworld.txt";
const ChecksumAlgorithm = "SHA256";
const ChecksumCRC32 = "i9aeUg==";

await client.putObject({ Bucket, Key, Body, ChecksumAlgorithm, ChecksumCRC32 });

Observed Behavior

Following error is thrown

/local/home/trivikr/workspace/test/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4972
  const exception = new InvalidRequest({
                    ^

InvalidRequest: Expecting a single x-amz-checksum- header. Multiple checksum Types are not allowed.
    at de_InvalidRequestRes (/local/home/trivikr/workspace/test/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4972:21)
    at de_CommandError (/local/home/trivikr/workspace/test/node_modules/@aws-sdk/client-s3/dist-cjs/index.js:4907:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async /local/home/trivikr/workspace/test/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /local/home/trivikr/workspace/test/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:485:18
    at async /local/home/trivikr/workspace/test/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /local/home/trivikr/workspace/test/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/index.js:263:18
    at async /local/home/trivikr/workspace/test/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:110:22
    at async /local/home/trivikr/workspace/test/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/index.js:138:14
    at async /local/home/trivikr/workspace/test/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:34:22 {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: 'GC7RZ5W4SC6A73V4',
    extendedRequestId: 'GWDJ2fbzh4YEiIhKZf/qvMaS3b3vIPAdheIl3WmmJ9wLdjph+QP4Cj29d4g/GXP+pMhQ3RkJHRs=',
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  Code: 'InvalidRequest',
  RequestId: 'GC7RZ5W4SC6A73V4',
  HostId: 'GWDJ2fbzh4YEiIhKZf/qvMaS3b3vIPAdheIl3WmmJ9wLdjph+QP4Cj29d4g/GXP+pMhQ3RkJHRs='
}

Expected Behavior

SDK should skip computing checksum, as application has already provided one.

Possible Solution

No response

Additional Information/Context

No response

@trivikr trivikr added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. queued This issues is on the AWS team's backlog p3 This is a minor priority issue and removed needs-triage This issue or PR still needs to be triaged. labels Dec 17, 2024
@trivikr trivikr changed the title SDK computes the checksum even if one is passed by the customer SDK computes the checksum even if one is passed by application Dec 17, 2024
@trivikr
Copy link
Member Author

trivikr commented Dec 17, 2024

While working on a fix in #6745, I noticed that the use case in minimal repro is invalid.

If user provided checksum is different from the algorithm, then S3 will return error

InvalidRequest: Value for x-amz-sdk-checksum-algorithm header is invalid.
    at de_InvalidRequestRes (/local/home/trivikr/workspace/aws-sdk-js-v3/clients/client-s3/dist-cjs/index.js:4972:21)
    at de_CommandError (/local/home/trivikr/workspace/aws-sdk-js-v3/clients/client-s3/dist-cjs/index.js:4907:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async /local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /local/home/trivikr/workspace/aws-sdk-js-v3/packages/middleware-sdk-s3/dist-cjs/index.js:485:18
    at async /local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /local/home/trivikr/workspace/aws-sdk-js-v3/packages/middleware-sdk-s3/dist-cjs/index.js:110:22
    at async /local/home/trivikr/workspace/aws-sdk-js-v3/packages/middleware-sdk-s3/dist-cjs/index.js:138:14
    at async /local/home/trivikr/workspace/aws-sdk-js-v3/packages/middleware-logger/dist-cjs/index.js:34:22
    at async file:///local/home/trivikr/workspace/test/test.mjs:10:1 {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: '1JPTJE2S4GT3EREQ',
    extendedRequestId: '166DxId3BXynPeLbGuwdpq57D/Vi5yVxI2Pf9kyn//agHWzfJJ/lb7IZ3oJzCxG4xXbFo+tf4Lo=',
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  Code: 'InvalidRequest',
  RequestId: '1JPTJE2S4GT3EREQ',
  HostId: '166DxId3BXynPeLbGuwdpq57D/Vi5yVxI2Pf9kyn//agHWzfJJ/lb7IZ3oJzCxG4xXbFo+tf4Lo='
}

Having said that, the fix still avoids additional computation of checksum if user already provided one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. p3 This is a minor priority issue queued This issues is on the AWS team's backlog
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant