Passing endpoint_url to s3 client will alter s3 key

[runawaycoast]

Describe the bug

When pass the endpoint_url to constructor of boto3 s3 client, it will alter the key on uploading by adding the bucket name as prefix

tmp/test.txt becomes bucket_name/tmp/text.txt

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

passing endpoint_url should not create a new folder and alter the s3 object key

Current Behavior

it alters the intended s3 key

Reproduction Steps

bucket_name = 'hello'
endpoint_url = "https://hello.s3.amazonaws.com"
local_file = "/tmp/1.yaml"
s3_key = local_file[1:]
s3_client = boto3.client('s3')
s3_client.upload_file(Filename=local_file Bucket=bucket_name, Key=s3_key)
# created a object in hello bucket with tmp/1.yaml key

s3_client = boto3.client('s3', endpoint_url=endpoint_url)
s3_client.upload_file(Filename=local_file Bucket=bucket_name, Key=s3_key)
# created a object in hello bucket with hello/tmp/1.yaml key

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.35.41

Environment details (OS name and version, etc.)

Debian GNU/Linux 12 (bookworm)

[adev-code]

Hi @runawaycoast, thanks for reaching out. Can you go into more detail about why you are specifying an endpoint_url? An S3 client without specifying an endpoint_url would automatically connect to the S3 endpoint. Specifying an endpoint_url is only necessary if you're using non-AWS S3 storage. Thanks!

[runawaycoast]

Hi, @adev-code Thanks for taking a look at this issue. I was using endpoint_url to point to localstack for my local development, but instead of setting it to None for higher environment, I set it to the bucket url. I am just to point out the issue and see if we can solve the inconsistent behavior.

[adev-code]

Hi @runawaycoast, thanks for the update. When endpoint_url is used, it only has to be
endpoint_url = "https://s3.amazonaws.com".

• Then for doing a request using Path Style (Path Style is by default if addressing_style is not specified):

s3_client1 = boto3.client('s3', endpoint_url=endpoint_url)
s3_client1.upload_file(Filename=local_file, Bucket=bucket_name, Key=s3_key)

• OR by doing a virtual request (explicitly specifying addressing_style):

s3_client2 = boto3.client('s3', endpoint_url=endpoint_url, config=Config(s3={'addressing_style': 'virtual'}))
s3_client2.upload_file(Filename=local_file, Bucket=bucket_name, Key=s3_key)

Please let me know if you have any follow up questions. Thanks.

[runawaycoast]

Thank you so much for providing the solution in detail, no further questions from my side.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.