AzCopy Azure government - Copy finish sucessfully but some operations return 401 Server failed to authenticate the request

[rodolforeinaperficient]

Which version of the AzCopy was used?

Note: The version is visible when running AzCopy without any argument

10.26

Which platform are you using? (ex: Windows, Mac, Linux)

Test on windows and linux

What command did you run?

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.

Environment variables set:

set AZCOPY_AUTO_LOGIN_TYPE=WORKLOAD
set AZURE_AUTHORITY_HOST=https://login.microsoftonline.us/
set AZURE_CLIENT_ID={azure-goverment-cli}
set AZURE_FEDERATED_TOKEN_FILE={token file}
set AZURE_TENANT_ID={azure government tenant id}
set AZCOPY_TENANT_ID={azure government tenant id}

AZCOPY COPY Command

azcopy copy "https://{storage}.blob.core.usgovcloudapi.net/{container}/vehicle_small.mp4" "https://{storage}.blob.core.usgovcloudapi.net/{storage}/vehicle_small-copy1.mp4" --block-blob-tier Hot

What problem was encountered?

First operation:

GET https://{storage}.blob.core.usgovcloudapi.net/{container}?restype=container HTTP/2
host: {storage}.blob.core.usgovcloudapi.net
accept: application/xml
x-ms-version: 2023-08-03
user-agent: AzCopy/10.26.0 azsdk-go-azblob/v1.4.0 (go1.22.5; Windows_NT)
x-ms-client-request-id: 9ac4b375-a688-40cf-721e-d7089f1ff792
accept-encoding: gzip

Result:

HTTP/1.1 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
Content-Length: 302
Content-Type: application/xml
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 1ab5d7c6-001e-004c-4fe1-f76ff7000000
x-ms-client-request-id: 9ac4b375-a688-40cf-721e-d7089f1ff792
x-ms-version: 2023-08-03
x-ms-error-code: NoAuthenticationInformation
WWW-Authenticate: Bearer authorization_uri=https://login.microsoftonline.us/{tenantid}/oauth2/authorize resource_id=https://storage.azure.com/
Date: Mon, 26 Aug 2024 18:00:40 GMT

<?xml version="1.0" encoding="utf-8"?><Error><Code>NoAuthenticationInformation</Code><Message>Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:1ab5d7c6-001e-004c-4fe1-f76ff7000000
Time:2024-08-26T18:00:40.4765439Z</Message></Error>

Second operation

HEAD https://{storage}.blob.core.usgovcloudapi.net/{container}/vehicle_small.mp4 HTTP/2
host: devpublicdataingestionst.blob.core.usgovcloudapi.net
x-ms-client-request-id: fea49e04-f27f-4afd-4d3c-241d587129b6
accept: application/xml
x-ms-version: 2023-08-03
user-agent: AzCopy/10.26.0 azsdk-go-azblob/v1.4.0 (go1.22.5; Windows_NT)

Response

HTTP/1.1 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: c1faf108-701e-0034-15e1-f7cc0f000000
x-ms-client-request-id: fea49e04-f27f-4afd-4d3c-241d587129b6
x-ms-version: 2023-08-03
x-ms-error-code: NoAuthenticationInformation
WWW-Authenticate: Bearer authorization_uri=https://login.microsoftonline.us/{tenantId}/oauth2/authorize resource_id=https://storage.azure.com/
Date: Mon, 26 Aug 2024 18:00:42 GMT

How can we reproduce the problem in the simplest way?

Run the command azcopy copy using Azure government, use a sniff tool to see the request and response

Have you found a mitigation/solution?

Looking the response of both request, the resource id is not correct for Azure Goverment Cloud

[vibhansa-msft]

Kindly share the AzCopy logs for us to investigate further on this.

[adreed-msft]

Hey there, the resource https://storage.azure.com/ is actually correct in that environment. Could you please share the logs with us?

[rodolforeinaperficient]

Hi, @vibhansa-msft @adreed-msft sure these are the logs . The file CopyUsingWorkloadAZGov-edit.json is and export from fiddler which show all the http request made, in the lines 937 and 1051 you can see the status code 401, and in previous line the request. about https://storage.azure.com/ we are excpecting seeing something like https://storage.usgovcloudapi.net/ because is Azure government.

This error is part for something we are using azcopy continously and in parallel so we are facing many request problems #2799

CopyUsingWorkloadAZGov-edit.json
42d16479-aaf5-614a-5e73-6769fb3afb15-chunks.log
42d16479-aaf5-614a-5e73-6769fb3afb15-scanning.log
42d16479-aaf5-614a-5e73-6769fb3afb15.log