get_available_regions() empty for several services

[apparentorder]

Describe the bug

get_available_regions() returns an empty list for a handful of services, for example bedrock and timestream-query.

This seems to be the same issue as #2324 which refers to sagemaker-a2i-runtime.

Expected Behavior

Correct list of regions is returned.

Current Behavior

Test script output:

apprunner: ap-northeast-1,ap-southeast-1,ap-southeast-2,eu-central-1,eu-west-1,us-east-1,us-east-2,us-west-2
finspace: ca-central-1,eu-west-1,us-east-1,us-east-2,us-west-2
sagemaker-a2i-runtime: 
timestream-query: 
timestream-write: ap-northeast-1,ap-southeast-2,eu-central-1,eu-west-1,us-east-1,us-east-2,us-west-2
bedrock: 

Reproduction Steps

Test script:

import botocore.session

session = botocore.session.get_session()
print("apprunner: " + ",".join(session.get_available_regions('apprunner')))
print("finspace: " + ",".join(session.get_available_regions('finspace')))
print("sagemaker-a2i-runtime: " + ",".join(session.get_available_regions('sagemaker-a2i-runtime')))
print("timestream-query: " + ",".join(session.get_available_regions('timestream-query')))
print("timestream-write: " + ",".join(session.get_available_regions('timestream-write')))
print("bedrock: " + ",".join(session.get_available_regions('bedrock')))

Possible Solution

No response

Additional Information/Context

No response

SDK version used

commit 96fd848

Environment details (OS name and version, etc.)

n/a

[tim-finnigan]

Hi @apparentorder thanks for reaching out. We have received several related issues in the past similar to this. I want to highlight that the get_available_regions documentation notes:

The list of regions returned by this method are regions that are explicitly known by the client to exist and is not comprehensive. A region not returned in this list may still be available for the provided service.

Here is the latest team update on this topic: aws/aws-sdk#206 (comment)

Upon discussing this further with a colleague we wanted to clarify that the Python SDK team is currently investigating paths forward for this method, and deprecation is one possible outcome. Here is why:

As mentioned above, the source of information returned by get_available_regions is the endpoints.json file. This file is embedded in botocore and until version 1.28.0 was used for endpoint resolution in botocore.

The get_available_regions method was introduced many years ago as a secondary use of endpoints.json data, long before other sources of service availability were available. This has always been a sub-optimal solution because the information in endpoints.json is inherently frozen at release time. And even at release time the data in endpoints.json may lag behind reality due to how it is generated internally. The boto3 documentation addresses this (the function is not documented in the botocore documentation).

Since get_available_regions was introduced, the SSM Parameter Store API was introduced to solve the same user need better in 2019. Endpoint resolution in SDKs migrated to a different data source starting with botocore 1.28.0. As a result, endpoints.json will become an even less complete and up-to-date data source.

If completeness and recency of information is important for your use, switching to the SSM API is, and has been, the recommended approach.

Due to the documented limitations and background described above, the team recommends using the SSM APIs referenced in the linked blog post in place of the get_available_regions command. For example with Bedrock you could run this boto3 method or CLI command:

aws ssm get-parameters-by-path \
  --path /aws/service/global-infrastructure/services/bedrock/regions --output json | \
  jq '.Parameters[].Value'

That currently returns:

"ap-southeast-1"
"us-east-1"
"us-west-2"

Which does correspond to the supported regions section documented here: https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-service.html#bedrock-regions

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.