Support for InvokeModelWithResponseStream param in invoke_model and InvokeModelWithResponseStream in Bedrock

[realyze]

Describe the feature

AWS Bedrock now supports LLM prompt caching (as a beta feature) and it added a new explicitPromptCaching parameter to invoke_model and invoke_model_with_response_stream. But this param is not yet reflected in the boto request shape so trying to use the param fails validation. It would be great if we could add it!

Use Case

We'd love to use the new bedrock prompt caching feature to reduce time to first token.

Proposed Solution

Add explicitPromptCaching to InvokeModelRequest and InvokeModelWithResponseStream in botocore/data/bedrock-runtime/<date>/service-2.json.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

1.35.*

Environment details (OS name and version, etc.)

Any environment

[tim-finnigan]

Thanks for reaching out. Here are the Bedrock docs on prompt caching: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html, where it currently notes at the top:

Amazon Bedrock prompt caching is currently only available to a select number of customers. To learn more about participating in the preview, see Amazon Bedrock prompt caching.

For now you can request access to the prompt caching preview in order to try using this feature. Since the Bedrock team owns the upstream InvokeModel API that the Boto3 invoke_model command calls, new features must be supported by service APIs in order for SDKs like Boto3 to access them. Please stay tuned to the blog and CHANGELOG for updates.