Bedrock Claude 3.5 Sonnet v2 is not supporting new attachments (PDF) - previous bug number 4335

[pawelwasylyszyn]

Describe the bug

Recently Anthropic has released a new version of Claude 3.5 Sonnet v2 and it's also available via Amazon Bedrock. Breaking changes in it includes the support of PDF attachments (earlier it was limited to images). This is in beta state and can be enabled with anthropic-beta header (See docs). But when I use their model via Bedrock, it is giving me error.

Per previous bug - #4335 - it should be already fixed in the newest SDK, but even after upgrading boto3 to the latest version it throws an error.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

PDF support should work in Anthropic's Claude 3.5 Sonnet v2 model with anthropic-beta header.

Current Behavior

Exception in thread "main" software.amazon.awssdk.services.bedrockruntime.model.ValidationException : messages.0.content.1: Input tag 'document' found using 'type' does not match any of the expected tags: 'text', 'image', 'tool_use', 'tool_result' (Service: BedrockRuntime, Status Code: 400, Request ID: something

Reproduction Steps

import os
import json
from io import BytesIO
from base64 import b64encode
from dotenv import load_dotenv

import boto3
from botocore.config import Config
from anthropic import AnthropicBedrock

def query_claude_v2():
load_dotenv()

prompt = "extract this file"
input_pdf = "/Users/shivam/Downloads/11-1-43-9-11.pdf"
message_content = [{"type": "text", "text": prompt}]

with open(input_pdf, "rb") as file:
    message_content.append(
        {
            "type": "document",
            "attrs": {
                "format": "pdf",
                "name": "12.pdf",
                "source": {
                    "bytes": list(file.read())
                }
            }
        }
    )

client = boto3.Session().client(
    service_name="bedrock-runtime",
    region_name=os.environ["region"],
    aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
    aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
    aws_session_token=os.environ["AWS_SESSION_TOKEN"],
    config=Config(read_timeout=600) # 10 minutes
)

def add_custom_headers(request, **kwargs):
    request.headers['anthropic-beta'] = 'pdfs-2024-09-25'
    print(request)

client.meta.events.register('before-send.bedrock-runtime.InvokeModel', add_custom_headers)

native_request = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 8192,
    "messages": [{"role": "user", "content": message_content}],
    "temperature": 0.2
}
request = json.dumps(native_request)
response = client.invoke_model(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    body=request
)
response_body = json.loads(response["body"].read())
response_text = response_body["content"][0]["text"]

return response_text

def query_claude_v2_anthropic():
load_dotenv()

prompt = "extract this file"
input_pdf = "/Users/shivam/Downloads/11-1-43-9-11.pdf"
with open(input_pdf, "rb") as f:
    pdf_stream = BytesIO(f.read())

bedrock_client = AnthropicBedrock(
    aws_access_key=os.environ["AWS_ACCESS_KEY_ID"],
    aws_secret_key=os.environ["AWS_SECRET_ACCESS_KEY"],
    aws_session_token=os.environ["AWS_SESSION_TOKEN"],
    aws_region=os.environ["region"],
    timeout=600
)
response = bedrock_client.beta.messages.create(
    max_tokens=8192,
    messages=[{
        "role": "user", "content": [
            {"type": "text", "text": prompt},
            {"type": "document", "source": {
                "type": "base64",
                "media_type": "application/pdf",
                "data": b64encode(pdf_stream.getvalue()).decode("utf-8")
            }}
        ]
    }],
    model="anthropic.claude-3-5-sonnet-20241022-v2:0",
    temperature=0.2,
    betas=['pdfs-2024-09-25']
)

return response.content[0].text

print(query_claude_v2())

print(query_claude_v2_anthropic())

Possible Solution

No response

Additional Information/Context

No response

SDK version used

boto3 1.35.59

Environment details (OS name and version, etc.)

MacOS 14.7.1

[tim-finnigan]

Thanks for reaching out — the previous issue you linked (#4335) is a feature request and not a bug. PDF support is currently supported in public beta through Anthropic, but this is not currently supported through Bedrock. Please refer to my comment on the previous issue:

Thanks for reaching out. This use case is currently not supported. The Bedrock team would need to add support in order to enable AWS SDKs like Boto3 and the Java SDK to support this, as they maintain the underlying APIs like InvokeModel.

I searched internally and found that the team is already tracking this feature request in their backlog. We cannot guarantee if or when a service would implement a new feature like this, but I recommend referring to the blog and CHANGELOG for updates going forward. I will close this as not planned on the Boto3 side because the Bedrock team would need to implement this change.

[ashokrs]

Please provide an update when this issue is addressed.