Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bedrock.ChatCompletion.create Raises ValidationError for Non-Integer Token Values in Python 3.9 #40

Closed
5c0rp opened this issue Oct 30, 2023 · 1 comment

Comments

@5c0rp
Copy link

5c0rp commented Oct 30, 2023

Python 3.9

Code to reproduce:

import os
# set env for prompt builder
os.environ["BEDROCK_PROMPT"] = "anthropic"
os.environ["AWS_REGION"] = "us-east-1"
os.environ["AWS_ACCESS_KEY_ID"] = "xxx"
os.environ["AWS_SECRET_ACCESS_KEY"] = "xxx"

from easyllm.clients import bedrock
from easyllm.schema.base import ChatMessage

response = bedrock.ChatCompletion.create(
    model="anthropic.claude-v2",
    messages=[
        ChatMessage(role="user", content="What is 2 + 2?"),
    ],
    temperature=0.9,
    top_p=0.6,
    max_tokens=1024,
    debug=False,
)
print(response)

Result:

Traceback (most recent call last):
  File "<path_to>\main.py", line 27, in <module>
    response = bedrock.ChatCompletion.create(
  File "<path_to>\lib\site-packages\easyllm\clients\bedrock.py", line 208, in create
    usage=Usage(
  File "<path_to>\lib\site-packages\pydantic\main.py", line 159, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 2 validation errors for Usage
completion_tokens
  Input should be a valid integer, got a number with a fractional part [type=int_from_float, input_value=2.25, input_type=float]
    For further information visit https://errors.pydantic.dev/2.1/v/int_from_float
total_tokens
  Input should be a valid integer, got a number with a fractional part [type=int_from_float, input_value=11.25, input_type=float]
    For further information visit https://errors.pydantic.dev/2.1/v/int_from_float

Expected result:
{'id': 'hf-iE53Kvwnlr', 'object': 'chat.completion', 'created': 1698683234, 'model': 'anthropic.claude-v2', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': '2 + 2 = 4'}, 'finish_reason': 'stop_sequence'}], 'usage': {'prompt_tokens': 9, 'completion_tokens': 2, 'total_tokens': 11}}

@5c0rp 5c0rp changed the title pydantic ValidationError for bedrock ChatCompletion bedrock.ChatCompletion.create Raises ValidationError for Non-Integer Token Values in Python 3.9 Oct 30, 2023
@philschmid
Copy link
Owner

Is fixed in #42 and available in 0.6.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants