Changing GPT version

[rayxfelipe]

I tried changing the environment variable in the webapp to gpt4o and when I do, the app stops working. I'm getting "application error". The gpt4o exist as a deployment in AOAI. Is there anything else I need to update besides the env variable for gpt deployment in the web app?

[ravikhunt]

I tried changing the environment variable in the webapp to gpt4o and when I do, the app stops working. I'm getting "application error". The gpt4o exist as a deployment in AOAI. Is there anything else I need to update besides the env variable for gpt deployment in the web app?

You can do two things and it will work.
-Update the code base
app/backend/core/modelhelper.py

MODELS_2_TOKEN_LIMITS = {
    "gpt-35-turbo": 4097,
    "gpt-3.5-turbo": 4097,
    "gpt-35-turbo-16k": 16385,
    "gpt-3.5-turbo-16k": 16385,
    "gpt-4": 8192,
    "gpt-4-32k": 32768,
    "gpt-4o": 128000
}

1. Add the "get-4o" in the MODELS_2_TOKEN_LIMITS
2. update the "tiktoken" python package version to "tiktoken==0.7.0" in both functions and app

[dayland]

Information Assistant now supports use of gpt-4o OOTB. This change was added in #729