-
Notifications
You must be signed in to change notification settings - Fork 178
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AUTH-421 Fargate FastAPI container #15287
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left a few comments but they aren't a blocker.
the changes make sense to me.
Thank you for doing this!
I want to understand more on 176s. Sometimes, to generate protocol we may need more than 3 mins, do you think 176s is related to this? |
178 seconds is the maximum possible for this architecture and is already not optimal for waiting on a POST request to return a response. There are many other ways to do this we will need to iterate on performance and other methods getting the response. |
|
||
## Install a dev dependency | ||
|
||
`python -m pipenv install pytest==8.2.0 --dev` | ||
|
||
## Install a production dependency | ||
|
||
`python -m pipenv install openai==1.25.1` | ||
`python -m pipenv install openai==1.30.4` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should not keep changing the version please. Reproducibility is already difficult due to GPT model itself.
@@ -4,26 +4,29 @@ verify_ssl = true | |||
name = "pypi" | |||
|
|||
[packages] | |||
openai = "==1.25.1" | |||
openai = "==1.30.4" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
better to stick to 1.25.1, AI projects are updated fast. it may damage things unexpectedly.
Switch architecture yet again
This is all deployed on staging and is responding to requests from #15193