add bedrock cerebras and pixtral #324
Conversation
app/lib/.server/llm/model.ts
Outdated
| @@ -56,6 +62,25 @@ export function getGroqModel(apiKey: string, model: string) { | |||
| return openai(model); | |||
| } | |||
|
|
|||
| export function getAmazonBedrockModel(apiKey: string, model: string) { | |||
There was a problem hiding this comment.
Seems like we will need to add more to add support for this, as it requires 3 parameters instead of one API key.
There was a problem hiding this comment.
I had a test run with this, and it seems to be working. If you wanna see just lmk.
There was a problem hiding this comment.
What I mean is that its not possible to set API key from UI right?
It only works when set locally in env parameters
There was a problem hiding this comment.
I think we can add it later it just should not show in the list if API key is not set
Or, we should allow setting it from UI somehow.
We will need for some providers like openai like or local ones like ollama and lm studio to give url + api key + additional parameters
app/utils/constants.ts
Outdated
| staticModels: [ | ||
| { name: 'anthropic.claude-3-5-sonnet-20241022-v2:0', label: 'Claude 3.5 Sonnet v2', provider: 'Bedrock' }, | ||
| ], | ||
| getApiKeyLink: 'https://platform.deepseek.com/api_keys' |
There was a problem hiding this comment.
Not sure but I would probably link to this
https://docs.aws.amazon.com/bedrock/latest/userguide/getting-started-api.html
There was a problem hiding this comment.
Oh that's a mistake, I forgot to change the url when I was merging changes manually after syncing my fork. I'll correct that asap
|
There is also conflict. Or at least hide it from UI if API key is not set If hosted without a key it will shop up in the list and not work. |
|
Could you look into it? I would, but I prefer working on the backend, and it'd be greatly 👍 appreciated. |
|
Bit later than. Need to do some other things first. |
|
@ArulGandhi can you please update this |
dustinwloring1988
added
WIP
|
I'll update soon. Seeing that token limits are now implemented smoothly, we can make this work, 100%. Give me a day |
|
Looking forward. Bedrock support would be amazing |
Added amazon bedrock (token limit manual change reqd to use) and added cerebras as asked in another pr. pixtral was restored. @coleam00 Please look into the artificial api token limit for amazon bedrock, it's important to have bedrock as well.