Adding a Groq provider. #65
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gsuuon
reviewed
Sep 16, 2024
There was a problem hiding this comment.
I totally led you astray here, I'm sorry!
- I didn't realize (actually, forgot) that Groq had an OpenAI compatible API - the way to talk to these is to set the url in the options for a prompt targeting the OpenAI provider.
- I was going to suggest adding a starter chat prompt instead, but found that I'd already added one awhile ago. Must've been a late night coding session, I completely forgot about it.
That said, obviously there's a docs miss here and I think a section highlighting the use of OpenAI compatible API's (maybe with Groq as a specific example) could be really helpful. Really embarrassed about this one, sorry again.
|
@gsuuon I totally missed this too! Yes, it's probably better to add this as another example in the docs if anything. I'll try this out and I'll be happy to modify the PR to only include the example in the README. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a Groq Cloud API provider to the defaults. Groq's API is already blazingly fast, so we probably don't need streaming (atleast for text modalities). The chat example is also minimal (and can be replicated for other models/modalities, it's pretty simple)!