OpenAI asynchronous support Guidance reborn #478
Replies: 4 comments 4 replies
-
I use asyncio for a similar task, wouldn't this work with guidance?: `
|
Beta Was this translation helpful? Give feedback.
-
I will face this problem (asynchronous requests) within 2-3 days, so I will put my results here (if not, you can send me a reminder in this thread) |
Beta Was this translation helpful? Give feedback.
-
Hi! We don't ever call the async openai What we do really need, but have not finished yet is the streaming api from the 0.0.64 version where you can get a partial state as the program executes. That is on the list of things to add :) |
Beta Was this translation helpful? Give feedback.
-
So, if I want to run 100 requests at the same time, how to I do it? Do you please have a code sample for this / a small manual? |
Beta Was this translation helpful? Give feedback.
-
Hi Guidance contributors and community,
I'm wanting to make the change to the new Guidance 0.1 but I wanted to verify first whether I can run chat based OpenAI model asynchronously in the new paradigm? The old Guidance version allowed for an async_mode parameter in the program definition but when I try and search in the new source code, there seems to be no implementations of .acreate calls to OpenAI.
It would be of great help to me if someone could either confirm or disconfirm this, and if the former is the case, what would be the minimally viable example implementation of this pattern?
Thanks a lot in advance and thank you for the great work on Guidance!
Nick
Beta Was this translation helpful? Give feedback.
All reactions