Guaranteeing valid output syntax with API based models

[woolr]

I'm following along the tutorial notebooks to get up to speed with the new Guidance - but having trouble adapting this one to work with OpenAI models. Is there a notebook that covers achieving this with API based models?

The specific issue I'm hitting is the need for the context managers, not clear which should be used here and how.

Make sure you are using guidance context managers like with system():, with user(): and with assistant(): to appropriately format your guidance program for this type of model.

[riedgar-ms]

If you have a completion based API, then you should be able to just change the model creation (lm=guidance.models....) line at the top of the notebook. Changing to a chat-based model is slightly more involved, but would look something like:

with system():
    lm += "Create a role-playing character in response to the user input"
with user():
   lm += "A character attuned to the forest"
with assistant():
  lm += gen_json(schema=character_schema_obj, name="next_character")