Cannot reproduce example about using with_structured_output with a Pydantic class on AzureChatOpenAI

[ldbolanos]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from typing import Optional

from langchain_openai import AzureChatOpenAI
from pydantic import BaseModel, Field

llm = AzureChatOpenAI(
    model="gpt-4o-2024-08-06",
    openai_api_key="XXX",
    azure_endpoint="XXX",
    openai_api_version="XXX",
)

# Pydantic
class Joke(BaseModel):
    """Joke to tell user."""

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: Optional[int] = Field(
        default=None, description="How funny the joke is, from 1 to 10"
    )


structured_llm = llm.with_structured_output(Joke)

structured_llm.invoke("Tell me a joke about cats")

Description

The line

structured_llm = llm.with_structured_output(Joke)

gives me error

ValueError: Tool choice {'type': 'function', 'function': {'name': 'any'}} was specified, but the only provided tool was Joke.

System Info

System Information
------------------
> OS:  Darwin
> OS Version:  Darwin Kernel Version 24.1.0: Thu Oct 10 21:05:14 PDT 2024; root:xnu-11215.41.3~2/RELEASE_ARM64_T8103
> Python Version:  3.11.7 (main, Jan 16 2024, 17:45:44) [Clang 14.0.3 (clang-1403.0.22.14.1)]

Package Information
-------------------
> langchain_core: 0.3.15
> langchain: 0.3.7
> langchain_community: 0.0.17
> langsmith: 0.1.139
> langchain_experimental: 0.0.51
> langchain_openai: 0.0.5
> langchain_text_splitters: 0.3.2
> langchainhub: 0.1.14

Optional packages not installed
-------------------------------
> langgraph
> langserve

Other Dependencies
------------------
> aiohttp: 3.9.3
> aiosqlite: Installed. No version info available.
> aleph-alpha-client: Installed. No version info available.
> anthropic: 0.11.0
> arxiv: Installed. No version info available.
> assemblyai: Installed. No version info available.
> async-timeout: Installed. No version info available.
> atlassian-python-api: Installed. No version info available.
> azure-ai-documentintelligence: Installed. No version info available.
> beautifulsoup4: Installed. No version info available.
> bibtexparser: Installed. No version info available.
> cassio: Installed. No version info available.
> chardet: Installed. No version info available.
> cohere: Installed. No version info available.
> dashvector: Installed. No version info available.
> databricks-vectorsearch: Installed. No version info available.
> dataclasses-json: 0.6.4
> datasets: 2.16.1
> dgml-utils: Installed. No version info available.
> elasticsearch: Installed. No version info available.
> esprima: Installed. No version info available.
> faiss-cpu: 1.8.0.post1
> faker: Installed. No version info available.
> feedparser: Installed. No version info available.
> fireworks-ai: Installed. No version info available.
> geopandas: Installed. No version info available.
> gitpython: Installed. No version info available.
> google-cloud-documentai: Installed. No version info available.
> gql: Installed. No version info available.
> gradientai: Installed. No version info available.
> hdbcli: Installed. No version info available.
> hologres-vector: Installed. No version info available.
> html2text: Installed. No version info available.
> httpx: 0.26.0
> javelin-sdk: Installed. No version info available.
> jinja2: 3.1.3
> jq: Installed. No version info available.
> jsonpatch: 1.33
> jsonschema: Installed. No version info available.
> lxml: 5.1.0
> markdownify: Installed. No version info available.
> motor: Installed. No version info available.
> msal: 1.28.0
> mwparserfromhell: Installed. No version info available.
> mwxml: Installed. No version info available.
> newspaper3k: Installed. No version info available.
> numexpr: Installed. No version info available.
> numpy: 1.26.3
> oci: Installed. No version info available.
> openai: 1.36.1
> openapi-pydantic: Installed. No version info available.
> oracle-ads: Installed. No version info available.
> orjson: 3.10.11
> packaging: 23.2
> pandas: 2.2.0
> pdfminer-six: 20231228
> pgvector: Installed. No version info available.
> praw: Installed. No version info available.
> presidio-analyzer: Installed. No version info available.
> presidio-anonymizer: Installed. No version info available.
> psychicapi: Installed. No version info available.
> py-trello: Installed. No version info available.
> pydantic: 2.8.2
> pymupdf: 1.23.21
> pypdf: 4.0.1
> pypdfium2: 4.26.0
> pyspark: Installed. No version info available.
> PyYAML: 6.0.1
> rank-bm25: Installed. No version info available.
> rapidfuzz: 3.6.1
> rapidocr-onnxruntime: Installed. No version info available.
> rdflib: Installed. No version info available.
> requests: 2.31.0
> requests-toolbelt: 1.0.0
> rspace_client: Installed. No version info available.
> scikit-learn: 1.4.0
> sentence-transformers: Installed. No version info available.
> SQLAlchemy: 2.0.25
> sqlite-vss: Installed. No version info available.
> streamlit: Installed. No version info available.
> sympy: 1.12
> tabulate: Installed. No version info available.
> telethon: Installed. No version info available.
> tenacity: 8.2.3
> tiktoken: 0.7.0
> timescale-vector: Installed. No version info available.
> tqdm: 4.66.1
> typer: 0.9.0
> types-requests: 2.31.0.20240218
> typing-extensions: 4.9.0
> upstash-redis: Installed. No version info available.
> vowpal-wabbit-next: Installed. No version info available.
> xata: Installed. No version info available.
> xmltodict: Installed. No version info available.
> zhipuai: Installed. No version info available.

[ldbolanos]

@dosu

[dosubot[bot]]

The error you're encountering is related to the with_structured_output method in AzureChatOpenAI, which is not supporting the tool_choice values of "any", "required", or True when there are multiple tools provided. This is a known limitation ^[1].

To work around this issue, you can use alternative methods for achieving structured output. Here are two examples:

1. Using json_mode:

   from langchain_openai import AzureChatOpenAI
   from langchain_core.prompts import ChatPromptTemplate
   from pydantic import BaseModel, Field
   
   class Joke(BaseModel):
       setup: str = Field(description="The setup of the joke")
       punchline: str = Field(description="The punchline to the joke")
       rating: Optional[int] = Field(default=None, description="How funny the joke is, from 1 to 10")
   
   llm = AzureChatOpenAI(
       model="gpt-4o-2024-08-06",
       openai_api_key="XXX",
       azure_endpoint="XXX",
       openai_api_version="XXX",
   )
   structured_llm = llm.with_structured_output(Joke, method="json_mode")
   system = '''You are a world class assistant for extracting information in structured JSON formats. Extract a valid JSON blob from the user input that matches the following JSON Schema: {output_schema}'''
   prompt = ChatPromptTemplate.from_messages([("system", system), ("human", "{input}")])
   chain = prompt | structured_llm
   result = chain.invoke({"input": "Tell me a joke about cats"})
   print(result)

2. Using function_calling mode with a Pydantic schema:

   from langchain_openai import AzureChatOpenAI
   from pydantic import BaseModel, Field
   
   class Joke(BaseModel):
       setup: str = Field(description="The setup of the joke")
       punchline: str = Field(description="The punchline to the joke")
       rating: Optional[int] = Field(default=None, description="How funny the joke is, from 1 to 10")
   
   llm = AzureChatOpenAI(
       model="gpt-4o-2024-08-06",
       openai_api_key="XXX",
       azure_endpoint="XXX",
       openai_api_version="XXX",
   )
   structured_llm = llm.with_structured_output(Joke, method="function_calling", include_raw=True)
   result = structured_llm.invoke("Tell me a joke about cats")
   print(result)

These methods should help you achieve structured output with AzureChatOpenAI without encountering the ValueError ^[1].

To continue talking to Dosu, mention @dosu.

[ldbolanos]

@dosu maybe method was removed in an earlier version?

My current version gives me

ValueError: Received unsupported arguments {'method': 'json_mode'}

[dosubot[bot]]

Could you provide the exact version of the langchain_openai package you are using? It seems like the method argument might not be available in your version. Additionally, if you have any specific requirements or constraints for your implementation, please let me know so I can suggest an alternative approach that fits your needs.

To continue talking to Dosu, mention @dosu.

[ldbolanos]

@dosu

langchain==0.3.7
langchain-community==0.0.17
langchain-core==0.3.15
langchain-experimental==0.0.51
langchain-openai==0.0.5
langchain-text-splitters==0.3.2
langchainhub==0.1.14

[dosubot[bot]]

In langchain_openai version 0.0.5, the with_structured_output function does indeed have a method parameter, which can be set to "function_calling", "json_mode", or "json_schema" ^[1]. If you're encountering a ValueError related to unsupported arguments, it might be due to a mismatch between the version of the package and the code you're trying to run.

Given that you're using version 0.0.5, the method parameter should be available. Double-check that the package is correctly installed and that there are no conflicting versions. If the issue persists, you might want to ensure that your environment is using the correct version of the package. You can try reinstalling the package to see if that resolves the issue:

pip uninstall langchain-openai
pip install langchain-openai==0.0.5

If you have any specific requirements or constraints, let me know, and I can suggest an alternative approach.

To continue talking to Dosu, mention @dosu.