ConversationRetrievalChain LCEL method is returning the condensed question output

[hteeyeoh]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.chains import create_history_aware_retriever, create_retrieval_chain

condense_question_system_template = (
    "Given a chat history and the latest user question "
    "which might reference context in the chat history, "
    "formulate a standalone question which can be understood "
    "without the chat history. Do NOT answer the question, "
    "just reformulate it if needed and otherwise return it as is."
)

condense_question_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", condense_question_system_template),
        ("placeholder", "{chat_history}"),
        ("human", "{input}"),
    ]
)
history_aware_retriever = create_history_aware_retriever(
    llm, vectorstore.as_retriever(), condense_question_prompt
)

system_prompt = (
    "You are an assistant for question-answering tasks. "
    "Use the following pieces of retrieved context to answer "
    "the question. If you don't know the answer, say that you "
    "don't know. Use three sentences maximum and keep the "
    "answer concise."
    "\n\n"
    "{context}"
)

qa_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        ("placeholder", "{chat_history}"),
        ("human", "{input}"),
    ]
)
qa_chain = create_stuff_documents_chain(llm, qa_prompt)

convo_qa_chain = create_retrieval_chain(history_aware_retriever, qa_chain)

convo_qa_chain.invoke(
    {
        "input": "What are autonomous agents?",
        "chat_history": [],
    }
)

Description

Im migrating my code which using LEGACY method: ConversationalRetrievalChain.from_llm to LCEL method (create_history_aware_retriever, create_stuff_documents_chain and create_retrieval_chain)

When I stream the result, I observed that the the condensed_question that being generated also being returned. Is this expected?

System Info

langchain=0.2.10

[dosubot[bot]]

Yes, it is expected that the condensed_question being generated is also being returned when migrating code from using the legacy method ConversationalRetrievalChain.from_llm to the LCEL method (create_history_aware_retriever, create_stuff_documents_chain, and create_retrieval_chain), provided that the return_generated_question attribute is set to True ^[1].

The default value of the return_generated_question attribute is False, so if you do not want the condensed_question to be returned, ensure that this attribute is not set to True ^[1].

To continue talking to Dosu, mention @dosu.

[dosubot[bot]]

To ensure that the condensed_question is not returned, you need to set the return_generated_question attribute to False when creating the retrieval chain. Here is how you can modify your code to include this attribute:

from langchain.chains import create_history_aware_retriever, create_retrieval_chain

condense_question_system_template = (
    "Given a chat history and the latest user question "
    "which might reference context in the chat history, "
    "formulate a standalone question which can be understood "
    "without the chat history. Do NOT answer the question, "
    "just reformulate it if needed and otherwise return it as is."
)

condense_question_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", condense_question_system_template),
        ("placeholder", "{chat_history}"),
        ("human", "{input}"),
    ]
)
history_aware_retriever = create_history_aware_retriever(
    llm, vectorstore.as_retriever(), condense_question_prompt
)

system_prompt = (
    "You are an assistant for question-answering tasks. "
    "Use the following pieces of retrieved context to answer "
    "the question. If you don't know the answer, say that you "
    "don't know. Use three sentences maximum and keep the "
    "answer concise."
    "\n\n"
    "{context}"
)

qa_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        ("placeholder", "{chat_history}"),
        ("human", "{input}"),
    ]
)
qa_chain = create_stuff_documents_chain(llm, qa_prompt)

# Set return_generated_question to False
convo_qa_chain = create_retrieval_chain(history_aware_retriever, qa_chain, return_generated_question=False)

convo_qa_chain.invoke(
    {
        "input": "What are autonomous agents?",
        "chat_history": [],
    }
)

By setting return_generated_question=False in the create_retrieval_chain function, the condensed_question will not be included in the output ^[1].

To continue talking to Dosu, mention @dosu.

[ahmedbr]

When I added that parameter as you suggested, I got the following error:

TypeError: create_retrieval_chain() got an unexpected keyword argument 'return_generated_question'

Hi @hteeyeoh,

What I want is the opposite. I want to return condensed question. How did you get it in default?

[hteeyeoh]

@ahmedbr you may refer to my code sample above which i paste when raising the question.

[ahmedbr]

@ahmedbr you may refer to my code sample above which i paste when raising the question.

I'm almost using the same method. But still i don't get the standalone question in the output. Which version of langchain and other compenents do you use?

[hteeyeoh]

@ahmedbr im using langchain=0.2.10 other component version im not really sure dy but i have resolved my issue last time which i did not wish to have intermediate steps/ questions return in the streaming response.