Compatibility Question: LangChain with Mistral-Large-2411's New Prompt Format

[JetXu-LLM]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_mistralai.chat_models import ChatMistralAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser

llm = ChatMistralAI(mistral_api_key=mistral_api_key, model="mistral-large-2411")

system_message = "..."
human_message = "..."

context = ChatPromptTemplate.from_messages([
    ("system", system_message), #prompt
    ("human", human_message) #user query
])

chain = (
    context
    | llm
    | StrOutputParser()
)

response = ""
for chunk in chain.stream({"para": para}):
    response += chunk

Description

Question

I've noticed that Mistral has changed their prompt format in version 2411:
https://huggingface.co/mistralai/Mistral-Large-Instruct-2411

Previous format (2407):

<s>[INST] user message[/INST] assistant message</s>[INST] system prompt + "\n\n" + user message[/INST]

New format (2411):

<s>[SYSTEM_PROMPT] system prompt[/SYSTEM PROMPT][INST] user message[/INST] assistant message</s>[INST] user message[/INST]

Currently, I'm using LangChain's ChatPromptTemplate with Mistral (please refer to my sample code)

Questions:

1. Does this prompt format change in Mistral 2411 require any updates to LangChain's Mistral integration?
2. Is the current approach of using ChatPromptTemplate with system and human messages still valid?
3. Are there any recommended changes to ensure optimal compatibility with Mistral 2411?

Additional Context

• The format change seems to affect how Mistral handles system prompts and context adherence
• Current implementation works with Mistral-large-2407, but want to ensure it's following best practices with the new format
• Using streaming responses with StrOutputParser

What I've Tried

• Current implementation still works but unsure if it's optimal for the new format
• Reviewed LangChain documentation but found no specific guidance for Mistral 2411
• Tested with both formats and noticed some differences in context adherence

Expected Behavior

Clear guidance on:

1. Whether current implementation needs updates
2. Best practices for using LangChain with Mistral 2411
3. Any recommended changes to prompt templates or chain structure

System Info

• LangChain Version: latest
• Python Version: 3.11
• Operating System: AWS Lambda

[JetXu-LLM]

I'm following up on my own question with our findings after extensive testing and implementation in production.

TL;DR: The current LangChain implementation works correctly with Mistral-Large-2411 through its API abstraction layer. No changes are needed in the code structure.

Detailed Findings:

1. API Integration Analysis
   Through our testing with a custom HTTP client interceptor, we confirmed that LangChain correctly handles the new prompt format when interacting with Mistral's API. The messages are properly formatted as:

{
    "messages": [
        {
            "role": "system",
            "content": "system_message"
        },
        {
            "role": "user",
            "content": "user_message"
        }
    ],
    "model": "mistral-large-2411"
}

2. Prompt Structure
   The current ChatPromptTemplate approach remains valid:

context = ChatPromptTemplate.from_messages([
    ("system", system_message),
    ("human", human_message)
])

3. Performance Validation
   We've successfully processed thousands of code reviews using this implementation, confirming its reliability in production.

I've documented our complete technical journey and findings here: Medium article link

Marking this as resolved since we've confirmed the compatibility and optimal implementation approach. 🎉