max_tokens param not work in ChatPerplexity model

[YJU-KimJeongSu]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

chat_prompt = ...

model = ChatPerplexity(
    pplx_api_key=PERPLEXITY_API_KEY,
    model="llama-3.1-sonar-small-128k-online",
    temperature=0.1,
    max_tokens=200,
)

chain = chat_prompt | model

response = chain.invoke(...)

Error Message and Stack Trace (if applicable)

No response

Description

Perplexity Docs Example :

import requests

url = "https://api.perplexity.ai/chat/completions"

payload = {
    "max_tokens": 100,
    "model": "llama-3.1-sonar-small-128k-online",
    "messages": [
        {
            "content": "read me a long fairy tale",
            "role": "user"
        }
    ]
}
headers = {
    "Authorization": ...,
    "Content-Type": "application/json"
}

---

Langchain’s Current Implementation :

langchain/libs/community/langchain_community/chat_models/perplexity.py

Lines 147 to 156 in 1691884

	@property
	def _default_params(self) -> Dict[str, Any]:
	"""Get the default parameters for calling PerplexityChat API."""
	return {
	"request_timeout": self.request_timeout,
	"max_tokens": self.max_tokens,
	"stream": self.streaming,
	"temperature": self.temperature,
	**self.model_kwargs,
	}

langchain/libs/community/langchain_community/chat_models/perplexity.py

Lines 267 to 275 in 1691884

	@property
	def _invocation_params(self) -> Mapping[str, Any]:
	"""Get the parameters used to invoke the model."""
	pplx_creds: Dict[str, Any] = {
	"api_key": self.pplx_api_key,
	"api_base": "https://api.perplexity.ai",
	"model": self.model,
	}
	return {**pplx_creds, **self._default_params}

---

Suggestion :

 @property 
 def _default_params(self) -> Dict[str, Any]: 
     """Get the default parameters for calling PerplexityChat API.""" 
     return { 
         "request_timeout": self.request_timeout, 
         "stream": self.streaming, 
         "temperature": self.temperature, 
         **self.model_kwargs, 
     } 

 @property 
 def _invocation_params(self) -> Mapping[str, Any]: 
     """Get the parameters used to invoke the model.""" 
     pplx_creds: Dict[str, Any] = { 
         "api_key": self.pplx_api_key, 
         "api_base": "https://api.perplexity.ai", 
         "model": self.model, 
         "max_tokens": self.max_tokens,
     } 
     return {**pplx_creds, **self._default_params} 

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 24.1.0: Thu Oct 10 22:08:48 PDT 2024; root:xnu-11215.41.3~5/RELEASE_ARM64_T6000
Python Version: 3.10.14 (main, Mar 19 2024, 21:46:16) [Clang 15.0.0 (clang-1500.3.9.4)]

Package Information

langchain_core: 0.3.17
langchain: 0.2.15
langchain_community: 0.2.14
langsmith: 0.1.128
langchain_anthropic: 0.3.0
langchain_experimental: 0.0.57
langchain_google_genai: 2.0.4
langchain_google_vertexai: 1.0.4
langchain_mongodb: 0.2.0
langchain_openai: 0.0.5
langchain_text_splitters: 0.2.2

Optional packages not installed

langgraph
langserve

Other Dependencies

aiohttp: 3.9.3
anthropic: 0.39.0
anthropic[vertexai]: Installed. No version info available.
async-timeout: 4.0.3
dataclasses-json: 0.6.7
defusedxml: 0.7.1
faker: Installed. No version info available.
google-cloud-aiplatform: 1.51.0
google-cloud-storage: 2.16.0
google-generativeai: 0.8.3
httpx: 0.27.2
jinja2: 3.1.2
jsonpatch: 1.33
numpy: 1.23.1
openai: 1.10.0
orjson: 3.10.2
packaging: 23.2
pandas: 2.2.0
pillow: 10.1.0
presidio-analyzer: Installed. No version info available.
presidio-anonymizer: Installed. No version info available.
pydantic: 2.9.2
pymongo: 4.8.0
PyYAML: 6.0.1
requests: 2.31.0
sentence-transformers: Installed. No version info available.
SQLAlchemy: 2.0.25
tabulate: 0.9.0
tenacity: 8.2.3
tiktoken: 0.5.2
typing-extensions: 4.9.0
vowpal-wabbit-next: Installed. No version info available.

[fanericcw]

Hello, we are a group of students from the University of Toronto Scarborough. We are interested in working on this issue, and we hope to have a working PR soon.

[arnavp103]

Hi, I'm part of the team investigating this. I was able to successfully reproduce this error and found that not only is max_tokens not respected, neither is temperature and I presume the other kwargs as well. Your suggested fix unfortunately didn't work, but we're investigating another fix and should have have it by tomorrow.