Trying to get ChatHuggingFace working with with_structured_output. Keep getting "Tool with name any not found". No tools are defined so why?
#25675
Replies: 3 comments 1 reply
-
|
Hey there, @michael-newsrx! I'm here to help you with any bugs, questions, or contributions you might have. Feel free to ask for assistance while waiting for a human maintainer. I found a similar issue that might help you resolve the "Tool with name any not found" error when using To resolve this error, ensure that you are using a chat model that supports tool calling. The error suggests that the model you are using does not support the required tool calling functionality. Here is an example of how to use a supported model, such as from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_huggingface import ChatHuggingFace, HuggingFaceEndpoint
class ArticleAnalysis(BaseModel):
entities: list[str] = Field(description="Most important entities and concepts in the article")
topic: str = Field(description="General topic of the article")
source_type: str = Field(description="Type of source article")
audience: str = Field(description="The target audience for the source article")
taxonomy: str = Field(description="The taxonomy assignment based on the subject material")
subheading: str = Field(description="A one or two word topic subheading that matches the taxonomy assignment")
outline: list[str] = Field(description="A succinct cliff notes outline summary in a list format")
quality: str = Field(description="Quality of the source article", examples=["High", "Good", "Poor", "Bad"], pattern="High|Good|Poor|Bad")
# Ensure you are using a chat model that supports tools.
llm = HuggingFaceEndpoint(
repo_id="microsoft/Phi-3-mini-4k-instruct",
task="text-generation",
max_new_tokens=512,
do_sample=False,
repetition_penalty=1.03,
)
model = ChatHuggingFace(llm=llm, verbose=True)
structured_llm = model.with_structured_output(ArticleAnalysis)
output_msg = structured_llm.invoke("Analyze the following article: ...")
print(output_msg)Make sure to reference the documentation to see an up-to-date list of which models support Additionally, if you are implementing a custom chat model, ensure that it inherits from For example, the def bind_tools(
self,
tools: Sequence[Union[Dict[str, Any], Type[BaseModel], Callable, BaseTool]],
*,
tool_choice: Optional[
Union[dict, str, Literal["auto", "none", "required", "any"], bool]
] = None,
**kwargs: Any,
) -> Runnable[LanguageModelInput, BaseMessage]:
formatted_tools = [convert_to_openai_tool(tool)["function"] for tool in tools]
formatted_tool_choice = "required" if tool_choice == "any" else tool_choice
return super().bind(
available_tools=formatted_tools, tool_choice=formatted_tool_choice, **kwargs
)Ensure your custom chat model has a similar implementation to support tool calling [2][3]. |
Beta Was this translation helpful? Give feedback.
-
|
Not helpful. I'm not defining any tools to be called. |
Beta Was this translation helpful? Give feedback.
-
|
+1 facing same issue |
Beta Was this translation helpful? Give feedback.
-
|
+1 facing same issue as well |
Beta Was this translation helpful? Give feedback.
-
Checked other resources
Commit to Help
Example Code
ERROR RECEIVED
Tool error: Tool with name any not found
{ "error": "Tool error: Tool with name any not found", "error_type": "tool_error" }JSON Body Sent
{ "model": "tgi", "messages": [ { "role": "system", "content": "\nYou are a helpful, respectful and honest editor working for a peer reviewed journal.\n\nYou always answer as tersely as\npossible, while being safe. Your answers do not include any harmful, unethical,\nracist, sexist, toxic, dangerous, or illegal content. Your ensure that responses\nare socially unbiased and positive in nature.\n\nIf a question or instruction does not make any sense to you, or is not factually coherent,\nyou explain why instead of answering something not correct. If you don't know the answer to\na question, or can't follow instructions, you won't share false information.\n\nYou don't include information that is not in the supplied article.\nYou always write in the 3rd person.\nYou use markdown formatting.\nYou are detailed.\nThe reader does not have access to this\narticle so you always use direct quotes to support your answers.\n\nYou always surround direct quotations with \u201c and \u201d or \".\n\nYou never use any of the phrases \"the paper\", \"this paper\", \"the study\", \"the article\",\n\"this study\", \"this article\", \"Original article\" or similar in any of your responses.\n" }, { "role": "user", "content": "\nAnalyze the following article:\n\n* Identify the five most important entities and concepts in the article. Place in a list.\n* Describe the general topic.\n* Indicate the type of source article.\n* Indicate the target audience.\n* Suggest one taxonomy assignment based on the subject material. Format the taxonomy as\n\"Topic / Subtopic / Subsubtopic / ...\"\n* Create a one or two word topic subheading that matches the taxonomy assignment. Label it as \"Subheading:\"\n* Create a succinct \"cliff notes\" outline summary in a numbered list format.\n* Only use adjectives and adverbs included in the original source material or data.\n* If an institution is mentioned in the cliff notes, it's country also needs to be\nmentioned.\n* Never use wording like \"(Original article)\n\nArticle:\n\nOriginal title: Can language models read the genome? This one decoded mRNA to make better vaccines.\n\nThe same class of artificial intelligence that made headlines coding software and passing the bar exam has learned to\nread a different kind of text - the genetic code.\n\nThat code contains instructions for all of life's functions and follows rules not unlike those that govern human\nlanguages. Each sequence in a genome adheres to an intricate grammar and syntax, the structures that give rise to\nmeaning. Just as changing a few words can radically alter the impact of a sentence, small variations in a biological\nsequence can make a huge difference in the forms that sequence encodes.\n\nNow Princeton University researchers led by machine learning expert Mengdi Wang are using language models to home in on\npartial genome sequences and optimize those sequences to study biology and improve medicine. And they are already\nunderway.\n\nIn a paper published April 5 in the journal Nature Machine Intelligence, the authors detail a language model that used\nits powers of semantic representation to design a more effective mRNA vaccine such as those used to protect against\nCOVID-19.\n\nFound in Translation\n\nScientists have a simple way to summarize the flow of genetic information. They call it the central dogma of biology.\nInformation moves from DNA to RNA to proteins. Proteins create the structures and functions of living cells.\n\nMessenger RNA, or mRNA, converts the information into proteins in that final step, called translation. But mRNA is\ninteresting. Only part of it holds the code for the protein. The rest is not translated but controls vital aspects of\nthe translation process.\n\nGoverning the efficiency of protein production is a key mechanism by which mRNA vaccines work. The researchers focused\ntheir language model there, on the untranslated region, to see how they could optimize efficiency and improve vaccines.\n\nAfter training the model on a small variety of species, the researchers generated hundreds of new optimized sequences\nand validated those results through lab experiments. The best sequences outperformed several leading benchmarks for\nvaccine development, including a 33% increase in the overall efficiency of protein production.\n\nIncreasing protein production efficiency by even a small amount provides a major boost for emerging therapeutics,\naccording to the researchers. Beyond COVID-19, mRNA vaccines promise to protect against many infectious diseases and\ncancers.\n\nWang, a professor of electrical and computer engineering and the principal investigator in this study, said the model's\nsuccess also pointed to a more fundamental possibility. Trained on mRNA from a handful of species, it was able to decode\nnucleotide sequences and reveal something new about gene regulation. Scientists believe gene regulation, one of life's\nmost basic functions, holds the key to unlocking the origins of disease and disorder. Language models like this one\ncould provide a new way to probe.\n\nWang's collaborators include researchers from the biotech firm RVAC Medicines as well as the Stanford University School\nof Medicine.\n\nThe Language of Disease\n\nThe new model differs in degree, not kind, from the large language models that power today's AI chat bots. Instead of\nbeing trained on billions of pages of text from the internet, their model was trained on a few hundred thousand\nsequences. The model also was trained to incorporate additional knowledge about the production of proteins, including\nstructural and energy-related information.\n\nThe research team used the trained model to create a library of 211 new sequences. Each was optimized for a desired\nfunction, primarily an increase in the efficiency of translation. Those proteins, like the spike protein targeted by\nCOVID-19 vaccines, drive the immune response to infectious disease.\n\nPrevious studies have created language models to decode various biological sequences, including proteins and DNA, but\nthis was the first language model to focus on the untranslated region of mRNA. In addition to a boost in overall\nefficiency, it was also able to predict how well a sequence would perform at a variety of related tasks.\n\nWang said the real challenge in creating this language model was in understanding the full context of the available\ndata. Training a model requires not only the raw data with all its features but also the downstream consequences of\nthose features. If a program is designed to filter spam from email, each email it trains on would be labeled \"spam\" or\n\"not spam.\" Along the way, the model develops semantic representations that allow it to determine what sequences of\nwords indicate a \"spam\" label. Therein lies the meaning.\n\nWang said looking at one narrow dataset and developing a model around it was not enough to be useful for life\nscientists. She needed to do something new. Because this model was working at the leading edge of biological\nunderstanding, the data she found was all over the place.\n\n\"Part of my dataset comes from a study where there are measures for efficiency,\" Wang said. \"Another part of my dataset\ncomes from another study \\[that\\] measured expression levels. We also collected unannotated data from multiple\nresources.\" Organizing those parts into one coherent and robust whole - a multifaceted dataset that she could use to\ntrain a sophisticated language model - was a massive challenge.\n\n\"Training a model is not only about putting together all those sequences, but also putting together sequences with the\nlabels that have been collected so far. This had never been done before.\"\n\nThe paper, \"A 5' UTR Language Model for Decoding Untranslated Regions of mRNA and Function Predictions,\" was published\nin Nature Machine Learning. Additional authors include Dan Yu, Yupeng Li, Yue Shen and Jason Zhang, from RVAC Medicines;\nLe Cong from Stanford; and Yanyi Chu and Kaixuan Huang from Princeton.\n\n" } ], "frequency_penalty": null, "logit_bias": null, "logprobs": null, "max_tokens": null, "n": null, "presence_penalty": null, "response_format": null, "seed": null, "stop": null, "temperature": null, "tool_choice": { "type": "function", "function": { "name": "any" } }, "tool_prompt": null, "tools": [ { "type": "function", "function": { "name": "ArticleAnalysis", "description": "Article analysis for use by a reporter creating a new short news article.", "parameters": { "type": "object", "properties": { "entities": { "description": "Most important entities and concepts in the article", "type": "array", "items": { "type": "string" } }, "topic": { "description": "General topic of the article", "type": "string" }, "source_type": { "description": "Type of source article", "type": "string" }, "audience": { "description": "The target audience for the source article", "type": "string" }, "taxonomy": { "description": "The taxonomy assignment based on the subject material", "type": "string" }, "subheading": { "description": "A one or two word topic subheading that matches the taxonomy assignment", "type": "string" }, "outline": { "description": "A succinct cliff notes outline summary in a list format", "type": "array", "items": { "type": "string" } }, "quality": { "description": "Quality of the source article", "examples": [ "High", "Good", "Poor", "Bad" ], "pattern": "High|Good|Poor|Bad", "type": "string" } }, "required": [ "entities", "topic", "source_type", "audience", "taxonomy", "subheading", "outline", "quality" ] } } } ], "top_logprobs": null, "top_p": null, "stream": false }Beta Was this translation helpful? Give feedback.
All reactions