Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can one simply return the response of the tool, instead of routing the response to a final result handler? #127

Open
pedroallenrevez opened this issue Dec 3, 2024 · 5 comments · May be fixed by #142

Comments

@pedroallenrevez
Copy link

I'm assuming a result_tool is always needed.
In this case I want the tool to be another agent, and I just want the response from that agent, without any other LLM call.

@pedroallenrevez pedroallenrevez closed this as not planned Won't fix, can't repro, duplicate, stale Dec 3, 2024
@samuelcolvin
Copy link
Member

I don't really understand the question I'm afraid.

Result tools are not required - if the return type is str, no tool is used.

And there aren't any LLM calls after the result is returned.

@dmontagu
Copy link
Contributor

dmontagu commented Dec 4, 2024

@pedroallenrevez I've implemented one idea @samuelcolvin and I discussed for addressing this in #142. I think it works and makes it possible for tool calls to be the "result tool", and makes it possible to disable the default schema-based result tool.

But I don't love the approach. Repeating what I wrote at the end of the PR body:

However, I personally find this to be kind of an awkward set of APIs to implement what I feel is a fairly reasonable pattern of wanting to require the model to call a particular function as the way it ends its execution. I would propose we add a new decorator @agent.result_tool that requires you to return something compatible with the return type of the agent and ends the run when it is called. And if you use the @agent.result_tool decorator anywhere, we disable the default (schema-based) result tool.

Do you have any reactions to either of these proposals? In principle we could support both but I suspect @samuelcolvin might not want two different ways to override the default behavior. While I think the @agent.result_tool decorator is a better experience most of the time, the downside is that without the ctx.end_run feature, there wouldn't be a way to have a tool exit the run early. I haven't been able to think of scenarios where that would be significantly more useful than just requiring separate functions for @agent.tool tools and @agent.result_tool tools, but I also wouldn't be surprised if good use cases for dynamically deciding whether a tool call should end the run do exist.

@jlowin — @samuelcolvin mentioned you had some ideas/thoughts on this issue, feedback on either of the proposals above is very much welcome.

@jlowin
Copy link
Collaborator

jlowin commented Dec 5, 2024

Thanks @dmontagu for the thoughtful proposals. We had a chance to discuss offline, so I will try to summarize here --

I think we can distill or reframe the core issue and solution:

The fundamental problem is that using a user-supplied tool in Pydantic AI currently requires two LLM calls: one to use the tool and another to generate the final response. We want to enable single-call workflows if the user knows that a single call satisfies their objective, while maintaining strong typing.

Rather than embedding termination logic within tools via ctx.end_run(), I believe the cleaner approach is your proposed @agent.result_tool decorator (or result_tool_plain, or however you spell it). This lets us:

  • Keep tools focused purely on business logic while moving termination decisions to the agent configuration level
  • Maintain strong typing by using the tool's return type
  • Support multiple potential terminal actions through decoration/configuration (because multiple tools with simple args >> one tool with complex args)

For users who need dynamic termination logic, they can achieve this by giving their agent both regular and result tools, letting the agent itself make the choice of which to call. This keeps the architecture clean while supporting all needed use cases. The ultimate consequence of all of this is that the user is supplying the terminal tool(s) instead of having PydanticAI auto-generate one.

I think this aligns well with Pydantic AI's focus on composable, strongly-typed LLM invocations!

@pedroallenrevez
Copy link
Author

Even though I just mildly understand ctx.end_run means at this point, I think the decorator approach feels a lot more ergonomic. I think there is space for managing state of an agent inside the tools by defining an EndAgentRun, but doesn't feel intuitive for now.

Just a small observation:

  • Wouldn't it feel more natural for this specific case to have an argument in the decorator such as:
    @agent.tool(bypass=True) or some other taxonomy.
  • Also to the point of using a result_tool, I might have some logic that requires calling an LLM afterwards, and another that doesn't, so having one tool feels awkward here, and I would naturally just default for my tools to handle all the logic, and the result tool be an awkward bypass of information. Unless I'm not understanding the full scope.

Keep tools focused purely on business logic while moving termination decisions to the agent configuration level

I feel this is the key-point to achieve natural intuiteveness when building agents.
Because there is also a world where things get more complicated and there can be (strongly typed) trajectories between tools. So, termination of state, as well as routing to a next step should be manageable to the user, though I feel this might not be of immediate need for pydantic-ai.

Thanks people :)

@jlowin
Copy link
Collaborator

jlowin commented Dec 9, 2024

There is a question of how to perform validation when registering multiple response tools. Presumably a single validation function still works but the pattern of using isinstance to match the result to the type may not. For example if I have two result tools that both return a list of ints. My recommendation would be that your custom function is called and does its validation right then and there, and its output is used as the agent's final result:

@result_tool
def f(x:list[int]) → list[int]:
    ... # validate and return

@result_tool
def g(x: list[int]) → list[int]:
    ... # validate and return

(maybe this is obvious and falls out of the design but just in case)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants