From fabec4fed3f79dd524952279faf6b56bb405eaa2 Mon Sep 17 00:00:00 2001 From: Yiran Wu <32823396+kevin666aa@users.noreply.github.com> Date: Sun, 17 Nov 2024 01:47:34 -0500 Subject: [PATCH 01/11] update --- website/blog/2024-11-17-Swarm/index.mdx | 32 ++++ .../docs/topics/groupchat/using_swarm.ipynb | 178 ++++++++++++++++++ 2 files changed, 210 insertions(+) create mode 100644 website/blog/2024-11-17-Swarm/index.mdx create mode 100644 website/docs/topics/groupchat/using_swarm.ipynb diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx new file mode 100644 index 000000000..094813bbd --- /dev/null +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -0,0 +1,32 @@ +title: Building Swarm-based agents with GroupChat +authors: + - yiranwu + - marklysze +tags: [groupchat, swarm] +--- + +AutoGen Groupchat now supports a new speaker selection method, `swarm`, which allows quick implementation of the swarm orchestration from OpenAI's [Swarm](https://github.com/openai/swarm) framework. + +Here are two key features of the `swarm` speaker selection method: +- **Headoffs**: Agents can transfer control to another agent via function calls, enabling smooth transitions within workflows. +- **Context Variables**: Agents can dynamically update shared variables through function calls, maintaining context and adaptability throughout the process. + + +## Tips for Building Swarm-based Agents + +1. The `swarm` speaker selection method needs to be used in conjunction with `SwarmAgent` and `SwarmResult` classes. + +```python +from autogen.agentchat.swarm.swarm_agent import SwarmAgent, SwarmResult +``` + +2. The `Swarm` orchestration require human involvement. When a user (human) sends requests, the swarm would transfer controls within agents so that the most suitable agent can respond. +Thus, a `UserProxyAgent` must be present in the agent list to accept user requests. + +```python +from autogen import UserProxyAgent +``` + +## Getting Started + +Please setup LLM configs. diff --git a/website/docs/topics/groupchat/using_swarm.ipynb b/website/docs/topics/groupchat/using_swarm.ipynb new file mode 100644 index 000000000..18cf8a55f --- /dev/null +++ b/website/docs/topics/groupchat/using_swarm.ipynb @@ -0,0 +1,178 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Using Swarm Ochestration\n", + "\n", + "You can set the speaker selection method to `swarm` to use the swarm-like orchestration.\n", + "\n", + "- **Headoffs**: Agents can transfer control to another agent via function calls, enabling smooth transitions within workflows.\n", + "- **Context Variables**: Agents can dynamically update shared variables through function calls, maintaining context and adaptability throughout the process.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## SwarmAgent and SwarmResult\n", + "\n", + "To allow easier setup of the swarm, a `SwarmAgent` class is provided. When passing functions to this agent, the agent will execute any tool calls from LLMs directly. \n", + "\n", + "```python\n", + "agent = SwarmAgent(\n", + " ..., # same arguments as AssistantAgent\n", + " functions=[func_1, func_2, ...] # a new parameter that allows passing a list of functions\n", + ")\n", + "```\n", + "\n", + "We also create a `SwarmResult` class, where you can return a value, the next agent to call and update context variables at the same time.\n", + "\n", + "```python\n", + "def func_name() -> SwarmResult:\n", + " return SwarmResult(value=\"value\", agent=, context_variables={\"key\": \"value\"})\n", + "```\n", + "\n", + "When defining function calls, you can return one of the following types:\n", + "- `str`\n", + "- `SwarmAgent`: if a `SwarmAgent` object is returned, it will be called next.\n", + "- `SwarmResult`: if a `SwarmResult` object is returned, we will use it to update the context variables, return a value and call the next agent.\n", + "\n", + "\n", + "**Notes for creating the function calls**\n", + "- For input arguments, you must define the type of the argument, otherwise the registration will fail (e.g. `arg_name: str`). \n", + "- The docstring of the function will be used as the prompt. So make sure to write a clear description.\n", + "- The function name will be used as the tool name.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from autogen.agentchat.swarm.swarm_agent import SwarmAgent, SwarmResult\n", + "\n", + "llm_config = \"Setup your config here\"\n", + "\n", + "def update_context_1(context_variables: dict) -> str:\n", + " context_variables[\"1\"] = True\n", + " return SwarmResult(value=\"success\", context_variables=context_variables)\n", + "\n", + "def update_context_2_and_transfer_to_3(context_variables: dict) -> str:\n", + " context_variables[\"2\"] = True\n", + " return SwarmResult(value=\"success\", context_variables=context_variables, agent=agent_3)\n", + "\n", + "def update_context_3(context_variables: dict) -> str:\n", + " context_variables[\"3\"] = True\n", + " return SwarmResult(value=\"success\", context_variables=context_variables)\n", + "\n", + "def transfer_to_agent_2() -> SwarmAgent:\n", + " return agent_2\n", + "\n", + "agent_1 = SwarmAgent(\n", + " name=\"Agent_1\",\n", + " system_message=\"You are Agent 1, first, call the function to update context 1, and transfer to Agent 2\",\n", + " llm_config=llm_config,\n", + " functions=[update_context_1, transfer_to_agent_2],\n", + ")\n", + "\n", + "agent_2 = SwarmAgent(\n", + " name=\"Agent_2\",\n", + " system_message=\"You are Agent 2, call the function that updates context 2 and transfer to Agent 3\",\n", + " llm_config=llm_config,\n", + " functions=[update_context_2_and_transfer_to_3],\n", + ")\n", + "agent_3 = SwarmAgent(\n", + " name=\"Agent_3\",\n", + " system_message=\"You are Agent 3, first, call the function to update context 3, and then reply TERMINATE\",\n", + " llm_config=llm_config,\n", + " functions=[update_context_3],\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Define a UserProxyAgent\n", + "\n", + "The `Swarm` orchestration require human involvement. When a user (human) sends requests, the swarm would transfer controls within agents so that the most suitable agent can respond.\n", + "\n", + "To use swarm, you need to define a `UserProxyAgent` class so that you can interact with the swarm. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from autogen import UserProxyAgent\n", + "\n", + "user = UserProxyAgent(\n", + " name=\"Human_User\",\n", + " system_message=\"Human user\",\n", + " human_input_mode=\"ALWAYS\",\n", + " code_execution_config=False,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Start the GroupChat\n", + "\n", + "Follow the GroupChat convention to define the groupchat and start the conversation.\n", + "\n", + "You can pass in a `context_variables` dictionary to groupchat to initialize the context variables. Note this variable is only used for `swarm` orchestration.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from autogen import GroupChat, GroupChatManager\n", + "\n", + "context_variables = {\"1\": False, \"2\": False, \"3\": False}\n", + "\n", + "groupchat = GroupChat(\n", + " agents=[user, agent_1, agent_2, agent_3],\n", + " messages=[],\n", + " max_round=10,\n", + " speaker_selection_method=\"swarm\",\n", + " context_variables=context_variables,\n", + ")\n", + "manager = GroupChatManager(groupchat=groupchat, llm_config=None)\n", + "chat_result = user.initiate_chat(manager, message=\"start\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "autodev", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.19" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 23183352fad1b18512ca632fa3a06bbd6d4e1011 Mon Sep 17 00:00:00 2001 From: Yiran Wu <32823396+kevin666aa@users.noreply.github.com> Date: Mon, 18 Nov 2024 20:16:55 -0500 Subject: [PATCH 02/11] update --- website/blog/2024-11-17-Swarm/index.mdx | 19 +--- .../docs/topics/groupchat/using_swarm.ipynb | 92 +++++++++---------- 2 files changed, 50 insertions(+), 61 deletions(-) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index 094813bbd..8d06e7592 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -1,4 +1,4 @@ -title: Building Swarm-based agents with GroupChat +title: Building Swarm-based agents with AutoGen authors: - yiranwu - marklysze @@ -12,21 +12,10 @@ Here are two key features of the `swarm` speaker selection method: - **Context Variables**: Agents can dynamically update shared variables through function calls, maintaining context and adaptability throughout the process. -## Tips for Building Swarm-based Agents - -1. The `swarm` speaker selection method needs to be used in conjunction with `SwarmAgent` and `SwarmResult` classes. - -```python -from autogen.agentchat.swarm.swarm_agent import SwarmAgent, SwarmResult -``` +## Getting Started -2. The `Swarm` orchestration require human involvement. When a user (human) sends requests, the swarm would transfer controls within agents so that the most suitable agent can respond. -Thus, a `UserProxyAgent` must be present in the agent list to accept user requests. +The `swarm` speaker selection method needs to be used in conjunction with `SwarmAgent` and `SwarmResult` classes. Use the `initialize_swarm_chat` to get a swarm reply. ```python -from autogen import UserProxyAgent +from autogen.agentchat.swarm.swarm_agent import SwarmAgent, SwarmResult, initialize_swarm_chat ``` - -## Getting Started - -Please setup LLM configs. diff --git a/website/docs/topics/groupchat/using_swarm.ipynb b/website/docs/topics/groupchat/using_swarm.ipynb index 18cf8a55f..30ddaf5bf 100644 --- a/website/docs/topics/groupchat/using_swarm.ipynb +++ b/website/docs/topics/groupchat/using_swarm.ipynb @@ -36,6 +36,18 @@ " return SwarmResult(value=\"value\", agent=, context_variables={\"key\": \"value\"})\n", "```\n", "\n", + "To only define transitions between agents, we provide an quick way to define the handoffs:\n", + "\n", + "```python\n", + "swarm_agent.handoff(agent, condition=\"when should transition happen\")\n", + "```\n", + "We will register a transfer function to llm config:\n", + "```\n", + "def transfer_to_agent_name():\n", + " \"\"\"\"\"\"\n", + " return agent_name\n", + "```\n", + "\n", "When defining function calls, you can return one of the following types:\n", "- `str`\n", "- `SwarmAgent`: if a `SwarmAgent` object is returned, it will be called next.\n", @@ -54,30 +66,37 @@ "metadata": {}, "outputs": [], "source": [ - "from autogen.agentchat.swarm.swarm_agent import SwarmAgent, SwarmResult\n", + "from autogen.agentchat.contrib.swarm_agent import SwarmAgent, SwarmResult\n", "\n", "llm_config = \"Setup your config here\"\n", "\n", + "\n", "def update_context_1(context_variables: dict) -> str:\n", " context_variables[\"1\"] = True\n", - " return SwarmResult(value=\"success\", context_variables=context_variables)\n", + " return SwarmResult(\n", + " value=\"success\", context_variables=context_variables\n", + " ) # Update context variables and return success\n", + "\n", "\n", "def update_context_2_and_transfer_to_3(context_variables: dict) -> str:\n", " context_variables[\"2\"] = True\n", - " return SwarmResult(value=\"success\", context_variables=context_variables, agent=agent_3)\n", + " return SwarmResult(\n", + " value=\"success\", context_variables=context_variables, agent=agent_3\n", + " ) # Update context variables, return success, and transfer to agent_3\n", + "\n", "\n", "def update_context_3(context_variables: dict) -> str:\n", " context_variables[\"3\"] = True\n", - " return SwarmResult(value=\"success\", context_variables=context_variables)\n", + " return SwarmResult(\n", + " value=\"success\", context_variables=context_variables\n", + " ) # Update context variables and return success\n", "\n", - "def transfer_to_agent_2() -> SwarmAgent:\n", - " return agent_2\n", "\n", "agent_1 = SwarmAgent(\n", " name=\"Agent_1\",\n", " system_message=\"You are Agent 1, first, call the function to update context 1, and transfer to Agent 2\",\n", " llm_config=llm_config,\n", - " functions=[update_context_1, transfer_to_agent_2],\n", + " functions=[update_context_1],\n", ")\n", "\n", "agent_2 = SwarmAgent(\n", @@ -86,50 +105,34 @@ " llm_config=llm_config,\n", " functions=[update_context_2_and_transfer_to_3],\n", ")\n", + "\n", "agent_3 = SwarmAgent(\n", " name=\"Agent_3\",\n", " system_message=\"You are Agent 3, first, call the function to update context 3, and then reply TERMINATE\",\n", " llm_config=llm_config,\n", " functions=[update_context_3],\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Define a UserProxyAgent\n", - "\n", - "The `Swarm` orchestration require human involvement. When a user (human) sends requests, the swarm would transfer controls within agents so that the most suitable agent can respond.\n", + ")\n", "\n", - "To use swarm, you need to define a `UserProxyAgent` class so that you can interact with the swarm. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from autogen import UserProxyAgent\n", - "\n", - "user = UserProxyAgent(\n", - " name=\"Human_User\",\n", - " system_message=\"Human user\",\n", - " human_input_mode=\"ALWAYS\",\n", - " code_execution_config=False,\n", - ")" + "agent_1.hand_off(agent_2, condition=\"handoff to agent 2\") # register handoff using hand_off method." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Start the GroupChat\n", + "## Use initialize_swarm_chat\n", "\n", - "Follow the GroupChat convention to define the groupchat and start the conversation.\n", + "To get response from the swarm agent, you can use the `initialize_swarm_chat` function. This function will return the response from the agent and the updated context variables.\n", "\n", - "You can pass in a `context_variables` dictionary to groupchat to initialize the context variables. Note this variable is only used for `swarm` orchestration.\n" + "```python\n", + "chat_history, context_variables, last_agent = initialize_swarm_chat(\n", + " init_agent, # which agent to start with\n", + " messages, # list of messages\n", + " agents, # pass in all the swarm agents\n", + " max_rounds, # maximum number of rounds\n", + " context_variables, # initial context variables\n", + ")\n", + "```\n" ] }, { @@ -138,19 +141,16 @@ "metadata": {}, "outputs": [], "source": [ - "from autogen import GroupChat, GroupChatManager\n", + "from autogen.agentchat.contrib.swarm_agent import initialize_swarm_chat\n", "\n", "context_variables = {\"1\": False, \"2\": False, \"3\": False}\n", - "\n", - "groupchat = GroupChat(\n", - " agents=[user, agent_1, agent_2, agent_3],\n", - " messages=[],\n", - " max_round=10,\n", - " speaker_selection_method=\"swarm\",\n", + "chat_result, context_variables, last_agent = initialize_swarm_chat(\n", + " init_agent=agent_1,\n", + " agents=[agent_1, agent_2, agent_3],\n", + " messages=\"start\",\n", " context_variables=context_variables,\n", ")\n", - "manager = GroupChatManager(groupchat=groupchat, llm_config=None)\n", - "chat_result = user.initiate_chat(manager, message=\"start\")" + "print(context_variables)" ] } ], From e4f8f362404aa115e1bb21ca2733ff2167df7fa2 Mon Sep 17 00:00:00 2001 From: Mark Sze Date: Tue, 19 Nov 2024 21:44:13 +0000 Subject: [PATCH 03/11] Move file and update blog --- website/blog/2024-11-17-Swarm/index.mdx | 22 ++++++++++++++----- .../using_swarm.ipynb => swarm.ipynb} | 2 +- 2 files changed, 18 insertions(+), 6 deletions(-) rename website/docs/topics/{groupchat/using_swarm.ipynb => swarm.ipynb} (99%) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index 8d06e7592..33269c9ac 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -1,19 +1,31 @@ -title: Building Swarm-based agents with AutoGen +title: Building Swarm-based agents with AG2 authors: - yiranwu - marklysze tags: [groupchat, swarm] --- -AutoGen Groupchat now supports a new speaker selection method, `swarm`, which allows quick implementation of the swarm orchestration from OpenAI's [Swarm](https://github.com/openai/swarm) framework. +AG2 now provides an implementation of the swarm orchestration from OpenAI's [Swarm](https://github.com/openai/swarm) framework with some additional features. -Here are two key features of the `swarm` speaker selection method: -- **Headoffs**: Agents can transfer control to another agent via function calls, enabling smooth transitions within workflows. -- **Context Variables**: Agents can dynamically update shared variables through function calls, maintaining context and adaptability throughout the process. +*Background*: the swarm orchestration is a multi-agent collaboration where agents execute tasks and are responsible for handing-off to other agents. +Here are two key features of the swarm orchestration: +- **Hand offs**: Agents can transfer control to another agent, enabling smooth and direct transitions within workflows. +- **Context Variables**: Agents can dynamically update a shared dictionary through function calls, maintaining context and adaptability throughout the process. + +In addition to the OpenAI implementation, AG2 provides: +- **Built-in human-in-the-loop**: Adding a user agent (UserProxyAgent) to your swarm will allow swarm agents to transition back to the user. Provides a means to clarify and confirm questions without breaking out of the swarm. +- **Fall-back agent**: The ability to automatically transition to a nominated swarm agent when an agent has completed their task. ## Getting Started +To create a swarm: +1. Define the functions that can be used by your `SwarmAgent`'s. +2. Create your `SwarmAgent`s (which derive from `ConversableAgent`). +3. For each agent, specify the hand-offs (transitions to another agent) and what to do when they have finished their work (termed *After Work*). + +UP TO HERE + The `swarm` speaker selection method needs to be used in conjunction with `SwarmAgent` and `SwarmResult` classes. Use the `initialize_swarm_chat` to get a swarm reply. ```python diff --git a/website/docs/topics/groupchat/using_swarm.ipynb b/website/docs/topics/swarm.ipynb similarity index 99% rename from website/docs/topics/groupchat/using_swarm.ipynb rename to website/docs/topics/swarm.ipynb index 30ddaf5bf..ad9d812ec 100644 --- a/website/docs/topics/groupchat/using_swarm.ipynb +++ b/website/docs/topics/swarm.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Using Swarm Ochestration\n", + "# Swarm Ochestration\n", "\n", "You can set the speaker selection method to `swarm` to use the swarm-like orchestration.\n", "\n", From 658ddee9f9bea29bcaba80d942b05a529e52f366 Mon Sep 17 00:00:00 2001 From: Yiran Wu <32823396+kevin666aa@users.noreply.github.com> Date: Tue, 19 Nov 2024 18:00:31 -0500 Subject: [PATCH 04/11] partial update --- website/docs/topics/swarm.ipynb | 116 ++++++++++++++++++++++++-------- 1 file changed, 89 insertions(+), 27 deletions(-) diff --git a/website/docs/topics/swarm.ipynb b/website/docs/topics/swarm.ipynb index ad9d812ec..c6d5e1674 100644 --- a/website/docs/topics/swarm.ipynb +++ b/website/docs/topics/swarm.ipynb @@ -6,58 +6,120 @@ "source": [ "# Swarm Ochestration\n", "\n", - "You can set the speaker selection method to `swarm` to use the swarm-like orchestration.\n", - "\n", + "With AG2, you can initiate a Swarm Chat similar to OpenAI's [Swarm](https://github.com/openai/swarm). In this orchestration, there are two main features:\n", "- **Headoffs**: Agents can transfer control to another agent via function calls, enabling smooth transitions within workflows.\n", "- **Context Variables**: Agents can dynamically update shared variables through function calls, maintaining context and adaptability throughout the process.\n", "\n", - "\n" + "\n", + "Instead of sending a task to a sinlge LLM agent, you can send it to a swarm of agents. Each agent in the swarm can decide whether to handoff the task to another agent. \n", + "When the last active agent's response is only a string (it doesn't suggest tool call or handoff), the chat will be terminated." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## SwarmAgent and SwarmResult\n", + "We now introduce the main components that need to be used to create a swarm chat.\n", + "\n", + "## Create `SwarmAgent`\n", + "All the agents passed to the swarm chat should be instances of this class. `SwarmAgent` is very similar to `AssistantAgent`, but it has some additional features to support function registration and handoffs.\n", "\n", - "To allow easier setup of the swarm, a `SwarmAgent` class is provided. When passing functions to this agent, the agent will execute any tool calls from LLMs directly. \n", + "### Initialization\n", + "When creating a `SwarmAgent`, you can pass in a list of functions. These functions will be converted to schemas to be passed to the LLMs, and you don't need to worry about registering the functions for execution. You can also pass back a `SwarmResult` class, where you can return a value, the next agent to call and update context variables at the same time.\n", + "\n", + "**Notes for creating the function calls**\n", + "- For input arguments, you must define the type of the argument, otherwise the registration will fail (e.g. `arg_name: str`). \n", + "- If your function requires access or modification of the context variables, you must pass in `context_variables: dict` as one argument. This argument will not be visible to the LLM (removed when registering the function schema). But when being called, the global context variables will be passed in by the swarm chat.\n", + "- The docstring of the function will be used as the prompt. So make sure to write a clear description.\n", + "- The function name will be used as the tool name.\n", "\n", "```python\n", - "agent = SwarmAgent(\n", + "def func_1(arg1: str, arg2: int) -> str:\n", + " \"\"\"Doc String\"\"\"\n", + " return \"response\"\n", + "\n", + "def transfer_to_some_agent() -> SwarmAgent:\n", + " return SwarmAgent(...)\n", + "\n", + "def func_3(context_variables: dict, arg2: str) -> SwarmResult:\n", + " # ... update context variables\n", + " context_variables[\"key\"] = \"value\"\n", + " return SwarmResult(value=\"value\", agent=, context_variables=context_variables)\n", + "\n", + "SwarmAgent(\n", " ..., # same arguments as AssistantAgent\n", - " functions=[func_1, func_2, ...] # a new parameter that allows passing a list of functions\n", + " functions=[func_1, transfer_to_some_agent, func_3] # a new parameter that allows passing a list of functions\n", ")\n", "```\n", "\n", - "We also create a `SwarmResult` class, where you can return a value, the next agent to call and update context variables at the same time.\n", + "### Registering Handoffs\n", + "While you can create an function to decide what next agent to call, we provide a quick way to register the handoff use `ON_CONDITION`. We will craft this transition function and add it to the LLM config directly.\n", "\n", "```python\n", - "def func_name() -> SwarmResult:\n", - " return SwarmResult(value=\"value\", agent=, context_variables={\"key\": \"value\"})\n", + "agent_2 = SwarmAgent(...)\n", + "agent_3 = SwarmAgent(...)\n", + "\n", + "# Register the handoff\n", + "agent_1 = SwarmAgent(...)\n", + "agent_1.handoff(hand_to=[ON_CONDITION(agent_2, \"condition_1\"), ON_CONDITION(agent_3, \"condition_2\")])\n", + "\n", + "# This is equivalent to:\n", + "def transfer_to_agent_2():\n", + " \"\"\"condition_1\"\"\"\n", + " return agent_2\n", + "\n", + "def transfer_to_agent_3():\n", + " \"\"\"condition_1\"\"\"\n", + " return agent_3\n", + " \n", + "agent_1 = SwarmAgent(..., functions=[transfer_to_agent_2, transfer_to_agent_3])\n", + "# You can also use agent_1.add_functions to add more functions after initialization\n", "```\n", "\n", - "To only define transitions between agents, we provide an quick way to define the handoffs:\n", + "### AFTER_WORK\n", + "\n", + "When the last active agent's response doesn't suggest a tool call or handoff, the chat will terminate by default. However, you can register an `AFTER_WORK` handoff to define a fallback agent if you don't want the chat to end at this agent. At the swarm chat level, you also pass in an `AFTER_WORK` handoff to define the fallback mechanism for the entire chat.\n", + "If this parameter is set for both the agent and the chat, we will prioritize the agent's setting.\n", + "\n", + "Besides fallback to an agent, we provide 3 `AfterWorkOption`:\n", + "- `TERMINATE`: Terminate the chat\n", + "- `STAY`: Stay at the current agent\n", + "- `REVERT_TO_USER`: Revert to the user agent. Only if an user agent is passed in when initializing. (See below for more details)\n", "\n", "```python\n", - "swarm_agent.handoff(agent, condition=\"when should transition happen\")\n", - "```\n", - "We will register a transfer function to llm config:\n", + "agent_1 = SwarmAgent(...)\n", + "\n", + "# Register the handoff\n", + "agent_1.handoff(hand_to=[\n", + " ON_CONDITION(...), \n", + " ON_CONDITION(...),\n", + " AFTER_WORK(agent_4) # Fallback to agent_4 if no handoff is suggested\n", + "])\n", + "\n", + "agent_2.handoff(hand_to=[AFTER_WORK(TERMINATE)]) # Terminate the chat if no handoff is suggested\n", "```\n", - "def transfer_to_agent_name():\n", - " \"\"\"\"\"\"\n", - " return agent_name\n", + "\n", + "## Initialize SwarmChat with `initiate_swarm_chat`\n", + "\n", + "After a set of swarm agents are created, you can initiate a swarm chat by calling `initiate_swarm_chat`.\n", + "\n", + "```python\n", + "chat_history, context_variables, last_active_agent = initiate_swarm_chat(\n", + " init_agent=agent_1, # the first agent to start the chat\n", + " agents=[agent_1, agent_2, agent_3], # a list of agents\n", + " messages=[{\"role\": \"user\", \"content\": \"Hello\"}], # a list of messages to start the chat\n", + " user_agent=user_agent, # optional, if you want to revert to user agent\n", + " context_variables={\"key\": \"value\"} # optional, initial context variables\n", + ")\n", "```\n", "\n", - "When defining function calls, you can return one of the following types:\n", - "- `str`\n", - "- `SwarmAgent`: if a `SwarmAgent` object is returned, it will be called next.\n", - "- `SwarmResult`: if a `SwarmResult` object is returned, we will use it to update the context variables, return a value and call the next agent.\n", + "### How is context variables updated\n", "\n", + "### When to pass in a user agent\n", "\n", - "**Notes for creating the function calls**\n", - "- For input arguments, you must define the type of the argument, otherwise the registration will fail (e.g. `arg_name: str`). \n", - "- The docstring of the function will be used as the prompt. So make sure to write a clear description.\n", - "- The function name will be used as the tool name.\n" + "\n", + "\n", + "\n" ] }, { @@ -120,9 +182,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Use initialize_swarm_chat\n", + "## Use initiate_swarm_chat\n", "\n", - "To get response from the swarm agent, you can use the `initialize_swarm_chat` function. This function will return the response from the agent and the updated context variables.\n", + "To get response from the swarm agent, you can use the `initiate_swarm_chat` function. This function will return the response from the agent and the updated context variables.\n", "\n", "```python\n", "chat_history, context_variables, last_agent = initialize_swarm_chat(\n", From 10d1c47282aa39db8e1f8db9e65a685a284ed51a Mon Sep 17 00:00:00 2001 From: Mark Sze Date: Wed, 20 Nov 2024 03:37:18 +0000 Subject: [PATCH 05/11] First full blog post --- website/blog/2024-11-17-Swarm/index.mdx | 337 +++++++++++++++++++++++- 1 file changed, 325 insertions(+), 12 deletions(-) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index 33269c9ac..1f82634a8 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -1,3 +1,4 @@ +--- title: Building Swarm-based agents with AG2 authors: - yiranwu @@ -5,29 +6,341 @@ authors: tags: [groupchat, swarm] --- -AG2 now provides an implementation of the swarm orchestration from OpenAI's [Swarm](https://github.com/openai/swarm) framework with some additional features. +AG2 now provides an implementation of the swarm orchestration from OpenAI's [Swarm](https://github.com/openai/swarm) framework, with some additional features! *Background*: the swarm orchestration is a multi-agent collaboration where agents execute tasks and are responsible for handing-off to other agents. -Here are two key features of the swarm orchestration: +Here are the key features of the swarm orchestration: - **Hand offs**: Agents can transfer control to another agent, enabling smooth and direct transitions within workflows. -- **Context Variables**: Agents can dynamically update a shared dictionary through function calls, maintaining context and adaptability throughout the process. +- **Self-execute functions**: Agents execute their own functions +- **Context variables**: Agents can dynamically update a shared memory through function calls, maintaining context and adaptability throughout the process. In addition to the OpenAI implementation, AG2 provides: -- **Built-in human-in-the-loop**: Adding a user agent (UserProxyAgent) to your swarm will allow swarm agents to transition back to the user. Provides a means to clarify and confirm questions without breaking out of the swarm. +- **Built-in human-in-the-loop**: Adding a user agent (UserProxyAgent) to your swarm will allow swarm agents to transition back to the user. Provides a means to clarify and confirm with the user without breaking out of the swarm. - **Fall-back agent**: The ability to automatically transition to a nominated swarm agent when an agent has completed their task. -## Getting Started +## Hand offs + +Before we dive into a swarm example, an important concept in the swarm orchestration is when and how an agent hands off to another agent. + +These are the points for hand offs for each iteration of the swarm. + +1. **Agent-level: Calls a tool that returns a swarm agent**: A swarm agent calls a tool and the associated function returns the next agent to hand off to. +2. **Agent-level: Calls a pre-defined conditional hand off**: A swarm agent has an `ON_CONDITION` hand off that is chosen by the LLM (behaves like a tool call). +3. **Agent-level: After work hand off**: When no tool calls are made it can use an, optional, `AFTER_WORK` hand off that can be a number of preset options or a nominated swarm agent. +4. **Swarm-level: After work hand off**: If the agent does not have an `AFTER_WORK` hand off, the swarm's `AFTER_WORK` hand off will be used. + +In this example of agent-level hand offs registration for a `SwarmAgent` named `responder` has: +- Two conditional hand offs are registered (`ON_CONDITION`), specifying the agent to hand off to and the condition to trigger the hand off. +- An after work hand off (`AFTER_WORK`) is nominated using one of the preset options (`TERMINATE`, `REVERT_TO_USER`, `STAY`). This can also be a swarm agent. + +```python +responder.register_hand_off( + hand_to=[ + ON_CONDITION(weather, "If you need weather data, hand off to the Weather_Agent"), + ON_CONDITION(travel_advisor, "If you have weather data but need formatted recommendations, hand off to the Travel_Advisor_Agent"), + AFTER_WORK(AfterWorkOption.REVERT_TO_USER), + ] +) +``` + +You can specify the swarm-level after work hand off when initiating the swarm (here we terminate): +```python +history, context, last_agent = initiate_swarm_chat( + init_agent=responder, + agents=my_list_of_swarm_agents, + max_rounds=50, + messages=messages, + after_work=AFTER_WORK(AfterWorkOption.TERMINATE) +) +``` + +## Creating a swarm -To create a swarm: -1. Define the functions that can be used by your `SwarmAgent`'s. -2. Create your `SwarmAgent`s (which derive from `ConversableAgent`). -3. For each agent, specify the hand-offs (transitions to another agent) and what to do when they have finished their work (termed *After Work*). +1. Define the functions that can be used by your `SwarmAgent`s. +2. Create your `SwarmAgent`s (which derives from `ConversableAgent`). +3. For each swarm agent, specify the hand offs (transitions to another agent) and what to do when they have finished their work (termed *After Work*). +4. Optionally, create your context dictionary. +5. Call `initiate_swarm_chat`. -UP TO HERE +### Example -The `swarm` speaker selection method needs to be used in conjunction with `SwarmAgent` and `SwarmResult` classes. Use the `initialize_swarm_chat` to get a swarm reply. +This example of managing refunds demonstrates the context handling, swarm and agent-level conditional and after work hand offs, and the human-in-the-loop feature. ```python -from autogen.agentchat.swarm.swarm_agent import SwarmAgent, SwarmResult, initialize_swarm_chat +from autogen.agentchat.contrib.swarm_agent import initiate_swarm_chat, SwarmAgent, SwarmResult, ON_CONDITION, AFTER_WORK, AfterWorkOption +from autogen import UserProxyAgent +import os + +# All our swarm agents will use GPT-4o-mini from OpenAI +llm_config = { + "config_list": + [ + { + "api_type": "openai", + "model": "gpt-4o-mini", + "api_key": os.getenv("OPENAI_API_KEY"), + } + ] +} + +# We configure our starting context dictionary +context_variables = { + "passport_number": "", + "customer_verified": False, + "refund_approved": False, + "payment_processed": False +} + +# Functions that our swarm agents will be assigned + +# They can return a SwarmResult, a SwarmAgent, or a string +# SwarmResult allows you to update context_variables +# and/or hand off to another agent +def verify_customer_identity(passport_number: str, context_variables: dict) -> str: + context_variables["passport_number"] = passport_number + context_variables["customer_verified"] = True + return SwarmResult(values="Customer identity verified", context_variables=context_variables) + +def approve_refund_and_transfer(context_variables: dict) -> str: + context_variables["refund_approved"] = True + return SwarmResult(values="Refund approved", context_variables=context_variables, agent=payment_processor) + +def process_refund_payment(context_variables: dict) -> str: + context_variables["payment_processed"] = True + return SwarmResult(values="Payment processed successfully", context_variables=context_variables) + +# Swarm Agents, similar to ConversableAgent, +# but with functions and hand offs (specified later) + +customer_service = SwarmAgent( + name="CustomerServiceRep", + system_message="""You are a customer service representative. + First verify the customer's identity by asking for the customer's passport number, + then calling the verify_customer_identity function, + finally, transfer the case to the refund specialist.""", + llm_config=llm_config, + functions=[verify_customer_identity], +) + +refund_specialist = SwarmAgent( + name="RefundSpecialist", + system_message="""You are a refund specialist. + Review the case and approve the refund, then transfer to the payment processor.""", + llm_config=llm_config, + functions=[approve_refund_and_transfer], +) + +payment_processor = SwarmAgent( + name="PaymentProcessor", + system_message="""You are a payment processor. + Process the refund payment and provide a confirmation message to the customer.""", + llm_config=llm_config, + functions=[process_refund_payment], +) + +satisfaction_surveyor = SwarmAgent( + name="SatisfactionSurveyor", + system_message="""You are a customer satisfaction specialist. + Ask the customer to rate their experience with the refund process.""", + llm_config=llm_config, +) + +# Conditional and After work hand offs + +customer_service.register_hand_off( + hand_to=[ + ON_CONDITION(refund_specialist, "After customer verification, transfer to refund specialist"), + AFTER_WORK(AfterWorkOption.REVERT_TO_USER) + ] +) + +payment_processor.register_hand_off( + hand_to=[ + AFTER_WORK(satisfaction_surveyor), + ] +) + +# Our human, you, allowing swarm agents to revert back for more information +user = UserProxyAgent( + name="User", + system_message="The customer, always answers questions when agents require more information.", + human_input_mode="ALWAYS", + code_execution_config=False, +) + +# Initiate the swarm +# Returns the ChatResult, final context, and last speaker + +chat_result, context_variables, last_speaker = initiate_swarm_chat( + init_agent=customer_service, # Starting agent + agents=[customer_service, refund_specialist, payment_processor, satisfaction_surveyor], + user_agent=user, # Human user + messages="Customer requesting refund for order #12345", + context_variables=context_variables, # Context + after_work=AFTER_WORK(AfterWorkOption.TERMINATE) # Swarm-level after work hand off +) + + +print(f"Context Variables:\n{json.dumps(context_variables, indent=2)}") ``` + +And here's the output, showing the flow of agents. + +```shell +User (to chat_manager): + +Customer requesting refund for order #12345 + +-------------------------------------------------------------------------------- + +Next speaker: CustomerServiceRep + +CustomerServiceRep (to chat_manager): + +To assist you with the refund for order #12345, +I need to verify your identity. +Could you please provide your passport number? + +-------------------------------------------------------------------------------- + +Next speaker: User + +Replying as User. Provide feedback to chat_manager. +Press enter to skip and use auto-reply, +or type 'exit' to end the conversation: AUS923828C + +User (to chat_manager): + +AUS923828C + +-------------------------------------------------------------------------------- + +Next speaker: CustomerServiceRep + +CustomerServiceRep (to chat_manager): + +***** Suggested tool call (call_wfx2VoCmuCKDFKV5xcmB6kSc): verify_customer_identity ***** +Arguments: +{"passport_number":"AUS923828C"} +***************************************************************************************** + +-------------------------------------------------------------------------------- + +Next speaker: Tool_Execution + + +>>>>>>>> EXECUTING FUNCTION verify_customer_identity... +Tool_Execution (to chat_manager): + +***** Response from calling tool (call_wfx2VoCmuCKDFKV5xcmB6kSc) ***** +Customer identity verified +********************************************************************** + +-------------------------------------------------------------------------------- + +Next speaker: CustomerServiceRep + +CustomerServiceRep (to chat_manager): + +***** Suggested tool call (call_Jz1viRLeJuOltPRcKfYZ8bgH): transfer_to_RefundSpecialist ***** +Arguments: +{} +********************************************************************************************* + +-------------------------------------------------------------------------------- + +Next speaker: Tool_Execution + + +>>>>>>>> EXECUTING FUNCTION transfer_to_RefundSpecialist... +Tool_Execution (to chat_manager): + +***** Response from calling tool (call_Jz1viRLeJuOltPRcKfYZ8bgH) ***** +SwarmAgent --> RefundSpecialist +********************************************************************** + +-------------------------------------------------------------------------------- + +Next speaker: RefundSpecialist + +RefundSpecialist (to chat_manager): + +***** Suggested tool call (call_c4uhy8Mi3Ihe3tVRFVG9M8Uw): approve_refund_and_transfer ***** +Arguments: +{} +******************************************************************************************** + +-------------------------------------------------------------------------------- + +Next speaker: Tool_Execution + + +>>>>>>>> EXECUTING FUNCTION approve_refund_and_transfer... +Tool_Execution (to chat_manager): + +***** Response from calling tool (call_c4uhy8Mi3Ihe3tVRFVG9M8Uw) ***** +Refund approved +********************************************************************** + +-------------------------------------------------------------------------------- + +Next speaker: PaymentProcessor + +PaymentProcessor (to chat_manager): + +***** Suggested tool call (call_qrs5ysx89rQqMFtVuqMUb9O0): process_refund_payment ***** +Arguments: +{} +*************************************************************************************** + +-------------------------------------------------------------------------------- + +Next speaker: Tool_Execution + + +>>>>>>>> EXECUTING FUNCTION process_refund_payment... +Tool_Execution (to chat_manager): + +***** Response from calling tool (call_qrs5ysx89rQqMFtVuqMUb9O0) ***** +Payment processed successfully +********************************************************************** + +-------------------------------------------------------------------------------- + +Next speaker: PaymentProcessor + +PaymentProcessor (to chat_manager): + +Your refund for order #12345 has been processed successfully. +You should see the amount credited back to your account shortly. +If you have any further questions or need assistance, feel free to reach out. +Thank you for your patience! + +-------------------------------------------------------------------------------- + +Next speaker: SatisfactionSurveyor + +SatisfactionSurveyor (to chat_manager): + +Thank you for your patience during the refund process! +We hope you are satisfied with your experience. +Could you please rate your satisfaction with the refund process from 1 to 5, +where 1 is very dissatisfied and 5 is very satisfied? +Your feedback is valuable to us! + +-------------------------------------------------------------------------------- +Context Variables: +{ + "customer_verified": true, + "refund_approved": true, + "payment_processed": true, + "passport_number": "AUS923828C" +} +``` + +### Notes + +- Behind-the-scenes, swarm agents are supported by a Tool Executor agent, which executes the tools on their behalf. Hence, the appearance of `Tool Execution` in the output. +- Only swarm agents can be added to a swarm. This is to maintain their ability to manage context variables, self-execute functions, and support hand offs. +- Would like to enhance the swarm feature or have found a bug? Please let us know by creating an issue on the [AG2 GitHub](https://github.com/ag2ai/ag2/issues). From ff1621032316a212a3a7b548e443401023a22ffe Mon Sep 17 00:00:00 2001 From: Mark Sze Date: Wed, 20 Nov 2024 03:48:24 +0000 Subject: [PATCH 06/11] Tweaks to blog --- website/blog/2024-11-17-Swarm/index.mdx | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index 1f82634a8..71e54da48 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -23,16 +23,18 @@ In addition to the OpenAI implementation, AG2 provides: Before we dive into a swarm example, an important concept in the swarm orchestration is when and how an agent hands off to another agent. -These are the points for hand offs for each iteration of the swarm. +Providing additional flexibility, we introduce the capability to define an after work hand off. Think of *after work* as the agent's next action after completing their task. It can be to hand off to another agent, revert to the user, stay with the agent for another iteration, or terminate the conversation. -1. **Agent-level: Calls a tool that returns a swarm agent**: A swarm agent calls a tool and the associated function returns the next agent to hand off to. +The following are the prioritised hand offs for each iteration of the swarm. + +1. **Agent-level: Calls a tool that returns a swarm agent**: A swarm agent's tool call returns the next agent to hand off to. 2. **Agent-level: Calls a pre-defined conditional hand off**: A swarm agent has an `ON_CONDITION` hand off that is chosen by the LLM (behaves like a tool call). -3. **Agent-level: After work hand off**: When no tool calls are made it can use an, optional, `AFTER_WORK` hand off that can be a number of preset options or a nominated swarm agent. +3. **Agent-level: After work hand off**: When no tool calls are made it can use an, optional, `AFTER_WORK` hand off that is a preset option or a nominated swarm agent. 4. **Swarm-level: After work hand off**: If the agent does not have an `AFTER_WORK` hand off, the swarm's `AFTER_WORK` hand off will be used. In this example of agent-level hand offs registration for a `SwarmAgent` named `responder` has: - Two conditional hand offs are registered (`ON_CONDITION`), specifying the agent to hand off to and the condition to trigger the hand off. -- An after work hand off (`AFTER_WORK`) is nominated using one of the preset options (`TERMINATE`, `REVERT_TO_USER`, `STAY`). This can also be a swarm agent. +- An after work hand off (`AFTER_WORK`) is nominated using one of the preset options (`TERMINATE`, `REVERT_TO_USER`, `STAY`). This could also be a swarm agent. ```python responder.register_hand_off( From d15b3190ca6343d6b756a8ffa3ef178759dab622 Mon Sep 17 00:00:00 2001 From: Mark Sze Date: Wed, 20 Nov 2024 03:51:07 +0000 Subject: [PATCH 07/11] Update tool execution note in blog --- website/blog/2024-11-17-Swarm/index.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index 71e54da48..13ed6da06 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -343,6 +343,6 @@ Context Variables: ### Notes -- Behind-the-scenes, swarm agents are supported by a Tool Executor agent, which executes the tools on their behalf. Hence, the appearance of `Tool Execution` in the output. +- Behind-the-scenes, swarm agents are supported by a tool execution agent, that executes tools on their behalf. Hence, the appearance of `Tool Execution` in the output. - Only swarm agents can be added to a swarm. This is to maintain their ability to manage context variables, self-execute functions, and support hand offs. -- Would like to enhance the swarm feature or have found a bug? Please let us know by creating an issue on the [AG2 GitHub](https://github.com/ag2ai/ag2/issues). +- Would you like to enhance the swarm feature or have found a bug? Please let us know by creating an issue on the [AG2 GitHub](https://github.com/ag2ai/ag2/issues). From 5663a83fc8675851e034df8fa69cd2f2a1ec131d Mon Sep 17 00:00:00 2001 From: Mark Sze Date: Wed, 20 Nov 2024 03:54:58 +0000 Subject: [PATCH 08/11] Grammar --- website/blog/2024-11-17-Swarm/index.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index 13ed6da06..16ab96efe 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -32,9 +32,9 @@ The following are the prioritised hand offs for each iteration of the swarm. 3. **Agent-level: After work hand off**: When no tool calls are made it can use an, optional, `AFTER_WORK` hand off that is a preset option or a nominated swarm agent. 4. **Swarm-level: After work hand off**: If the agent does not have an `AFTER_WORK` hand off, the swarm's `AFTER_WORK` hand off will be used. -In this example of agent-level hand offs registration for a `SwarmAgent` named `responder` has: -- Two conditional hand offs are registered (`ON_CONDITION`), specifying the agent to hand off to and the condition to trigger the hand off. -- An after work hand off (`AFTER_WORK`) is nominated using one of the preset options (`TERMINATE`, `REVERT_TO_USER`, `STAY`). This could also be a swarm agent. +In the following code sample a `SwarmAgent` named `responder` has: +- Two conditional hand offs registered (`ON_CONDITION`), specifying the agent to hand off to and the condition to trigger the hand off. +- An after work hand off (`AFTER_WORK`) nominated using one of the preset options (`TERMINATE`, `REVERT_TO_USER`, `STAY`). This could also be a swarm agent. ```python responder.register_hand_off( @@ -46,12 +46,12 @@ responder.register_hand_off( ) ``` -You can specify the swarm-level after work hand off when initiating the swarm (here we terminate): +You can specify the swarm-level after work hand off when initiating the swarm (here we nominate to terminate): ```python history, context, last_agent = initiate_swarm_chat( init_agent=responder, agents=my_list_of_swarm_agents, - max_rounds=50, + max_rounds=30, messages=messages, after_work=AFTER_WORK(AfterWorkOption.TERMINATE) ) From 676860cc5ef86b0507f690db27d28941ab367009 Mon Sep 17 00:00:00 2001 From: Yiran Wu <32823396+kevin666aa@users.noreply.github.com> Date: Wed, 20 Nov 2024 14:01:48 -0500 Subject: [PATCH 09/11] update --- website/blog/2024-11-17-Swarm/index.mdx | 31 +++++++++---------------- website/docs/topics/swarm.ipynb | 20 +++++++++------- 2 files changed, 23 insertions(+), 28 deletions(-) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index 16ab96efe..cce3a1a7c 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -12,12 +12,12 @@ AG2 now provides an implementation of the swarm orchestration from OpenAI's [Swa Here are the key features of the swarm orchestration: - **Hand offs**: Agents can transfer control to another agent, enabling smooth and direct transitions within workflows. -- **Self-execute functions**: Agents execute their own functions - **Context variables**: Agents can dynamically update a shared memory through function calls, maintaining context and adaptability throughout the process. -In addition to the OpenAI implementation, AG2 provides: +Beside these core features, AG2 provides: +- **Simple Interface for Tool Call/Handoff Registeration**: When creating a swarm agent, you can pass in a list of functions to be registered directly. We also provide a simple method to register handoffs. +- **Transition Beyond Tool Calls**: We enable the ability to automatically transfer to a nominated swarm agent when an agent has completed their task. In a future release, we may extend this to allo passing in a callable function to determine the next agent to hand off to. - **Built-in human-in-the-loop**: Adding a user agent (UserProxyAgent) to your swarm will allow swarm agents to transition back to the user. Provides a means to clarify and confirm with the user without breaking out of the swarm. -- **Fall-back agent**: The ability to automatically transition to a nominated swarm agent when an agent has completed their task. ## Hand offs @@ -41,7 +41,7 @@ responder.register_hand_off( hand_to=[ ON_CONDITION(weather, "If you need weather data, hand off to the Weather_Agent"), ON_CONDITION(travel_advisor, "If you have weather data but need formatted recommendations, hand off to the Travel_Advisor_Agent"), - AFTER_WORK(AfterWorkOption.REVERT_TO_USER), + AFTER_WORK(REVERT_TO_USER), ] ) ``` @@ -53,7 +53,7 @@ history, context, last_agent = initiate_swarm_chat( agents=my_list_of_swarm_agents, max_rounds=30, messages=messages, - after_work=AFTER_WORK(AfterWorkOption.TERMINATE) + after_work=AFTER_WORK(TERMINATE) ) ``` @@ -70,21 +70,13 @@ history, context, last_agent = initiate_swarm_chat( This example of managing refunds demonstrates the context handling, swarm and agent-level conditional and after work hand offs, and the human-in-the-loop feature. ```python -from autogen.agentchat.contrib.swarm_agent import initiate_swarm_chat, SwarmAgent, SwarmResult, ON_CONDITION, AFTER_WORK, AfterWorkOption +from autogen.agentchat.contrib.swarm_agent import initiate_swarm_chat, SwarmAgent, SwarmResult +from autogen.agentchat.contrib.swarm_agent import ON_CONDITION, AFTER_WORK, REVERT_TO_USER, TERMINATE from autogen import UserProxyAgent import os # All our swarm agents will use GPT-4o-mini from OpenAI -llm_config = { - "config_list": - [ - { - "api_type": "openai", - "model": "gpt-4o-mini", - "api_key": os.getenv("OPENAI_API_KEY"), - } - ] -} +llm_config = {...} # We configure our starting context dictionary context_variables = { @@ -95,7 +87,6 @@ context_variables = { } # Functions that our swarm agents will be assigned - # They can return a SwarmResult, a SwarmAgent, or a string # SwarmResult allows you to update context_variables # and/or hand off to another agent @@ -153,7 +144,7 @@ satisfaction_surveyor = SwarmAgent( customer_service.register_hand_off( hand_to=[ ON_CONDITION(refund_specialist, "After customer verification, transfer to refund specialist"), - AFTER_WORK(AfterWorkOption.REVERT_TO_USER) + AFTER_WORK(REVERT_TO_USER) ] ) @@ -180,7 +171,7 @@ chat_result, context_variables, last_speaker = initiate_swarm_chat( user_agent=user, # Human user messages="Customer requesting refund for order #12345", context_variables=context_variables, # Context - after_work=AFTER_WORK(AfterWorkOption.TERMINATE) # Swarm-level after work hand off + after_work=AFTER_WORK(TERMINATE) # Swarm-level after work hand off ) @@ -344,5 +335,5 @@ Context Variables: ### Notes - Behind-the-scenes, swarm agents are supported by a tool execution agent, that executes tools on their behalf. Hence, the appearance of `Tool Execution` in the output. -- Only swarm agents can be added to a swarm. This is to maintain their ability to manage context variables, self-execute functions, and support hand offs. +- Currently only swarm agents can be added to a swarm. This is to maintain their ability to manage context variables, auto-execute functions, and support hand offs. Eventually, we may allow ConversableAgent to have the same capability and make "SwarmAgent" a simpler subclass with certain defaults changed (like AssistantAgent and UserProxyAgent). - Would you like to enhance the swarm feature or have found a bug? Please let us know by creating an issue on the [AG2 GitHub](https://github.com/ag2ai/ag2/issues). diff --git a/website/docs/topics/swarm.ipynb b/website/docs/topics/swarm.ipynb index c6d5e1674..6469bc6d1 100644 --- a/website/docs/topics/swarm.ipynb +++ b/website/docs/topics/swarm.ipynb @@ -19,13 +19,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We now introduce the main components that need to be used to create a swarm chat.\n", + "## Components\n", + "We now introduce the main components that need to be used to create a swarm chat. \n", "\n", - "## Create `SwarmAgent`\n", - "All the agents passed to the swarm chat should be instances of this class. `SwarmAgent` is very similar to `AssistantAgent`, but it has some additional features to support function registration and handoffs.\n", + "### Create a `SwarmAgent`\n", "\n", - "### Initialization\n", - "When creating a `SwarmAgent`, you can pass in a list of functions. These functions will be converted to schemas to be passed to the LLMs, and you don't need to worry about registering the functions for execution. You can also pass back a `SwarmResult` class, where you can return a value, the next agent to call and update context variables at the same time.\n", + "All the agents passed to the swarm chat should be instances of `SwarmAgent`. `SwarmAgent` is very similar to `AssistantAgent`, but it has some additional features to support function registration and handoffs. When creating a `SwarmAgent`, you can pass in a list of functions. These functions will be converted to schemas to be passed to the LLMs, and you don't need to worry about registering the functions for execution. You can also pass back a `SwarmResult` class, where you can return a value, the next agent to call and update context variables at the same time.\n", "\n", "**Notes for creating the function calls**\n", "- For input arguments, you must define the type of the argument, otherwise the registration will fail (e.g. `arg_name: str`). \n", @@ -99,7 +98,7 @@ "agent_2.handoff(hand_to=[AFTER_WORK(TERMINATE)]) # Terminate the chat if no handoff is suggested\n", "```\n", "\n", - "## Initialize SwarmChat with `initiate_swarm_chat`\n", + "### Initialize SwarmChat with `initiate_swarm_chat`\n", "\n", "After a set of swarm agents are created, you can initiate a swarm chat by calling `initiate_swarm_chat`.\n", "\n", @@ -113,10 +112,15 @@ ")\n", "```\n", "\n", - "### How is context variables updated\n", + "## Q&As\n", "\n", - "### When to pass in a user agent\n", + "> How are context variables updated?\n", "\n", + "The context variables will only be updated through custom function calls when returning a `SwarmResult` object. In fact, all interactions with context variables will be done through function calls (accessing and updating). The context variables dictionary is a reference, and any modification will be done in-place.\n", + "\n", + "> When to pass in a user agent?\n", + "\n", + "If your application requires interactions with the user, you can pass in a user agent to the groupchat, so that don't need to write an outer loop to accept user inputs and calling swarm.\n", "\n", "\n", "\n" From 968960b0c8f8e43cfa8c66aa1119d85a03456b1d Mon Sep 17 00:00:00 2001 From: Yiran Wu <32823396+kevin666aa@users.noreply.github.com> Date: Wed, 20 Nov 2024 16:04:40 -0500 Subject: [PATCH 10/11] update --- website/blog/2024-11-17-Swarm/index.mdx | 63 ++- website/docs/topics/swarm.ipynb | 502 ++++++++++++++++++++---- 2 files changed, 449 insertions(+), 116 deletions(-) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index cce3a1a7c..6cb9b85de 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -8,52 +8,52 @@ tags: [groupchat, swarm] AG2 now provides an implementation of the swarm orchestration from OpenAI's [Swarm](https://github.com/openai/swarm) framework, with some additional features! -*Background*: the swarm orchestration is a multi-agent collaboration where agents execute tasks and are responsible for handing-off to other agents. +*Background*: the swarm orchestration is a multi-agent collaboration where agents execute tasks and are responsible for handing them off to other agents. Here are the key features of the swarm orchestration: -- **Hand offs**: Agents can transfer control to another agent, enabling smooth and direct transitions within workflows. +- **Hand-offs**: Agents can transfer control to another agent, enabling smooth and direct transitions within workflows. - **Context variables**: Agents can dynamically update a shared memory through function calls, maintaining context and adaptability throughout the process. -Beside these core features, AG2 provides: -- **Simple Interface for Tool Call/Handoff Registeration**: When creating a swarm agent, you can pass in a list of functions to be registered directly. We also provide a simple method to register handoffs. -- **Transition Beyond Tool Calls**: We enable the ability to automatically transfer to a nominated swarm agent when an agent has completed their task. In a future release, we may extend this to allo passing in a callable function to determine the next agent to hand off to. +Besides these core features, AG2 provides: +- **Simple Interface for Tool Call/Handoff Registration**: When creating a swarm agent, you can pass in a list of functions to be registered directly. We also provide a simple method to register handoffs. +- **Transition Beyond Tool Calls**: We enable the ability to automatically transfer to a nominated swarm agent when an agent has completed their task. We will extend this to allow other transitions in the future (e.g., use a function to determine the next agent ). - **Built-in human-in-the-loop**: Adding a user agent (UserProxyAgent) to your swarm will allow swarm agents to transition back to the user. Provides a means to clarify and confirm with the user without breaking out of the swarm. -## Hand offs +## Handoffs -Before we dive into a swarm example, an important concept in the swarm orchestration is when and how an agent hands off to another agent. +Before we dive into a swarm example, an important concept in swarm orchestration is when and how an agent hands off to another agent. -Providing additional flexibility, we introduce the capability to define an after work hand off. Think of *after work* as the agent's next action after completing their task. It can be to hand off to another agent, revert to the user, stay with the agent for another iteration, or terminate the conversation. +Providing additional flexibility, we introduce the capability to define an after-work handoff. Think of it as the agent's next action after completing their task. It can be to hand off to another agent, revert to the user, stay with the agent for another iteration, or terminate the conversation. -The following are the prioritised hand offs for each iteration of the swarm. +The following are the prioritized handoffs for each iteration of the swarm. 1. **Agent-level: Calls a tool that returns a swarm agent**: A swarm agent's tool call returns the next agent to hand off to. -2. **Agent-level: Calls a pre-defined conditional hand off**: A swarm agent has an `ON_CONDITION` hand off that is chosen by the LLM (behaves like a tool call). -3. **Agent-level: After work hand off**: When no tool calls are made it can use an, optional, `AFTER_WORK` hand off that is a preset option or a nominated swarm agent. -4. **Swarm-level: After work hand off**: If the agent does not have an `AFTER_WORK` hand off, the swarm's `AFTER_WORK` hand off will be used. +2. **Agent-level: Calls a pre-defined conditional handoff**: A swarm agent has an `ON_CONDITION` handoff that is chosen by the LLM (behaves like a tool call). +3. **Agent-level: After work hand off**: When no tool calls are made it can use an, optional, `AFTER_WORK` handoff that is a preset option or a nominated swarm agent. +4. **Swarm-level: After work handoff**: If the agent does not have an `AFTER_WORK` handoff, the swarm's `AFTER_WORK` handoff will be used. In the following code sample a `SwarmAgent` named `responder` has: -- Two conditional hand offs registered (`ON_CONDITION`), specifying the agent to hand off to and the condition to trigger the hand off. -- An after work hand off (`AFTER_WORK`) nominated using one of the preset options (`TERMINATE`, `REVERT_TO_USER`, `STAY`). This could also be a swarm agent. +- Two conditional handoffs registered (`ON_CONDITION`), specifying the agent to hand off to and the condition to trigger the handoff. +- An after-work handoff (`AFTER_WORK`) nominated using one of the preset options (`TERMINATE`, `REVERT_TO_USER`, `STAY`). This could also be a swarm agent. ```python responder.register_hand_off( hand_to=[ - ON_CONDITION(weather, "If you need weather data, hand off to the Weather_Agent"), - ON_CONDITION(travel_advisor, "If you have weather data but need formatted recommendations, hand off to the Travel_Advisor_Agent"), - AFTER_WORK(REVERT_TO_USER), - ] + ON_CONDITION(weather, "If you need weather data, hand off to the Weather_Agent"), + ON_CONDITION(travel_advisor, "If you have weather data but need formatted recommendations, hand off to the Travel_Advisor_Agent"), + AFTER_WORK(AfterWorkOption.REVERT_TO_USER), + ] ) ``` -You can specify the swarm-level after work hand off when initiating the swarm (here we nominate to terminate): +You can specify the swarm-level after work handoff when initiating the swarm (here we nominate to terminate): ```python history, context, last_agent = initiate_swarm_chat( init_agent=responder, agents=my_list_of_swarm_agents, max_rounds=30, messages=messages, - after_work=AFTER_WORK(TERMINATE) + after_work=AFTER_WORK(AfterWorkOption.TERMINATE) ) ``` @@ -61,7 +61,7 @@ history, context, last_agent = initiate_swarm_chat( 1. Define the functions that can be used by your `SwarmAgent`s. 2. Create your `SwarmAgent`s (which derives from `ConversableAgent`). -3. For each swarm agent, specify the hand offs (transitions to another agent) and what to do when they have finished their work (termed *After Work*). +3. For each swarm agent, specify the handoffs (transitions to another agent) and what to do when they have finished their work (termed *After Work*). 4. Optionally, create your context dictionary. 5. Call `initiate_swarm_chat`. @@ -70,8 +70,7 @@ history, context, last_agent = initiate_swarm_chat( This example of managing refunds demonstrates the context handling, swarm and agent-level conditional and after work hand offs, and the human-in-the-loop feature. ```python -from autogen.agentchat.contrib.swarm_agent import initiate_swarm_chat, SwarmAgent, SwarmResult -from autogen.agentchat.contrib.swarm_agent import ON_CONDITION, AFTER_WORK, REVERT_TO_USER, TERMINATE +from autogen.agentchat.contrib.swarm_agent import initiate_swarm_chat, SwarmAgent, SwarmResult, ON_CONDITION, AFTER_WORK, AfterWorkOption from autogen import UserProxyAgent import os @@ -88,8 +87,7 @@ context_variables = { # Functions that our swarm agents will be assigned # They can return a SwarmResult, a SwarmAgent, or a string -# SwarmResult allows you to update context_variables -# and/or hand off to another agent +# SwarmResult allows you to update context_variables and/or hand off to another agent def verify_customer_identity(passport_number: str, context_variables: dict) -> str: context_variables["passport_number"] = passport_number context_variables["customer_verified"] = True @@ -103,9 +101,7 @@ def process_refund_payment(context_variables: dict) -> str: context_variables["payment_processed"] = True return SwarmResult(values="Payment processed successfully", context_variables=context_variables) -# Swarm Agents, similar to ConversableAgent, -# but with functions and hand offs (specified later) - +# Swarm Agents, similar to ConversableAgent, but with functions and hand offs (specified later) customer_service = SwarmAgent( name="CustomerServiceRep", system_message="""You are a customer service representative. @@ -144,7 +140,7 @@ satisfaction_surveyor = SwarmAgent( customer_service.register_hand_off( hand_to=[ ON_CONDITION(refund_specialist, "After customer verification, transfer to refund specialist"), - AFTER_WORK(REVERT_TO_USER) + AFTER_WORK(AfterWorkOption.REVERT_TO_USER) ] ) @@ -155,12 +151,7 @@ payment_processor.register_hand_off( ) # Our human, you, allowing swarm agents to revert back for more information -user = UserProxyAgent( - name="User", - system_message="The customer, always answers questions when agents require more information.", - human_input_mode="ALWAYS", - code_execution_config=False, -) +user = UserProxyAgent(name="User", code_execution_config=False) # Initiate the swarm # Returns the ChatResult, final context, and last speaker @@ -171,7 +162,7 @@ chat_result, context_variables, last_speaker = initiate_swarm_chat( user_agent=user, # Human user messages="Customer requesting refund for order #12345", context_variables=context_variables, # Context - after_work=AFTER_WORK(TERMINATE) # Swarm-level after work hand off + after_work=AFTER_WORK(AfterWorkOption.TERMINATE) # Swarm-level after work hand off ) diff --git a/website/docs/topics/swarm.ipynb b/website/docs/topics/swarm.ipynb index 6469bc6d1..f960144bf 100644 --- a/website/docs/topics/swarm.ipynb +++ b/website/docs/topics/swarm.ipynb @@ -6,13 +6,12 @@ "source": [ "# Swarm Ochestration\n", "\n", - "With AG2, you can initiate a Swarm Chat similar to OpenAI's [Swarm](https://github.com/openai/swarm). In this orchestration, there are two main features:\n", - "- **Headoffs**: Agents can transfer control to another agent via function calls, enabling smooth transitions within workflows.\n", - "- **Context Variables**: Agents can dynamically update shared variables through function calls, maintaining context and adaptability throughout the process.\n", + "With AG2, you can initiate a Swarm Chat similar to OpenAI's [Swarm](https://github.com/openai/swarm). This orchestration offers two main features:\n", "\n", + "- **Headoffs**: Agents can transfer control to another agent via function calls, enabling smooth transitions within workflows. \n", + "- **Context Variables**: Agents can dynamically update shared variables through function calls, maintaining context and adaptability throughout the process.\n", "\n", - "Instead of sending a task to a sinlge LLM agent, you can send it to a swarm of agents. Each agent in the swarm can decide whether to handoff the task to another agent. \n", - "When the last active agent's response is only a string (it doesn't suggest tool call or handoff), the chat will be terminated." + "Instead of sending a task to a single LLM agent, you can assign it to a swarm of agents. Each agent in the swarm can decide whether to hand off the task to another agent. The chat terminates when the last active agent's response is a plain string (i.e., it doesn't suggest a tool call or handoff). " ] }, { @@ -24,35 +23,16 @@ "\n", "### Create a `SwarmAgent`\n", "\n", - "All the agents passed to the swarm chat should be instances of `SwarmAgent`. `SwarmAgent` is very similar to `AssistantAgent`, but it has some additional features to support function registration and handoffs. When creating a `SwarmAgent`, you can pass in a list of functions. These functions will be converted to schemas to be passed to the LLMs, and you don't need to worry about registering the functions for execution. You can also pass back a `SwarmResult` class, where you can return a value, the next agent to call and update context variables at the same time.\n", + "All the agents passed to the swarm chat should be instances of `SwarmAgent`. `SwarmAgent` is very similar to `AssistantAgent`, but it has some additional features to support function registration and handoffs. When creating a `SwarmAgent`, you can pass in a list of functions. These functions will be converted to schemas to be passed to the LLMs, and you don't need to worry about registering the functions for execution. You can also pass back a `SwarmResult` class, where you can return a value, the next agent to call, and update context variables at the same time.\n", "\n", - "**Notes for creating the function calls**\n", - "- For input arguments, you must define the type of the argument, otherwise the registration will fail (e.g. `arg_name: str`). \n", - "- If your function requires access or modification of the context variables, you must pass in `context_variables: dict` as one argument. This argument will not be visible to the LLM (removed when registering the function schema). But when being called, the global context variables will be passed in by the swarm chat.\n", - "- The docstring of the function will be used as the prompt. So make sure to write a clear description.\n", + "**Notes for creating the function calls** \n", + "- For input arguments, you must define the type of the argument, otherwise, the registration will fail (e.g. `arg_name: str`). \n", + "- If your function requires access or modification of the context variables, you must pass in `context_variables: dict` as one argument. This argument will not be visible to the LLM (removed when registering the function schema). But when called, the global context variables will be passed in by the swarm chat. \n", + "- The docstring of the function will be used as the prompt. So make sure to write a clear description. \n", "- The function name will be used as the tool name.\n", "\n", - "```python\n", - "def func_1(arg1: str, arg2: int) -> str:\n", - " \"\"\"Doc String\"\"\"\n", - " return \"response\"\n", - "\n", - "def transfer_to_some_agent() -> SwarmAgent:\n", - " return SwarmAgent(...)\n", - "\n", - "def func_3(context_variables: dict, arg2: str) -> SwarmResult:\n", - " # ... update context variables\n", - " context_variables[\"key\"] = \"value\"\n", - " return SwarmResult(value=\"value\", agent=, context_variables=context_variables)\n", - "\n", - "SwarmAgent(\n", - " ..., # same arguments as AssistantAgent\n", - " functions=[func_1, transfer_to_some_agent, func_3] # a new parameter that allows passing a list of functions\n", - ")\n", - "```\n", - "\n", "### Registering Handoffs\n", - "While you can create an function to decide what next agent to call, we provide a quick way to register the handoff use `ON_CONDITION`. We will craft this transition function and add it to the LLM config directly.\n", + "While you can create a function to decide what next agent to call, we provide a quick way to register the handoff using `ON_CONDITION`. We will craft this transition function and add it to the LLM config directly.\n", "\n", "```python\n", "agent_2 = SwarmAgent(...)\n", @@ -78,24 +58,24 @@ "### AFTER_WORK\n", "\n", "When the last active agent's response doesn't suggest a tool call or handoff, the chat will terminate by default. However, you can register an `AFTER_WORK` handoff to define a fallback agent if you don't want the chat to end at this agent. At the swarm chat level, you also pass in an `AFTER_WORK` handoff to define the fallback mechanism for the entire chat.\n", - "If this parameter is set for both the agent and the chat, we will prioritize the agent's setting.\n", + "If this parameter is set for the agent and the chat, we will prioritize the agent's setting. There should only be one `AFTER_WORK`. If multiple `AFTER_WORK` handoffs are passed, only the last one will be used.\n", "\n", - "Besides fallback to an agent, we provide 3 `AfterWorkOption`:\n", - "- `TERMINATE`: Terminate the chat\n", - "- `STAY`: Stay at the current agent\n", - "- `REVERT_TO_USER`: Revert to the user agent. Only if an user agent is passed in when initializing. (See below for more details)\n", + "Besides fallback to an agent, we provide 3 `AfterWorkOption`: \n", + "- `TERMINATE`: Terminate the chat \n", + "- `STAY`: Stay at the current agent \n", + "- `REVERT_TO_USER`: Revert to the user agent. Only if a user agent is passed in when initializing. (See below for more details) \n", "\n", "```python\n", "agent_1 = SwarmAgent(...)\n", "\n", "# Register the handoff\n", "agent_1.handoff(hand_to=[\n", - " ON_CONDITION(...), \n", - " ON_CONDITION(...),\n", - " AFTER_WORK(agent_4) # Fallback to agent_4 if no handoff is suggested\n", + " ON_CONDITION(...), \n", + " ON_CONDITION(...),\n", + " AFTER_WORK(agent_4) # Fallback to agent_4 if no handoff is suggested\n", "])\n", "\n", - "agent_2.handoff(hand_to=[AFTER_WORK(TERMINATE)]) # Terminate the chat if no handoff is suggested\n", + "agent_2.handoff(hand_to=[AFTER_WORK(AfterWorkOption.TERMINATE)]) # Terminate the chat if no handoff is suggested\n", "```\n", "\n", "### Initialize SwarmChat with `initiate_swarm_chat`\n", @@ -107,7 +87,7 @@ " init_agent=agent_1, # the first agent to start the chat\n", " agents=[agent_1, agent_2, agent_3], # a list of agents\n", " messages=[{\"role\": \"user\", \"content\": \"Hello\"}], # a list of messages to start the chat\n", - " user_agent=user_agent, # optional, if you want to revert to user agent\n", + " user_agent=user_agent, # optional, if you want to revert to the user agent\n", " context_variables={\"key\": \"value\"} # optional, initial context variables\n", ")\n", "```\n", @@ -116,14 +96,23 @@ "\n", "> How are context variables updated?\n", "\n", - "The context variables will only be updated through custom function calls when returning a `SwarmResult` object. In fact, all interactions with context variables will be done through function calls (accessing and updating). The context variables dictionary is a reference, and any modification will be done in-place.\n", + "The context variables will only be updated through custom function calls when returning a `SwarmResult` object. In fact, all interactions with context variables will be done through function calls (accessing and updating). The context variables dictionary is a reference, and any modification will be done in place.\n", + "\n", + "> What is the difference between ON_CONDITION and AFTER_WORK?\n", + "\n", + "When registering an ON_CONDITION handoff, we are creating a function schema to be passed to the LLM. The LLM will decide whether to call this function.\n", + "\n", + "When registering an AFTER_WORK handoff, we are defining the fallback mechanism when no tool calls are suggested. This is a higher level of control from the swarm chat level.\n", "\n", "> When to pass in a user agent?\n", "\n", - "If your application requires interactions with the user, you can pass in a user agent to the groupchat, so that don't need to write an outer loop to accept user inputs and calling swarm.\n", + "If your application requires interactions with the user, you can pass in a user agent to the groupchat, so that don't need to write an outer loop to accept user inputs and call swarm.\n", + "\n", "\n", + "## Demonstration\n", "\n", - "\n" + "\n", + "### Create Swarm Agents" ] }, { @@ -132,37 +121,75 @@ "metadata": {}, "outputs": [], "source": [ - "from autogen.agentchat.contrib.swarm_agent import SwarmAgent, SwarmResult\n", + "import autogen\n", "\n", - "llm_config = \"Setup your config here\"\n", + "config_list = autogen.config_list_from_json(...)\n", + "llm_config = {\"config_list\": config_list}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Agent 1 function schema:\n", + "{'type': 'function', 'function': {'description': '', 'name': 'update_context_1', 'parameters': {'type': 'object', 'properties': {}}}}\n", + "{'type': 'function', 'function': {'description': 'Transfer to agent 2', 'name': 'transfer_to_agent_2', 'parameters': {'type': 'object', 'properties': {}}}}\n", + "Agent 3 function schema:\n", + "{'type': 'function', 'function': {'description': 'Transfer to Agent 4', 'name': 'transfer_to_Agent_4', 'parameters': {'type': 'object', 'properties': {}}}}\n" + ] + } + ], + "source": [ + "import random\n", + "\n", + "from autogen.agentchat.contrib.swarm_agent import (\n", + " AFTER_WORK,\n", + " ON_CONDITION,\n", + " AfterWorkOption,\n", + " SwarmAgent,\n", + " SwarmResult,\n", + " initiate_swarm_chat,\n", + ")\n", "\n", "\n", + "# 1. A function that returns a value of \"success\" and updates the context variable \"1\" to True\n", "def update_context_1(context_variables: dict) -> str:\n", " context_variables[\"1\"] = True\n", - " return SwarmResult(\n", - " value=\"success\", context_variables=context_variables\n", - " ) # Update context variables and return success\n", + " return SwarmResult(value=\"success\", context_variables=context_variables)\n", + "\n", + "\n", + "# 2. A function that returns an SwarmAgent object\n", + "def transfer_to_agent_2() -> SwarmAgent:\n", + " \"\"\"Transfer to agent 2\"\"\"\n", + " return agent_2\n", "\n", "\n", + "# 3. A function that returns the value of \"success\", updates the context variable and transfers to agent 3\n", "def update_context_2_and_transfer_to_3(context_variables: dict) -> str:\n", " context_variables[\"2\"] = True\n", - " return SwarmResult(\n", - " value=\"success\", context_variables=context_variables, agent=agent_3\n", - " ) # Update context variables, return success, and transfer to agent_3\n", + " return SwarmResult(value=\"success\", context_variables=context_variables, agent=agent_3)\n", + "\n", + "\n", + "# 4. A function that returns a normal value\n", + "def get_random_number() -> str:\n", + " return random.randint(1, 100)\n", "\n", "\n", - "def update_context_3(context_variables: dict) -> str:\n", - " context_variables[\"3\"] = True\n", - " return SwarmResult(\n", - " value=\"success\", context_variables=context_variables\n", - " ) # Update context variables and return success\n", + "def update_context_3_with_random_number(context_variables: dict, random_number: int) -> str:\n", + " context_variables[\"3\"] = random_number\n", + " return SwarmResult(value=\"success\", context_variables=context_variables)\n", "\n", "\n", "agent_1 = SwarmAgent(\n", " name=\"Agent_1\",\n", " system_message=\"You are Agent 1, first, call the function to update context 1, and transfer to Agent 2\",\n", " llm_config=llm_config,\n", - " functions=[update_context_1],\n", + " functions=[update_context_1, transfer_to_agent_2],\n", ")\n", "\n", "agent_2 = SwarmAgent(\n", @@ -174,50 +201,365 @@ "\n", "agent_3 = SwarmAgent(\n", " name=\"Agent_3\",\n", - " system_message=\"You are Agent 3, first, call the function to update context 3, and then reply TERMINATE\",\n", + " system_message=\"You are Agent 3, tell a joke\",\n", " llm_config=llm_config,\n", - " functions=[update_context_3],\n", ")\n", "\n", - "agent_1.hand_off(agent_2, condition=\"handoff to agent 2\") # register handoff using hand_off method." + "agent_4 = SwarmAgent(\n", + " name=\"Agent_4\",\n", + " system_message=\"You are Agent 4, call the function to get a random number\",\n", + " llm_config=llm_config,\n", + " functions=[get_random_number],\n", + ")\n", + "\n", + "agent_5 = SwarmAgent(\n", + " name=\"Agent_5\",\n", + " system_message=\"Update context 3 with the random number.\",\n", + " llm_config=llm_config,\n", + " functions=[update_context_3_with_random_number],\n", + ")\n", + "\n", + "\n", + "# This is equivalent to writing a transfer function\n", + "agent_3.register_hand_off(ON_CONDITION(agent_4, \"Transfer to Agent 4\"))\n", + "\n", + "agent_4.register_hand_off([AFTER_WORK(agent_5)])\n", + "\n", + "print(\"Agent 1 function schema:\")\n", + "for func_schema in agent_1.llm_config[\"tools\"]:\n", + " print(func_schema)\n", + "\n", + "print(\"Agent 3 function schema:\")\n", + "for func_schema in agent_3.llm_config[\"tools\"]:\n", + " print(func_schema)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Use initiate_swarm_chat\n", - "\n", - "To get response from the swarm agent, you can use the `initiate_swarm_chat` function. This function will return the response from the agent and the updated context variables.\n", - "\n", - "```python\n", - "chat_history, context_variables, last_agent = initialize_swarm_chat(\n", - " init_agent, # which agent to start with\n", - " messages, # list of messages\n", - " agents, # pass in all the swarm agents\n", - " max_rounds, # maximum number of rounds\n", - " context_variables, # initial context variables\n", - ")\n", - "```\n" + "### Start Chat" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33mAgent_1\u001b[0m (to chat_manager):\n", + "\n", + "start\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_1\n", + "\u001b[0m\n", + "\u001b[33mAgent_1\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Suggested tool call (call_kfcEAY2IeRZww06CQN7lbxOf): update_context_1 *****\u001b[0m\n", + "Arguments: \n", + "{}\n", + "\u001b[32m*********************************************************************************\u001b[0m\n", + "\u001b[32m***** Suggested tool call (call_izl5eyV8IQ0Wg6XY2SaR1EJM): transfer_to_agent_2 *****\u001b[0m\n", + "Arguments: \n", + "{}\n", + "\u001b[32m************************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Tool_Execution\n", + "\u001b[0m\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION update_context_1...\u001b[0m\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION transfer_to_agent_2...\u001b[0m\n", + "\u001b[33mTool_Execution\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Response from calling tool (call_kfcEAY2IeRZww06CQN7lbxOf) *****\u001b[0m\n", + "\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m***** Response from calling tool (call_izl5eyV8IQ0Wg6XY2SaR1EJM) *****\u001b[0m\n", + "SwarmAgent --> Agent_2\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_2\n", + "\u001b[0m\n", + "\u001b[33mAgent_2\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Suggested tool call (call_Yf5DTGaaYkA726ubnfJAvQMq): update_context_2_and_transfer_to_3 *****\u001b[0m\n", + "Arguments: \n", + "{}\n", + "\u001b[32m***************************************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Tool_Execution\n", + "\u001b[0m\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION update_context_2_and_transfer_to_3...\u001b[0m\n", + "\u001b[33mTool_Execution\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Response from calling tool (call_Yf5DTGaaYkA726ubnfJAvQMq) *****\u001b[0m\n", + "\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_3\n", + "\u001b[0m\n", + "\u001b[33mAgent_3\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Suggested tool call (call_jqZNHuMtQYeNh5Mq4pV2uwAj): transfer_to_Agent_4 *****\u001b[0m\n", + "Arguments: \n", + "{}\n", + "\u001b[32m************************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Tool_Execution\n", + "\u001b[0m\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION transfer_to_Agent_4...\u001b[0m\n", + "\u001b[33mTool_Execution\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Response from calling tool (call_jqZNHuMtQYeNh5Mq4pV2uwAj) *****\u001b[0m\n", + "SwarmAgent --> Agent_4\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_4\n", + "\u001b[0m\n", + "\u001b[33mAgent_4\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Suggested tool call (call_KeNGv98klvDZsrAX10Ou3I71): get_random_number *****\u001b[0m\n", + "Arguments: \n", + "{}\n", + "\u001b[32m**********************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Tool_Execution\n", + "\u001b[0m\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION get_random_number...\u001b[0m\n", + "\u001b[33mTool_Execution\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Response from calling tool (call_KeNGv98klvDZsrAX10Ou3I71) *****\u001b[0m\n", + "27\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_4\n", + "\u001b[0m\n", + "\u001b[33mAgent_4\u001b[0m (to chat_manager):\n", + "\n", + "The random number generated is 27.\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_5\n", + "\u001b[0m\n", + "\u001b[33mAgent_5\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Suggested tool call (call_MlSGNNktah3m3QGssWBEzxCe): update_context_3_with_random_number *****\u001b[0m\n", + "Arguments: \n", + "{\"random_number\":27}\n", + "\u001b[32m****************************************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Tool_Execution\n", + "\u001b[0m\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION update_context_3_with_random_number...\u001b[0m\n", + "\u001b[33mTool_Execution\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Response from calling tool (call_MlSGNNktah3m3QGssWBEzxCe) *****\u001b[0m\n", + "\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_5\n", + "\u001b[0m\n", + "\u001b[33mAgent_5\u001b[0m (to chat_manager):\n", + "\n", + "The random number 27 has been successfully updated in context 3.\n", + "\n", + "--------------------------------------------------------------------------------\n" + ] + } + ], "source": [ - "from autogen.agentchat.contrib.swarm_agent import initialize_swarm_chat\n", - "\n", "context_variables = {\"1\": False, \"2\": False, \"3\": False}\n", - "chat_result, context_variables, last_agent = initialize_swarm_chat(\n", + "chat_result, context_variables, last_agent = initiate_swarm_chat(\n", " init_agent=agent_1,\n", - " agents=[agent_1, agent_2, agent_3],\n", + " agents=[agent_1, agent_2, agent_3, agent_4, agent_5],\n", " messages=\"start\",\n", " context_variables=context_variables,\n", - ")\n", + " after_work=AFTER_WORK(AfterWorkOption.TERMINATE), # this is the default\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'1': True, '2': True, '3': 27}\n" + ] + } + ], + "source": [ "print(context_variables)" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Demo with User Agent\n", + "\n", + "We pass in a user agent to the swarm chat to accept user inputs. With `agent_6`, we register an `AFTER_WORK` handoff to revert to the user agent when no tool calls are suggested. " + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33mUser\u001b[0m (to chat_manager):\n", + "\n", + "start\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_6\n", + "\u001b[0m\n", + "\u001b[33mAgent_6\u001b[0m (to chat_manager):\n", + "\n", + "Why did the scarecrow win an award? \n", + "\n", + "Because he was outstanding in his field! \n", + "\n", + "Want to hear another one?\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: User\n", + "\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to chat_manager):\n", + "\n", + "yes\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_6\n", + "\u001b[0m\n", + "\u001b[33mAgent_6\u001b[0m (to chat_manager):\n", + "\n", + "Why don't skeletons fight each other?\n", + "\n", + "They don't have the guts! \n", + "\n", + "How about another?\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: User\n", + "\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to chat_manager):\n", + "\n", + "transfer\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_6\n", + "\u001b[0m\n", + "\u001b[33mAgent_6\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Suggested tool call (call_gQ9leFamxgzQp8ZVQB8rUH73): transfer_to_Agent_7 *****\u001b[0m\n", + "Arguments: \n", + "{}\n", + "\u001b[32m************************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Tool_Execution\n", + "\u001b[0m\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION transfer_to_Agent_7...\u001b[0m\n", + "\u001b[33mTool_Execution\u001b[0m (to chat_manager):\n", + "\n", + "\u001b[32m***** Response from calling tool (call_gQ9leFamxgzQp8ZVQB8rUH73) *****\u001b[0m\n", + "SwarmAgent --> Agent_7\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[32m\n", + "Next speaker: Agent_7\n", + "\u001b[0m\n", + "\u001b[33mAgent_7\u001b[0m (to chat_manager):\n", + "\n", + "The joke about the scarecrow winning an award is a play on words. It utilizes the term \"outstanding,\" which can mean both exceptionally good (in the context of the scarecrow's performance) and literally being \"standing out\" in a field (where scarecrows are placed). So, the double meaning creates a pun that makes the joke humorous. \n", + "\n", + "The skeleton joke works similarly. When it says skeletons \"don't have the guts,\" it plays on the literal fact that skeletons don't have internal organs (guts), and metaphorically, \"having guts\" means having courage. The humor comes from this clever wordplay.\n", + "\n", + "--------------------------------------------------------------------------------\n" + ] + } + ], + "source": [ + "from autogen import UserProxyAgent\n", + "\n", + "user_agent = UserProxyAgent(name=\"User\", code_execution_config=False)\n", + "\n", + "agent_6 = SwarmAgent(\n", + " name=\"Agent_6\",\n", + " system_message=\"You are Agent 6. Your job is to tell jokes.\",\n", + " llm_config=llm_config,\n", + ")\n", + "\n", + "agent_7 = SwarmAgent(\n", + " name=\"Agent_7\",\n", + " system_message=\"You are Agent 7, explain the joke.\",\n", + " llm_config=llm_config,\n", + ")\n", + "\n", + "agent_6.register_hand_off(\n", + " [\n", + " ON_CONDITION(\n", + " agent_7, \"Used to transfer to Agent 7. Don't call this function, unless the user explicitly tells you to.\"\n", + " ),\n", + " AFTER_WORK(AfterWorkOption.REVERT_TO_USER),\n", + " ]\n", + ")\n", + "\n", + "chat_result, _, _ = initiate_swarm_chat(\n", + " init_agent=agent_6,\n", + " agents=[agent_6, agent_7],\n", + " user_agent=user_agent,\n", + " messages=\"start\",\n", + ")" + ] } ], "metadata": { From 2df1b5b210a20ca8d43a2b86436bceb74ae63b99 Mon Sep 17 00:00:00 2001 From: Yiran Wu <32823396+kevin666aa@users.noreply.github.com> Date: Wed, 20 Nov 2024 16:15:34 -0500 Subject: [PATCH 11/11] update import --- website/blog/2024-11-17-Swarm/index.mdx | 2 +- website/docs/topics/swarm.ipynb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/website/blog/2024-11-17-Swarm/index.mdx b/website/blog/2024-11-17-Swarm/index.mdx index 6cb9b85de..80e4ff43f 100644 --- a/website/blog/2024-11-17-Swarm/index.mdx +++ b/website/blog/2024-11-17-Swarm/index.mdx @@ -70,7 +70,7 @@ history, context, last_agent = initiate_swarm_chat( This example of managing refunds demonstrates the context handling, swarm and agent-level conditional and after work hand offs, and the human-in-the-loop feature. ```python -from autogen.agentchat.contrib.swarm_agent import initiate_swarm_chat, SwarmAgent, SwarmResult, ON_CONDITION, AFTER_WORK, AfterWorkOption +from autogen import initiate_swarm_chat, SwarmAgent, SwarmResult, ON_CONDITION, AFTER_WORK, AfterWorkOption from autogen import UserProxyAgent import os diff --git a/website/docs/topics/swarm.ipynb b/website/docs/topics/swarm.ipynb index f960144bf..0e7c5353c 100644 --- a/website/docs/topics/swarm.ipynb +++ b/website/docs/topics/swarm.ipynb @@ -147,7 +147,7 @@ "source": [ "import random\n", "\n", - "from autogen.agentchat.contrib.swarm_agent import (\n", + "from autogen import (\n", " AFTER_WORK,\n", " ON_CONDITION,\n", " AfterWorkOption,\n",