What type of communication, (and how much), between agents actually creates a better end product/plan?

[binderg]

Should there be meetings between agents to discuss plans and get feedback before going and executing tasks? And if so, how often should each level meet before it becomes unproductive? There are many reasons why hierarchical and group structures have regular meetings but how many of them apply here? After all, they all start out with similar experiences/intelligence. But on the other hand their given missions might lead them to having goals that could use some correcting from a higher up agent with a more managerial role who has more of a focus on the bigger picture?

(Maybe it's worth having the frequency of the communications part modular enough that it could be tweaked after the community plays around with it).

[daveshap]

this is why we need to experiment

[Czajnikuss]

I would say it will vary and greatly... an at lest for me the real question is not how often but with whom?
In post Making it AutoGen-like I went road of creating an environment for agents to talk to each other and to humans, it is not sufficiently generic...

I know it is a long introduction to the answer that may not satisfy You but please bear with me...

Agents them selves are not correct level of analysis, since what we need is for task to be completed, that means that we need an orchestrator or team lead, or group spokes-agent that would be the point of contact we may or may not chose for this agent to be also a planner agent, no matter, lets assume we have a task at hand, we will need to understand the task and assign a Team Lead to it, the n it will create a preliminary plan that will be focused on combination of talent and outsource data needed to accomplish the task. When we know the talent list, Tam lead asks Agent Creator agent to create needed talent. Agent Creator first scans previously used Agents to find a perfect match (so a list of agents is needed) if nothing found it creates a new one and registers it to the Agent Registry. After creation new Agent is assigned to the group orchestrated by Teal lead that asked creator for talent. By default this new agent for the duration of this task (or rather in the thread of this task) will communicate only with its team lead and/or team mates accordingly to rules set by team lead.

This wall of text is to say again it wearies and needs to be flexible. a single point of communication with the word by team lead is so it would be easier for SOB to operate since if this team have to be a guardian of morality and other aspects all finished tasks (maybe also starts) would go through it.

I would say we have to be flexible, there are tasks like coding in general where it is good to have a chain of interactions, planer- coder - tester - validator and we will have tasks that would benefit from lack of structure like brainstorming. Let teams decide.

[daveshap]

I wonder if we just need to give each agent an agent lookup table with like distribution lists and addresses to "know" who to talk to and who is available to talk to, as well as channels.

[Czajnikuss]

I would rather said that this lookup table should be available for team leads and creator for sure. But I see no utility in all agents having that. If a created tam fail maybe 3 times in a row, then Team lead should change team compositions, or be replaced... we can try to group teams by capabilities, since no one said that we can not have a team that consist of other teams...

But getting back to communication it would be Team lead dependent, it will decide how and to whom team mates will communicate by giving them access to different tools. This ensures flexibility and scalability

That leads us to tools that are not part of this discussion yet... so just to hin the possible solution I can see 2 different categories of tools, the ones that will be available fo all the agents, like ... commiunicate_to_the_team_lead or... comunicate_dow_the_chain... and second category of tools that would be group tools. These functions would only be available for team members that created them,

[daveshap]

I think all agents should have at least a few basic lookups but yeah, the global address list will get big. Maybe one of the main functinos is a lookup function

[JJourneys]

Org Bandwidth Problem
It occurs to me that this is (primarily) workplace process and systems thinking / org structure problem solving; something humans have been trying to figure out for a while. We will definitely need experimentation, and this format allows us to test faster, which is nice. But one piece I see missing is what I'm calling the bandwidth problem. (Maybe this should be it's own thread?)

Middle management exists fundamentally because humans only have so much time and energy, so information needs to be condensed and distributed only when relevant, leading to all sorts of problems along the hierarchy chain. Erasing human limitations (starting from scratch, first principles, etc. etc.), an ideal structure in my mind would be something like one entity that (1) understands an end goal and certain limitations (ethics, etc.); (2) finds the best approach to success (probably formatted in batches but as a roadmap/timeline accounting for dependencies); and (3) executes each task/batch as instantly as possible, always (4) checking to make sure the desired next task and execution approach hasn't changed based on learned information from previous completed tasks and correcting when necessary. Basically god, if you'd like, described as omnipotent and all-knowing.

Now I'll take a step back and look at structuring a single hive entity functioning to the blind as what I've described above, keeping in consideration the different bandwidth limitations humans and these models have. Each single "minor" entity within the hive can either be one running agent or a micro-hive / team of agents (I need better terms); this is basically just a technology question which will be answered with experimentation where certain tasks will be performed better as multiple agents rather than one and vice versa. Initial goal construction I think needs to be a human, or a human and agent(s) collaboration due to a potential alignment problem. Then setting the limitations (ethics, etc.) can be agent(s) continuing on in the project. Then the creator of agents, who's guidelines need to be approved by the SOB. SOB instructions can never be edited by the HAAS. All other agent instructions can be edited with approval from the SOB, ensuring no moral drift. (There's concern for hallucination where the SOB is convinced incorrectly by a persistent bad actor, intentional or not. SOB needs to be quick to eliminate and regenerate agents at any sign of disobedience to the goal.) The strategist / CEO needs to work backwards from the goal to get there from where it's at, each step being approved by the SOB. Then, I think, you have necessary domain expertise divided among the executive staff, and a new project for every task towards the goal, with it's own micro-goals either to completion or until changed for ongoing tasks like content posts. No middle management or directors. These tasks can be divided up into 3 sections, training/development, system optimization, and admin tasks such as progress reporting and scheduling. Training, development, and system optimization is different in humans, but with agents they can be combined to just changing agent instructions. There should be a team who's ongoing responsibility is to test new instructions on a small scale for improvements, and if successful, edit agent instructions with SOB approval for exponential improvement. Then it's just reporting and scheduling. Scheduling is taken care of by the Strategist and having dedicated available agents. That leaves reporting, which definitely needs a solution. Maybe this is where a vector database can come in, or something like MemGPT accessed by the Strategist only but written to by the project agents. I'm not sure, this will require more of that experimentation.

Anyway, these are my thoughts, hope it's not too long. Just wanted to throw it out incase this helps when thinking through the organization problem, we don't need to limit ourselves to human bandwidth constraints and traditional Org Charts.

[rudiheydra]

The discussion here points to a need for an efficient and scalable messaging system within our agent architecture. Apache Kafka could be a strong candidate for this role. It's designed to handle high volumes of data and enables messages to be passed between services in a fault-tolerant manner.

By leveraging Kafka, agents can function both as producers and consumers of messages, ensuring that each agent receives only the information relevant to its tasks. This aligns seamlessly with the modular communication strategy that's been suggested. Kafka's topics can be utilized to create channels for specific types of communication, controlled by the team lead, much like a Kafka broker would in its ecosystem.

Kafka's real-time data streaming capabilities also mean that agents can continuously learn from ongoing tasks and adapt their strategies accordingly. This dynamic adjustment is vital for maintaining the agility of the system.

Furthermore, Kafka's robustness and fault tolerance can uphold the reliability needed for strategic communications and operational directives. Given that Kafka is open source, it presents an opportunity for developers to contribute directly to the implementation and refinement of this messaging system.

Implementing Kafka would provide the necessary infrastructure to manage the complex communication needs of a multi-agent system, ensuring efficiency, scalability, and alignment with overarching goals. It's a flexible solution that can grow with the project, adapting to changing requirements and facilitating continuous improvement in agent interactions.