Add support for Anthropic AI

[emrgnt-cmplxty]

Description

Currently, our project solely supports OpenAI as the language model and embedding provider. To enhance the functionality of our service, we propose the integration of Anthropic AI. However, this integration presents certain challenges due to our current infrastructure.

Previously, we utilized a custom action framework, which we've now replaced with OpenAI's function support. This decision was driven by the excellent fine-tuning capabilities of OpenAI models, providing function outputs with a high degree of reliability. To incorporate Anthropic AI, we need to encapsulate the function output logic within an abstract action handler. Following this, we must mirror agent provider and construction logic similar to that in place for OpenAI.

Proposed Solution

Our solution includes the creation of several abstract classes in Python to facilitate the integration of Anthropic AI. These classes are:

• LLMCompletionResult: Defines different types of LLM completion results.
• LLMChatMessage: Defines different types of LLM chat messages.
• LLMConversation: An abstract base class for different types of LLM conversations.
• LLMConversationDatabaseProvider: An abstract base class for different types of database providers.
• LLMChatCompletionProvider: An abstract base class for different types of LLM chat completion providers.

You can find the exact definitions for these classes in the abstractions.py file in the repository.

In addition to the aforementioned classes, we will also need to adjust the logic around agents. We have an abstract Agent class and several related classes and enums that must be accounted for in our integration. These include:

• AgentToolkitNames: An enum for the different types of agent tools.
• AgentToolkitBuilder: An abstract class for building tools for providers.
• AgentInstance: An abstract class for implementing an agent instance.

The exact definitions for these classes are available in the agents.py file in the repository.

Tasks

To summarize, the tasks that need to be completed are as follows:

1. Create an abstract action handler to encapsulate the logic around the function output of OpenAI models.
2. Mirror agent provider and construction logic for Anthropic AI, similar to what we currently have for OpenAI.
3. Modify the abstract classes mentioned above to accommodate the integration of Anthropic AI. This includes creating subclasses or adjusting methods where necessary.
4. Test the integration to ensure functionality and compatibility.

Estimated Effort

This is a large-scale task that involves substantial changes to the codebase. We estimate this will require a non-trivial amount of work, depending on the developer's familiarity with the codebase and Anthropic AI.

Requested Reviewers

Given the complexity of the task, we request review and feedback from @anthropic-integration-lead and @Project-Lead before, during, and after implementation to ensure all changes align with our project goals.

Completion Criteria

The integration will be considered successful when:

1. Anthropic AI can be used as an alternative to OpenAI as the language model and embedding provider.
2. The abstract classes and methods can accommodate the functionality of both OpenAI and Anthropic AI.
3. All tests pass, and the system works without errors.
4. Code review and feedback from requested reviewers have been addressed and integrated.

Notes

Given that this is a major change to the codebase, we suggest implementing this on a separate branch and using feature flags to control the availability of this new integration. This way, we can seamlessly roll back the changes in case of any unforeseen issues.