This project provides a collective list of Generative AI (GenAI) and Retrieval-Augmented Generation (RAG) examples such as chatbot with question and answering (ChatQnA), code generation (CodeGen), document summary (DocSum), etc.
All the examples are well-validated on Intel platforms. In addition, these examples are:
-
Easy to use. Use ecosystem-compliant APIs to build the end-to-end GenAI examples
-
Easy to customize. Customize the example using different framework, LLM, embedding, serving etc.
-
Easy to deploy. Deploy the GenAI examples with performance on Intel platforms
Note: The below support matrix gives the validated configurations. Feel free to customize per your needs.
ChatQnA is an example of chatbot for question and answering through retrieval argumented generation (RAG).
| Framework | LLM | Embedding | Vector Database | Serving | HW | Description |
| LangChain | NeuralChat-7B | BGE-Base | Redis | TGI-Habana | Gaudi2 | Chatbot |
| LangChain | NeuralChat-7B | BGE-Base | Chroma | TGI-Habana | Gaudi2 | Chatbot |
| LangChain | Mistral-7B | BGE-Base | Redis | TGI-Habana | Gaudi2 | Chatbot |
CodeGen is an example of copilot designed for code generation in Visual Studio Code.
| Framework | LLM | Serving | HW | Description |
| LangChain | Deepseek-Coder-33B | TGI-Habana | Gaudi2 | Copilot |
DocSum is an example of chatbot for summarizing the content of documents or reports.
| Framework | LLM | Serving | HW | Description |
| LangChain | NeuralChat-7B | TGI-Habana | Gaudi2 | Chatbot |
| LangChain | Mistral-7B | TGI-Habana | Gaudi2 | Chatbot |
SearchQnA is an example of chatbot for using search engine to enhance QA quality.
| Framework | LLM | Serving | HW | Description |
| LangChain | NeuralChat-7B | TGI-Habana | Gaudi2 | Chatbot |
| LangChain | Mistral-7B | TGI-Habana | Gaudi2 | Chatbot |
VisualQnA is an example of chatbot for question and answering based on the images.
| Framework | LLM | Serving | HW | Description |
| LangChain | LLaVA-1.5-7B | TGI-Habana | Gaudi2 | Chatbot |