This project is derived from the Chatbot UI, a project by Mckay Wrigley. The original Chatbot UI allows you to enter API keys from cloud services (e.g., OpenAI and Azure) to chat with their LLMs. In this work, we allow it to:
- Chat with any priviately deployed LLMs, including llama2 models,
- Customize prompts, system prompts, model selection, temperature, token limits and retry limits on the server so that users have a conssistent experience across clients,
- Access vector databases, image processing, OCR, other AI models, and external web services,
- Move auth management (API keys, tokens and LLM access endpoints) to the server side for better security.
See a demo of a Chatbot "fine tuned" with knowledge of the Rust programming. You can ask it questions related to Rust programming. For example, if you just say "help me write a simple web server", it will give an example in Rust language complete with instructions on how to run it!
We accomplished all these by moving some of the work to the server side. The Chatbot UI is now simply a UI. It posts every message the user enters to a server, and then displays the server response. On the server side, we have flows.network. You just need to deploy a "flow function" and configure it with your private LLM endpoints or OpenAI API keys.
In the HTTP request to the server, the Chatbot UI uses the
x-conversation-idheader to designated a conversation ID so that the server can keep track of multiple conversations.
The flow function is written in Rust. But do not worry, you can just reuse our template without writing a single line of code. Once you are familiar with the system, you can start to modify or write your own flow functions to customize the prompts or perform pre- post-processing work that are specific to your needs.
- Deploy a flow function on flows.network. (see examples for ChatGPT and a private llama2 LLM)
- Copy the flow function's webhook URL in the form of
https://code.flows.network/webhook/UNIQUE-FLOW-ID - Load
https://flows-chat-ui.vercel.app/?chat_url=https://code.flows.network/webhook/UNIQUE-FLOW-IDin your browser.
That's it!
You can fork the project, customize the title, description and default webhook URL by environment variables, and then deploy it on Vercel for free.
| Environment variable | Value |
|---|---|
| SITE_TITLE | Title of the deployed chatbot UI |
| SITE_DESC | Description to display on the UI |
| CHAT_URL | The webhook URL from the flow function |