diff --git a/README.md b/README.md index e953511..13f87b8 100644 --- a/README.md +++ b/README.md @@ -1,40 +1,68 @@ # ChatterUI - A simple app for LLMs -ChatterUI is a native mobile frontend for managing chat files and character cards inspired by SillyTavern. -It aims to provide a mobile friendly experience to inferface with Large Language models. -ChatterUI supports multiple backends and can even run GGUF models locally on your device. +ChatterUI is a native mobile frontend for LLMs. -Support the development of this app here: +Run LLMs on device or connect to various commercial or open source APIs. ChatterUI aims to provide a mobile friendly experience to inferface with Large Language models, while also providing low level control on how your chats are structured. + +If you like the app, feel free support me here: Support me on ko-fi.com
- - - +Chat With Characters or Assistants +
+ + +
+Use on-device Models or APIs +
+ +
- - - +Modify and customize to your liking +
+ + +
+Personalize yourself +
+ +
-## Usage +## Features: -#### Android +- Run LLMs on-device in Local Mode +- Connect to various APIs in Remote Mode +- Chat with characters. (Supports the Character Card v2 specification.) +- Create and manage multiple chats per character. +- Customize Sampler fields and Instruct formatting +- Integrates your on-device TTS engine for TTS support. + +
+ +# Usage Download the and install latest APK from the [releases](https://github.com/Vali-98/ChatterUI/releases/latest) page. -IOS is Currently unavailable, I do not have the means to test and deploy on IOS :( +IOS is Currently unavailable, I do not have the hardware for IOS development + +## Local Mode + +ChatterUI uses a [llama.cpp](https://github.com/ggerganov/llama.cpp) under the hood to run gguf files on device. A custom adapter is used to integrate with react-native: [cui-llama.rn](https://github.com/Vali-98/cui-llama.rn) -## Supported Backends +To use on-device inferencing, first enable Local Mode, then go to Models > Import Model / Use External Model and choose a gguf model that can fit on your device's memory. The importing functions are as follows: -### On Device Inferencing: +- Import Model: Copies the model file into ChatterUI - this may speed up the startup process +- Use External Model: Uses a model from your device storage directly - this removes the need to copy large files into ChatterUI, but loads models slightly slower -- Local device inferencing using [cui-llama.rn](https://github.com/Vali-98/cui-llama.rn) +After that, you can load the model and begin chatting! -_To use Local inferencing, go to API > Local > Import Model and choose a gguf model that can fit on your device's memory. Then load the model to begin!_ +_Note: For devices with Snapdragon 8 Gen 1 and above or Exynos 2200+, it is recommended to use the Q4_0_4_8 quantization for enhanced prompt processing._ -_For devices with Snapdragon 8 Gen 1 and above or Exynos 2200+, it is recommended to use the Q4_0_4_8 quantization for enhanced prompt processing._ +## Remote Mode + +Remote Mode allows you to connect to a few common APIs from both commercial and open source projects. ### Open Source Backends: @@ -113,3 +141,8 @@ if (BuildConfig.DEBUG) { // ReactNativeFlipper.initializeFlipper(this, reactNativeHost.reactInstanceManager) } ``` + +## Acknowledgement + +- [llama.cpp](https://github.com/ggerganov/llama.cpp) - the underlying engine to run LLMs +- [llama.rn](https://github.com/mybigday/llama.rn) - the original react-native llama.cpp adapter diff --git a/assets/screenshots/api.png b/assets/screenshots/api.png new file mode 100644 index 0000000..205e48f Binary files /dev/null and b/assets/screenshots/api.png differ diff --git a/assets/screenshots/charactereditor.png b/assets/screenshots/charactereditor.png new file mode 100644 index 0000000..213b146 Binary files /dev/null and b/assets/screenshots/charactereditor.png differ diff --git a/assets/screenshots/characterlist.png b/assets/screenshots/characterlist.png new file mode 100644 index 0000000..85dbee8 Binary files /dev/null and b/assets/screenshots/characterlist.png differ diff --git a/assets/screenshots/charsmenu.png b/assets/screenshots/charsmenu.png deleted file mode 100644 index 80e9b29..0000000 Binary files a/assets/screenshots/charsmenu.png and /dev/null differ diff --git a/assets/screenshots/chat.png b/assets/screenshots/chat.png new file mode 100644 index 0000000..6dee287 Binary files /dev/null and b/assets/screenshots/chat.png differ diff --git a/assets/screenshots/instruct.png b/assets/screenshots/instruct.png deleted file mode 100644 index 2998230..0000000 Binary files a/assets/screenshots/instruct.png and /dev/null differ diff --git a/assets/screenshots/mainchat.png b/assets/screenshots/mainchat.png deleted file mode 100644 index b493343..0000000 Binary files a/assets/screenshots/mainchat.png and /dev/null differ diff --git a/assets/screenshots/models.png b/assets/screenshots/models.png new file mode 100644 index 0000000..5dc1281 Binary files /dev/null and b/assets/screenshots/models.png differ diff --git a/assets/screenshots/optionsmenu.png b/assets/screenshots/optionsmenu.png deleted file mode 100644 index d4fa897..0000000 Binary files a/assets/screenshots/optionsmenu.png and /dev/null differ diff --git a/assets/screenshots/recents.png b/assets/screenshots/recents.png deleted file mode 100644 index 9afffea..0000000 Binary files a/assets/screenshots/recents.png and /dev/null differ diff --git a/assets/screenshots/sampler.png b/assets/screenshots/sampler.png deleted file mode 100644 index f51a769..0000000 Binary files a/assets/screenshots/sampler.png and /dev/null differ diff --git a/assets/screenshots/settings.png b/assets/screenshots/settings.png new file mode 100644 index 0000000..3373b59 Binary files /dev/null and b/assets/screenshots/settings.png differ diff --git a/assets/screenshots/tts.png b/assets/screenshots/tts.png deleted file mode 100644 index 1ebabfb..0000000 Binary files a/assets/screenshots/tts.png and /dev/null differ diff --git a/assets/screenshots/usereditor.png b/assets/screenshots/usereditor.png new file mode 100644 index 0000000..2a34ecd Binary files /dev/null and b/assets/screenshots/usereditor.png differ diff --git a/assets/screenshots/userlist.png b/assets/screenshots/userlist.png new file mode 100644 index 0000000..fac65d1 Binary files /dev/null and b/assets/screenshots/userlist.png differ