From 619d91e6f8138222ee66e61003f30559e573f3ff Mon Sep 17 00:00:00 2001
From: Nick von Pentz <12549658+nvonpentz@users.noreply.github.com>
Date: Mon, 28 Oct 2024 14:20:08 -0400
Subject: [PATCH] Set Leo default model to Llama 3 8b (25% rollout) (#1225)

Related to https://github.com/brave/brave-core/pull/21398
Blocked on deploying https://github.com/brave/aichat-ops/pull/359 to
prod

This sets the default model for 25% of free users to `chat-basic`, which
corresponds to [Llama 3
8b](https://github.com/brave/brave-core/blob/c839e1676031a3d155b04b9dbfe49e11b3b8601b/components/ai_chat/core/browser/model_service.cc#L154).

The intention is to start this to progressively roll out this change
(25%, 50%, 75%, 100%), so we can ensure our single instance of Llama 3
can handle the increase in traffic. Based on these
[estimations](https://artificialanalysis.ai/?models_selected=llama-3-1-instruct-8b%2Cmixtral-8x7b-instruct),
one instance of llama 3 (one gpu) should have higher token throughput
than our single instance of mixtral (four gpus), so we expect this to
work with two llama instances which will be added in
https://github.com/brave/aichat-ops/pull/359.

cc @petemill @LorenzoMinto

Note:
* Chromium version 122.0.6261.57 was selected because that was the first
chromium version when 1.63.x was released (see
https://bravesoftware.slack.com/archives/C04PX1BUN/p1708629893634639),
which is when https://github.com/brave/brave-core/pull/21398 went in.
* I have included all platforms because we want to make this change
across all platforms, however this does differ from the
BraveAIChatEnabledStudy which applies only to desktop

---------

Co-authored-by: Nick von Pentz <nvonpentz@users.noreply.github.com>
Co-authored-by: Pete Miller <miller.pete@gmail.com>
Co-authored-by: Aleksey Khoroshilov <akhoroshilov@brave.com>
---
 studies/BraveAIChatDefaultModelStudy.json5 | 46 ++++++++++++++++++++++
 studies/BraveAIChatEnabledStudy.json5      |  1 +
 2 files changed, 47 insertions(+)
 create mode 100644 studies/BraveAIChatDefaultModelStudy.json5

diff --git a/studies/BraveAIChatDefaultModelStudy.json5 b/studies/BraveAIChatDefaultModelStudy.json5
new file mode 100644
index 00000000..8cc9bcbb
--- /dev/null
+++ b/studies/BraveAIChatDefaultModelStudy.json5
@@ -0,0 +1,46 @@
+[
+  {
+    name: 'BraveAIChatDefaultModelStudy',
+    experiment: [
+      {
+        name: 'DefaultLlama',
+        probability_weight: 25,
+        feature_association: {
+          enable_feature: [
+            'AIChat',
+          ],
+        },
+        param: [
+          {
+            name: 'default_model',
+            value: 'chat-basic',
+          },
+        ],
+      },
+      {
+        name: 'DefaultMixtral',
+        probability_weight: 75,
+        feature_association: {
+          enable_feature: [
+            'AIChat',
+          ],
+        },
+      },
+    ],
+    filter: {
+      min_version: '122.1.63.161',
+      channel: [
+        'RELEASE',
+        'BETA',
+        'NIGHTLY',
+      ],
+      platform: [
+        'WINDOWS',
+        'MAC',
+        'LINUX',
+        'ANDROID',
+        'IOS',
+      ],
+    },
+  },
+]
diff --git a/studies/BraveAIChatEnabledStudy.json5 b/studies/BraveAIChatEnabledStudy.json5
index e3b7e42f..9f61e4f6 100644
--- a/studies/BraveAIChatEnabledStudy.json5
+++ b/studies/BraveAIChatEnabledStudy.json5
@@ -18,6 +18,7 @@
     ],
     filter: {
       min_version: '119.1.60.0',
+      max_version: '122.1.63.160',
       channel: [
         'RELEASE',
       ],