Set Leo default model to Llama 3 8b (25% rollout) (#1225)

Related to brave/brave-core#21398 Blocked on deploying brave/aichat-ops#359 to prod This sets the default model for 25% of free users to `chat-basic`, which corresponds to [Llama 3 8b](https://github.com/brave/brave-core/blob/c839e1676031a3d155b04b9dbfe49e11b3b8601b/components/ai_chat/core/browser/model_service.cc#L154). The intention is to start this to progressively roll out this change (25%, 50%, 75%, 100%), so we can ensure our single instance of Llama 3 can handle the increase in traffic. Based on these [estimations](https://artificialanalysis.ai/?models_selected=llama-3-1-instruct-8b%2Cmixtral-8x7b-instruct), one instance of llama 3 (one gpu) should have higher token throughput than our single instance of mixtral (four gpus), so we expect this to work with two llama instances which will be added in brave/aichat-ops#359. cc @petemill @LorenzoMinto Note: * Chromium version 122.0.6261.57 was selected because that was the first chromium version when 1.63.x was released (see https://bravesoftware.slack.com/archives/C04PX1BUN/p1708629893634639), which is when brave/brave-core#21398 went in. * I have included all platforms because we want to make this change across all platforms, however this does differ from the BraveAIChatEnabledStudy which applies only to desktop --------- Co-authored-by: Nick von Pentz <[email protected]> Co-authored-by: Pete Miller <[email protected]> Co-authored-by: Aleksey Khoroshilov <[email protected]>
brave · Oct 28, 2024 · 619d91e · 619d91e
1 parent 6b8ba27
commit 619d91e
Show file tree

Hide file tree

Showing 2 changed files with 47 additions and 0 deletions.
diff --git a/studies/BraveAIChatDefaultModelStudy.json5 b/studies/BraveAIChatDefaultModelStudy.json5
@@ -0,0 +1,46 @@
+[
+  {
+    name: 'BraveAIChatDefaultModelStudy',
+    experiment: [
+      {
+        name: 'DefaultLlama',
+        probability_weight: 25,
+        feature_association: {
+          enable_feature: [
+            'AIChat',
+          ],
+        },
+        param: [
+          {
+            name: 'default_model',
+            value: 'chat-basic',
+          },
+        ],
+      },
+      {
+        name: 'DefaultMixtral',
+        probability_weight: 75,
+        feature_association: {
+          enable_feature: [
+            'AIChat',
+          ],
+        },
+      },
+    ],
+    filter: {
+      min_version: '122.1.63.161',
+      channel: [
+        'RELEASE',
+        'BETA',
+        'NIGHTLY',
+      ],
+      platform: [
+        'WINDOWS',
+        'MAC',
+        'LINUX',
+        'ANDROID',
+        'IOS',
+      ],
+    },
+  },
+]
diff --git a/studies/BraveAIChatEnabledStudy.json5 b/studies/BraveAIChatEnabledStudy.json5
@@ -18,6 +18,7 @@
     ],
     filter: {
       min_version: '119.1.60.0',
+      max_version: '122.1.63.160',
       channel: [
         'RELEASE',
       ],