feat: runtime configuration of vision-capable models #5919
Conversation
…ies to models through env var NEXT_PUBLIC_VISION_MODELS.
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
@Yiming3 is attempting to deploy a commit to the NextChat Team on Vercel. A member of the Team first needs to authorize it. |
WalkthroughThe changes in this pull request introduce two new constants in Changes
Possibly related PRs
Suggested labels
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Outside diff range and nitpick comments (1)
app/utils.ts (1)
263-264: Add Unit Tests forisVisionModelFunctionTo ensure that the
isVisionModelfunction correctly identifies vision models, consider adding unit tests that cover various model names, including edge cases and potential false positives or negatives.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (2)
app/constant.ts(1 hunks)app/utils.ts(2 hunks)
🔇 Additional comments (1)
app/constant.ts (1)
307-308: Ensure EXCLUDE_VISION_MODEL_REGEXES Accurately Excludes Intended Models
Verify that /claude-3-5-haiku-20241022/ precisely matches and excludes only the specific model "claude-3-5-haiku-20241022" without affecting other models named similarly.
Apply this diff to clarify the intent:
export const EXCLUDE_VISION_MODEL_REGEXES = [
- /claude-3-5-haiku-20241022/
+ /^claude-3-5-haiku-20241022$/
];This change ensures that only the exact model name is excluded.
| const envVisionModels = process.env.NEXT_PUBLIC_VISION_MODELS?.split(",").map( | ||
| (m) => m.trim(), | ||
| ); | ||
| if (envVisionModels?.includes(model)) { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Handle Potential Empty or Malformed Environment Variable
If NEXT_PUBLIC_VISION_MODELS is undefined, empty, or improperly formatted, envVisionModels might be undefined or contain unintended values. Consider adding a default value or additional validation to ensure robust handling.
Apply this diff to improve robustness:
const envVisionModels = process.env.NEXT_PUBLIC_VISION_MODELS?.split(",").map(
(m) => m.trim(),
);
+const visionModelsSet = new Set(envVisionModels ?? []);
-if (envVisionModels?.includes(model)) {
+if (visionModelsSet.has(model)) {
return true;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const envVisionModels = process.env.NEXT_PUBLIC_VISION_MODELS?.split(",").map( | |
| (m) => m.trim(), | |
| ); | |
| if (envVisionModels?.includes(model)) { | |
| return true; | |
| } | |
| const envVisionModels = process.env.NEXT_PUBLIC_VISION_MODELS?.split(",").map( | |
| (m) => m.trim(), | |
| ); | |
| const visionModelsSet = new Set(envVisionModels ?? []); | |
| if (visionModelsSet.has(model)) { | |
| return true; | |
| } |
| export const VISION_MODEL_REGEXES = [ | ||
| /vision/, | ||
| /gpt-4o/, | ||
| /claude-3/, | ||
| /gemini-1\.5/, | ||
| /gemini-exp/, | ||
| /learnlm/, | ||
| /qwen-vl/, | ||
| /qwen2-vl/, | ||
| /gpt-4-turbo(?!.*preview)/, // Matches "gpt-4-turbo" but not "gpt-4-turbo-preview" | ||
| /^dall-e-3$/, // Matches exactly "dall-e-3" | ||
| ]; |
There was a problem hiding this comment.
💡 Codebase verification
Let me generate the final response based on the test results.
Regex Patterns in VISION_MODEL_REGEXES Need Adjustments
The test results reveal some issues with the regex patterns:
- The
/gpt-4-turbo(?!.*preview)/pattern is not working as intended in bash regex - it's not matching "gpt-4-turbo" as expected - The
/gemini-1\.5/pattern works correctly, matching "gemini-1.5-pro" - The
/^dall-e-3$/pattern works as expected, matching exactly "dall-e-3" but not "dall-e-3-beta"
Consider:
- Using a simpler negative pattern for gpt-4-turbo, like
^gpt-4-turbo$to match exactly "gpt-4-turbo" - Adding test cases for other patterns (vision, gpt-4o, claude-3, etc.) to ensure they match their intended targets
🔗 Analysis chain
Verify Correctness of VISION_MODEL_REGEXES Patterns
Ensure that each regex in VISION_MODEL_REGEXES accurately matches the intended vision models without unintended matches. Pay special attention to:
/gemini-1\.5/: Confirm that the backslash is correctly escaping the dot to matchgemini-1.5./gpt-4-turbo(?!.*preview)/: Ensure that this pattern matches"gpt-4-turbo"but not"gpt-4-turbo-preview".
Run the following script to test the regex patterns against a list of model names:
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Test VISION_MODEL_REGEXES against existing model names
# Define model names for testing
model_names=(
"gpt-4-turbo"
"gpt-4-turbo-preview"
"gemini-1.5-pro"
"gemini-1x5-pro"
"dall-e-3"
"dall-e-3-beta"
)
# Define regex patterns
regex_patterns=(
"vision"
"gpt-4o"
"claude-3"
"gemini-1\\.5"
"gemini-exp"
"learnlm"
"qwen-vl"
"qwen2-vl"
"gpt-4-turbo(?!.*preview)"
"^dall-e-3$"
)
# Test each model name against each regex
for model in "${model_names[@]}"; do
for pattern in "${regex_patterns[@]}"; do
if [[ $model =~ $pattern ]]; then
echo "Model '$model' matches regex '$pattern'"
fi
done
done
Length of output: 5253
Yiming3
changed the title
💻 变更类型 | Change Type
🔀 变更说明 | Description of Change
This PR provides runtime configuration of vision-capable models suggested from #5843
NEXT_PUBLIC_VISION_MODELSto allow runtime configuration of vision-capable modelsKey Changes:
VISION_MODEL_REGEXESandEXCLUDE_VISION_MODEL_REGEXESinconstant.tsisVisionModel()to use regex patterns instead of string includesNEXT_PUBLIC_VISION_MODELSenvironment variable📝 补充信息 | Additional Information
Summary by CodeRabbit
New Features
Bug Fixes