Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for exo #458

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix for exo #458

wants to merge 1 commit into from

Conversation

rabejens
Copy link

When using the OpenAI compatible endpoint of Exo, getting v1/models will not expose the model list at data but rather return it directly:

$ curl http://192.168.10.118:52415/v1/models
[
    {
        "id": "llama-3.2-1b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llama-3.2-3b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llama-3.1-8b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llama-3.1-70b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llama-3.1-70b-bf16",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llama-3-8b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llama-3-70b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llama-3.1-405b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llama-3.1-405b-8bit",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "mistral-nemo",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "mistral-large",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "deepseek-coder-v2-lite",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "deepseek-coder-v2.5",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "llava-1.5-7b-hf",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-0.5b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-coder-1.5b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-coder-3b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-coder-7b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-coder-14b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-coder-32b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-7b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-math-7b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-14b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-72b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "qwen-2.5-math-72b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "nemotron-70b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "nemotron-70b-bf16",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "gemma2-9b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "gemma2-27b",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    },
    {
        "id": "dummy",
        "object": "model",
        "owned_by": "exo",
        "ready": true
    }
]

This led to an error when retrieving the model list because res.data was undefined in app/utils/constants.ts line 323.

My quick fix checks if res.data is there and uses it, otherwise, it uses res directly.

I also added a Nix flake and direnv configuration for people using Nix (Darwin or OS) with direnv to manage their development environments.

…he res directly; added flake and envrc for nix-based setups
@dustinwloring1988
Copy link
Collaborator

could you add exo as a provider

@dustinwloring1988 dustinwloring1988 added the question Further information is requested label Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants