OpenAILike & Ollama issues

[h2ogeek]

I tried to use the OpenAILike, but I am getting a 404. I used the code provided in the documentation. I am using Ollama in my local environment. I tested ollama with the openAI compatibility using curl and had no issues.

Error

Retrying llama_index.llms.openai.base.OpenAI._complete in 0.6627152141493229 seconds as it raised NotFoundError: 404 page not found.

Curl ✅

curl http://localhost:11434/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "llama2",
        "messages": [
            {
                "role": "system",
                "content": "You are a helpful assistant."
            },
            {
                "role": "user",
                "content": "Hello!"
            }
        ]
    }'

The Ref Code ❌

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(model="lama2", api_base="http://localhost:11434/v1", api_key="ollama")

response = llm.complete("Hello World!")
print(str(response))

[dosubot[bot]]

It looks like the issue might be related to the endpoint or request format used in your Python code compared to the curl command. Ensure that the complete method in the OpenAILike class targets the correct endpoint (/v1/chat/completions) and that the request format (headers, HTTP method, data/parameters) matches what's successful with your curl command.

Here are steps to troubleshoot and align your Python code with the curl example:

1. Verify Endpoint: Confirm that the OpenAILike class's complete method is designed to hit http://localhost:11434/v1/chat/completions. This might involve checking the implementation of the complete method to ensure it constructs the request URL correctly using the api_base you provided.

2. Request Format: Compare the request format (including headers and body) between your Python code and the curl command. Ensure that the Python request mimics the successful curl request as closely as possible.

3. Error Handling: Pay attention to any error messages or responses you're getting from your Python code. These can offer clues about what might be going wrong.

If after these checks you're still facing issues, consider providing more details about the error message and the version of the llama_index library you're using. This will help in diagnosing the problem more accurately.

Sources

• llama-index-legacy/tests/llms/test_openai_like.py
• llama-index-integrations/llms/llama-index-llms-openai-like/llama_index/llms/openai_like/base.py
• llama-index-integrations/readers/llama-index-readers-github/llama_index/readers/github/collaborators/github_client.py
• llama-index-integrations/tools/llama-index-tools-requests/llama_index/tools/requests/base.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[ogimgio]

Have the same problem

[Loongphy]

Add constructor arguments is_chat_model=True.
It will like this:

llm = OpenAILike(model=OPENAI_API_MODEL,api_base=OPENAI_API_BASE,api_key=OPENAI_API_KEY,is_chat_model=True)

[hu-qi]

is_chat_model=True

Thanks! Good job, It's worked~