Empty and redudant "assistant":"" message for external groq API calls

[arvislacis]

Version

1.53.1734584341

Describe the bug

I am aware that cody.dev.models is experimental feature but there is an issue with external OpenAI compatible (groq) API calls.

For example, given the following configuration:

    "cody.dev.models": [
        {
            "provider": "groq",
            "model": "llama-3.3-70b",
            "apiKey": "*************",
            "apiEndpoint": "https://api.venice.ai/api/v1/chat/completions"
        }
    ]

The Cody AI chat will fail, returning the 400 Bad Request satus from API provider with error that The last message must have the role "user" or "tool". @PriNova is aware of this issue because we had conversation about this on Discord.

As I debugged then the main cause of the mentioned issues is the fact that Cody AI always appends {"role":"assistant","content":""} as the last message of an API call although the most OpenAI compatible (groq) APIs expect the last message to be with role user.

This issue happens with Venice AI LLM provider..., I believe that some other API providers ignore this last, empty {"role":"assistant","content":""} message and works without problems. Anyways, it would be nice that API calls would be made in unified format, as expected from OpenAI compatible documentations.

Expected behavior

External groq API calls should not include the {"role":"assistant","content":""} message in the messages array sent to the API end-point.

Additional context

Discord conversation: https://discord.com/channels/969688426372825169/1318523844788752386

[linear]

BUGS-1004 Empty and redudant "assistant":"" message for external groq API calls

[PriNova]

@arvislacis

Thank you for reporting the issue and for the detailed discussion on Discord.