Add support for users to specify custom request settings, model and optionally provider specific

[JonasHelming]

What it does

Add support for users to specify custom request settings, model and optionally provider specific.
The reason for making them provider specific is that providers have different options and sometimes even different names for the same (see below)

How to test

Add the settings below and adapt it to models you have.
Qwen/Qwen2.5-Coder-32B-Instruct is always "warm" on serverless Hugginface
Starcoder2.3B can be downloaded here: https://huggingface.co/Mozilla/starcoder2-llamafile/tree/main
gemma2 can be directly downloaded with ollama (ollama serve and ollama run gemma2)

Two good test cases:

• set the max_length to something small and observe the model to stop
• set a stop token and ask the model to say it :-)

"ai-features.modelSettings.requestSettings": [
        {
            "modelId": "Qwen/Qwen2.5-Coder-32B-Instruct",
            "requestSettings": {
                "max_new_tokens": 2048,
                "stop": [
                    "<|im_end|>",
                ]
            },
            "providerId": "huggingface"
        },
        {
            "modelId": "gemma2",
            "requestSettings": {
                "num_predict": 1024,
                "stop": [
                    "<|endoftext|>"
                ],
            },
            "providerId": "ollama"
        },
        {
            "modelId": "StarCoder2.3B",
            "requestSettings": {
                "n_predict": 200,
                "stream": true,
                "stop": [
                    "<file_sep>",
                    "<|endoftext|>"
                ],
                "cache_prompt": true,
            },
            "providerId": "llamafile"
        },
        {
            "modelId": "gpt-4o-2024-05-13",
            "requestSettings": {
                "max_tokens": 10,
                "stop": [
                    "<|im_end|>"
                ]
            },
            "providerId": "openai"
        },
    ]

Follow-ups

This should be the last thing we add to the provider layer before we refactor it all together:

• Make the settings more concistent
• Remove a lot of code duplication

Review checklist

• As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• As a reviewer, I agree to behave in accordance with the review guidelines

[JonasHelming]

@dhuebner Could you check the Ollama adaptations please?

[JonasHelming]

See this for docu: eclipse-theia/theia-website#662

[dhuebner]

@JonasHelming
Sure, will do.

[dhuebner]

@JonasHelming
Ollama works as expected!

@sdirix
I have a question to another topic. Would it be possible to mark a LanguageModelRequest with an explicit flag that tells if it's a chat or a completion request? Having this information model implementors could decide if they should use stream or text response.

[JonasHelming]

@sdirix
I have a question to another topic. Would it be possible to mark a LanguageModelRequest with an explicit flag that tells if it's a chat or a completion request? Having this information model implementors could decide if they should use stream or text response.

@dhuebner I also already thought about this, would you mind creating a new ticket and mark me there?

[sdirix]

Thanks for the work ❤️ ! I found some inconsistencies which should be fixed before we merge.

[JonasHelming]

Thank you for the great review. I tried to address all comments (in individual commits) and tested all providers again with the final state.