Ollama, Mistral AI, etc.

[haraldschilly]

Description

• selecting a model in the dialog is now a dropdown, plus some small changes
  • this dropdown is used in account settings/others
• add dynamic frontend configuration for arbitrary ollama models
  • with that, backend logic to deal with arbitrary ollama-[model] models
  • my main test for this ended up to be google's gamma model
• many enhancements, refactorings and cleanups for LLM overall. This does not change the actual naming schema of the models, yet. This PR is already too complex.
• Slight changes to the system prompt:
  • it now depends on the vendor/model
  • in the future it might even depend on the file type (path)
• Many changes and fixes for the AI Formula generator
• Adding GPT-4 Turbo as yet another paid option
  • Since the minimum commitment is >$5, I've also added an "8k" variant of it. That way, the limits are lower, less "dangerous" to use
• adding Mistral models and the "Large" one is a paid model
  • The big issue with mistral is their badly coded lib. I had to fork it: https://github.com/sagemathinc/mistralai-client-js
  • and then figure out how to tell langchain to use it. The package.json config looks odd, I have the feeling there there is a bug related to using this in a workspace. In any case, once this is fixed upstream, this bandaid has to go away.
• fixing disable some AI integration instructor course setting seems to disable everything now #7341

testing

chat with GPT4 turbo (deliberately bad language on my side!) and testing reply-continuation.

I can also select an LLM in the slate editor in a chat input

course

1. partially disabled: explain cell and fix-error does work, and also replies
2. partially disabled: AI formula does say it is disabled
3. fully disabled: no explain button, no fix-error button
4. base case: no AI tools disabled: yes, students can do all of this

Ollama

This is the main part before this PR escalated: there is now a config parameter, which contains a dict of {[model name]: {config params}, ... } which is described in the description of that admin field and also has some detailed checks. The main point is, you can chat with any model that is available via the ollama api, you can have several ollama instances, and for each of them several models.

configuration example

and with that, there is a @gemma chat, a displayed name, a description, and of course it streams answers like the others

status

ok, the latest with this PR is everything is tested and works, but there seems to be a problem with the mistral lib itself, and modifying what fetch does. this is a bit ugly.

This seems to be the root cause: mistralai/client-js#42

Checklist:

• Testing instructions are provided, if not obvious
• Release instructions are provided, if not obvious

[haraldschilly]

this is super long and I stumbled over many obstacles. however, I'm now at a point where I think it's fine to merge. I didn't do all the refactoring I wanted to do, but at least a lot of things are collected in one place now. Further details are in the first message.