enh(model_loaders): Add GPU support.

[daavoo]

Set n_gpu_layers based on torch.cuda.is_available

This just requires to install the correct llama-cpp-python (https://llama-cpp-python.readthedocs.io/en/latest/#supported-backends). For example in a Google Colab:

!pip install https://github.com/abetlen/llama-cpp-python/releases/download/v0.3.4-cu122/llama_cpp_python-0.3.4-cp310-cp310-linux_x86_64.whl

Example running on GPU inside Google Colab:

Takes less than 2 min:

[Kostis-S-Z]

This PR only adds GPU support for the LLM whereas #55 adds support only for the TTS, so I don't understand why we should be closing #55.

[daavoo]

This PR only adds GPU support for the LLM whereas #55 adds support only for the TTS, so I don't understand why we should be closing #55.

This PR makes both text to text and text to speech use GPU. You can test it in #77

[Kostis-S-Z]

This PR only adds GPU support for the LLM whereas #55 adds support only for the TTS, so I don't understand why we should be closing #55.

This PR makes both text to text and text to speech use GPU. You can test it in #77

Right, you are loading the Oute model on the GPU, I was looking at the parler code. Shall we add support only for that one then?

[daavoo]

Right, you are loading the Oute model on the GPU, I was looking at the parler code. Shall we add support only for that one then?

I think we can go with this for now as the main purpose was to get the demo working on a GPU.