Transformer

Generative Pre-trained Transformer

Warnings

• This module uses a pre-trained transformer to generate predictive responses.
• Due to the size of machine learning models, this feature will be disabled in limited mode.

Requirements

Minimum RAM availability

• 8 GB to run the 7B models
• 16 GB to run the 13B models
• 32 GB to run the 33B models

References

• Model Artifactory: https://ollama.com/library
• Alternatives: https://huggingface.co/meta-llama
• Supported Models: https://github.com/ollama/ollama/blob/main/README.md#model-library

Future Iterations

• Customize prompt - 72b9e91
• Support running in Docker containers
• Write util scripts to, build model and initiate server independently (including docker)
• Add an option to host model on a remote server with an accessible endpoint