-
Notifications
You must be signed in to change notification settings - Fork 250
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Dj Walker-Morgan <[email protected]>
- Loading branch information
Showing
2 changed files
with
67 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
66 changes: 66 additions & 0 deletions
66
advocacy_docs/edb-postgres-ai/ai-accelerator/pipelines/models/using-models.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
--- | ||
title: Using Models in AI Accelerator Pipelines | ||
navTitle: Using Models | ||
description: How to register and use models in AI Accelerator Pipelines. | ||
--- | ||
|
||
Pipelines has a model registry that manages configured instances of models. Any Pipelines functions that use models, such as embedding and retrieving, must reference a registered model. Pipelines comes with a set of pre-registerd models that you can use out of the box. | ||
|
||
To find them, you can run the following query: | ||
|
||
```sql | ||
SELECT * FROM aidb.list_registered_models(); | ||
``` | ||
|
||
This will return a list of all the models that are currently registered in the system. If you have not registered any models, you'll see the default models that come with Pipelines. | ||
|
||
```text | ||
name | provider | options | ||
-----------------------+-------------------+----------------------------------- | ||
bert | bert_local | {"config={}"} | ||
clip | clip_local | {"config={}"} | ||
t5 | t5_local | {"config={}"} | ||
dummy | dummy | {"config={}"} | ||
``` | ||
|
||
The `bert`, `clip`, and `t5` models are all registered and ready to use. The `dummy` model is a placeholder model that can be used for testing purposes. | ||
|
||
You can also register your own models. To do this, you can use the `aidb.register_model` function. Here is an example of how to register a model: | ||
|
||
```sql | ||
SELECT aidb.register_model('my_model', 'bert_local'); | ||
``` | ||
|
||
This will register a model named `my_model` that uses the `bert_local` model provider. But, this is essentially the same as using the bert model thats already registered. | ||
|
||
This is where the other [supported models](./supported-models) come in. You can register a different model by specifying the model name in the configuration. The `OpenAI Completions` and `OpenAI Embeddings` models are both models which you can register to make use of OpenAI's completions and embeddings APIs. | ||
|
||
You need to provide more information to the `aidb.register_model` function when registering a model like these. Completions has a number of options, including selecting which model it will use on OpenAI. Both Completions and Embeddings requires API credentials. Here is an example of how to register the OpenAI Completions model: | ||
|
||
```sql | ||
SELECT aidb.register_model( | ||
'my_openai_model', | ||
'openai_completions', | ||
'{"model": "gpt-4o"}'::JSONB, | ||
'{"api_key": "sk-abc123xyz456def789ghi012jkl345mn"}'::JSONB | ||
}; | ||
|
||
You should replace the `api_key` value with your own OpenAI API key. Now you can use the `my_openai_model` model in your Pipelines functions and, in this example, leverage the GPT-4o model from OpenAI. | ||
|
||
You can also register the OpenAI Embeddings model in a similar way. | ||
|
||
```sql | ||
SELECT aidb.register_model( | ||
'my_openai_embeddings', | ||
'openai_embeddings', | ||
'{"model": "text-embedding-3-large"}'::JSONB, | ||
'{"api_key": "sk-abc123xyz456def789ghi012jkl345mn"}'::JSONB | ||
}; | ||
``` | ||
|
||
This will register the `text-embedding-3-large` model with the name `my_openai_embeddings`. You can now use this model in your Pipelines functions to generate embeddings for text data. | ||
|
||
These OpenAI models work with any OpenAI compatible API. This allows you to connect and use an even wider range of models, just by passing the appropriate API endpoint to the `url` option in the `aidb.register_model` function's options. | ||
For more information about the OpenAI models, see the [OpenAI Completions](./supported-models/openai-completions) and [OpenAI Embeddings](./supported-models/openai-embeddings) pages. | ||