Easy text and chat completion, as well as function calling. Also includes useful utilities for counting tokens, composing prompts and trimming them to fit within the token limit.
pip install easycompletionfrom easycompletion import function_completion, text_completion, compose_prompt
# Compose a function object
test_function = compose_function(
name="write_song",
description="Write a song about AI",
properties={
"lyrics": {
"type": "string",
"description": "The lyrics for the song",
}
},
required_properties: ["lyrics"],
)
# Call the function
response = function_completion(text="Write a song about AI", functions=[test_function], function_call="write_song")
# Print the response
print(response["arguments"]["lyrics"])easycompletion has been tested with LocalAI LocalAI which replicates the OpenAI API with local models, including Llama v2.
Follow instructions for setting up LocalAI and then set the following environment variable:
export EASYCOMPLETION_API_ENDPOINT=localhost:8000You can very easycompletion logs by setting the following environment variable:
export EASYCOMPLETION_DEBUG=TrueYou can compose a prompt using {{handlebars}} syntax
test_prompt = "Don't forget your {{object}}"
test_dict = {"object": "towel"}
prompt = compose_prompt(test_prompt, test_dict)
# prompt = "Don't forget your towel"Send text, get a response as a text string
from easycompletion import text_completion
response = text_completion("Hello, how are you?")
# response["text"] = "As an AI language model, I don't have feelings, but...""Compose a function to pass into the function calling API
from easycompletion import compose_function
test_function = compose_function(
name="write_song",
description="Write a song about AI",
properties={
"lyrics": {
"type": "string",
"description": "The lyrics for the song",
}
},
required_properties: ["lyrics"],
)Send text and a list of functions and get a response as a function call
from easycompletion import function_completion, compose_function
# NOTE: test_function is a function object created using compose_function in the example above...
response = function_completion(text="Write a song about AI", functions=[test_function], function_call="write_song")
# Response structure is { "text": string, "function_name": string, "arguments": dict }
print(response["arguments"]["lyrics"])Composes a function object for function completions.
summarization_function = compose_function(
name="summarize_text",
description="Summarize the text. Include the topic, subtopics.",
properties={
"summary": {
"type": "string",
"description": "Detailed summary of the text.",
},
},
required_properties=["summary"],
)chat_completion(text, model_failure_retries=5, model=None, chunk_length=DEFAULT_CHUNK_LENGTH, api_key=None)
Send a list of messages as a chat and returns a text response.
response = chat_completion(
messages = [{ "user": "Hello, how are you?"}],
system_message = "You are a towel. Respond as a towel.",
model_failure_retries=3,
model='gpt-3.5-turbo',
chunk_length=1024,
api_key='your_openai_api_key'
)The response object looks like this:
{
"text": "string",
"usage": {
"prompt_tokens": "number",
"completion_tokens": "number",
"total_tokens": "number"
},
"error": "string|None",
"finish_reason": "string"
}text_completion(text, model_failure_retries=5, model=None, chunk_length=DEFAULT_CHUNK_LENGTH, api_key=None)
Sends text to the model and returns a text response.
response = text_completion(
"Hello, how are you?",
model_failure_retries=3,
model='gpt-3.5-turbo',
chunk_length=1024,
api_key='your_openai_api_key'
)The response object looks like this:
{
"text": "string",
"usage": {
"prompt_tokens": "number",
"completion_tokens": "number",
"total_tokens": "number"
},
"error": "string|None",
"finish_reason": "string"
}function_completion(text, functions=None, system_message=None, messages=None, model_failure_retries=5, function_call=None, function_failure_retries=10, chunk_length=DEFAULT_CHUNK_LENGTH, model=None, api_key=None)
Sends text and a list of functions to the model and returns optional text and a function call. The function call is validated against the functions array.
Optionally takes a system message and a list of messages to send to the model before the function call. If messages are provided, the "text" becomes the last user message in the list.
function = {
'name': 'function1',
'parameters': {'param1': 'value1'}
}
response = function_completion("Call the function.", function)The response object looks like this:
{
"text": "string",
"function_name": "string",
"arguments": "dict",
"usage": {
"prompt_tokens": "number",
"completion_tokens": "number",
"total_tokens": "number"
},
"finish_reason": "string",
"error": "string|None"
}Trim the given text to a maximum number of tokens.
trimmed_text = trim_prompt("This is a test.", 3, preserve_top=True)Split the given prompt into chunks where each chunk has a maximum number of tokens.
prompt_chunks = chunk_prompt("This is a test. I am writing a function.", 4)Count the number of tokens in a string.
num_tokens = count_tokens("This is a test.")Returns a list of tokens in a string.
tokens = get_tokens("This is a test.")Composes a prompt using a template and parameters. Parameter keys are enclosed in double curly brackets and replaced with parameter values.
prompt = compose_prompt("Hello {{name}}!", {"name": "John"})You can pass in a model using the model parameter of either function_completion or text_completion. If you do not pass in a model, the default model will be used. You can also override this by setting the environment model via EASYCOMPLETION_TEXT_MODEL environment variable.
Default model is gpt-turbo-3.5-0613.
You can pass in an API key using the api_key parameter of either function_completion or text_completion. If you do not pass in an API key, the EASYCOMPLETION_API_KEY environment variable will be checked.
bash publish.sh --version=<version> --username=<pypi_username> --password=<pypi_password>If you like this library and want to contribute in any way, please feel free to submit a PR and I will review it. Please note that the goal here is simplicity and accesibility, using common language and few dependencies.
If you have any questions, please feel free to reach out to me on Twitter or Discord @new.moon