Releases: lennartpollvogt/ollama-instructor
v0.5.1
Bug fix
This release fixes a bug reported with issue #11
The bug occurred when a retry was executed due to an invalid response from the LLM.
As the Ollama server expects string objects for content in the messages list, the method prepare_messages of BaseOllamaInstructorClient was falsely handing over python objects for the retry.
It's recommended that everyone using version v0.5.0 updates to v0.5.1
Note:
What changed?
- Due to new version 0.4.1 of
ollamaI had to bump the pydantic version to^2.9.0as this is a dependency ofollama. So I had to refactor the functioncreate_partial_modelincleaner.py - Refactored the type hints regarding the
messagesargument.
Warnings!
- Unfortunately, with the new versions 0.4.1 of
ollamathe methodchat_completion_with_streamofOllamaInstructorClientis broken. I will try to finde a fix. I marked the method asdeprecated. Please usechat_completioninstead. Older versions ofollama-instructorwith usingollamaversions below 0.4.0 should work. I am sorry for the inconvenience!
v0.5.0
What changed?
- Separated the
OllamaInstructorBaseClientfromollama_instructor_client.pyintobase_client.py - Separated the functionality of methods like
handel_responseofOllamaInstructorBaseClientinto easier maintainable methods - Switched from
icecreamto pythonsloggingmodule for debugging. Deleted dependency foricecream. - Added the arguments
keep_aliveandoptionstochat_completionandchat_completion_with_stream(no more**kwargs) - Added support for
ollama-pythonMessageandOptionsclasses for code completion within IDE/Editor - Added example for multiple asynchronous operations (see here )
- new version v0.5.0
Details on switch from icecream to logging
As announced in the previous README.md I was planning to switch from the icecream module to pythons built-in logging module. When setting debug = True in the classes OllamaInstructorClient and OllamaInstructorAsyncClient you will now see something like the following:
2024-10-19 16:02:47,851 - ollama_instructor.base_client - DEBUG - ollama_instructor_client.py:369 - Reset state
2024-10-19 16:02:47,851 - ollama_instructor.base_client - DEBUG - ollama_instructor_client.py:371 - Create prompt
2024-10-19 16:02:47,852 - ollama_instructor.base_client - DEBUG - ollama_instructor_client.py:374 - Start while loop
2024-10-19 16:02:47,852 - ollama_instructor.base_client - DEBUG - ollama_instructor_client.py:379 - Prepare message and set left retries
2024-10-19 16:02:47,852 - ollama_instructor.base_client - DEBUG - ollama_instructor_client.py:381 - Request ollama client
...
Details on code completion for messages and options arguments
Code completion support for messages:
Code completion support for options:
Full Changelog: v0.4.2...v0.5.0
v0.4.2
What changed?
- New version
0.4.2
Bugfix
- added a
reset_statemethod toOllamaInstructorBaseClientclass. This is used with every request of the methodschat_completionandchat_completion_with_stream(sync + async). Otherwise the statesretry_counter,validation_errorandchat_historywould have the value of the last request which was leading into an exception/failing request. Now you can use the sameclientfor instance within afor- orwhile-loop and not having to create a newclientwith every request.
Example:
from ollama_instructor.ollama_instructor_client import OllamaInstructorClient
from pydantic import BaseModel
from enum import Enum
from typing import List
import rich
class Gender(Enum):
MALE = 'male'
FEMALE = 'female'
class Person(BaseModel):
name: str
age: int
gender: Gender
client = OllamaInstructorClient(host='http://localhost:11434')
list_of_text = [
"Jason is a 21 years old man.",
"Jane is a 31 years old womand.",
"Bob is a 35 years old man."
]
list_of_response = []
for text in list_of_text:
try:
response = client.chat_completion(
model='llama3.1:latest',
pydantic_model=Person,
messages=[
{
'role': 'user',
'content': text
}
],
format='json',
)
list_of_response.append(response['message']['content'])
except Exception as e:
pass
print(list_of_response)This was not possible before, due to missing reset of the states, when using the same client.
Documentation
- added in blog a creative story about
ollama-instructors intention and approach (with time there might be some pictures added ;-) ) - added the example Todos from Conversation for automatic todos creation from chat conversations or transcripts
- added a Best practices to the docs for having a place to quickly add them without writing bigger documentations. Will be extended or parts of it transferred into detailed documentation.
Tests
- added more tests
v0.4.1
What changed?
- New version
0.4.1 - Retry logic was fixed: The retry logic with the change in version 0.4.0 was not working correct. With the new version
0.4.1this was fixed. - Enhanced error guidance, when no code block was found in response: Firstly, if a code block is missing the
handle_responsefunction will now create a partial response where all values are set toNone. This partial response will be used for the validation and to append thechat_historyto create a proper error guidance. Secondly, the error guidance prompt itself was enhanced. - More test cases: More test cases where added in the tests folder to test the functionality of the code block extraction and response handling
- The documentation about the features of
ollama-instructorwas enriched with a detailed explanation how thereasoningfeature works (see docs)
Full Changelog: v0.4.0...v0.4.1
v0.4.0
New feature
With this release comes with a new feature, where you can use the format = '' in the chat_completion of OllamaInstructorClient and OllamaInstructorAsyncClient. This allows the LLM and vision model to reason before it response with the JSON output.
How is this done?
By setting format = '' the LLM is not forced to respond within JSON. Therefore, the instructions are crucial, to get a JSON anyway. But with format = '' the capabilities of LLMs to reason step by step before answering can be used. When you set format = '' the LLM will be instructed differently (have a look into the code in file 'prompt_manager.py'). After the step by step reasoning the LLM is instructed to respond within a code block starting with json and ending with . The content of this code block will be extracted and validated against the Pydantic model. When comments within the JSON code block occur, the code tries to delete them.
Why?
Letting the LLM reason before the final output seems to lead into better responses.
Example:
Here is an example for the usage:
import rich.markdown
from ollama_instructor.ollama_instructor_client import OllamaInstructorClient
from pydantic import BaseModel
from typing import List
from enum import Enum
import rich
class Gender(Enum):
male = "male"
female = "female"
other = "other"
class User(BaseModel):
name: str
email: str
age: int
gender: Gender
friends: List[str]
client = OllamaInstructorClient(
host='http://localhost:11434',
debug=True
)
response = client.chat_completion(
pydantic_model=User,
model='mistral',
messages=[
{
'role': 'user',
'content': 'Extract the name, email and age from this text: "My name is Jane Smith. I am 25 years old. My email is [email protected]. My friends are Karen and Becky."'
},
],
format='',
allow_partial=False
)
rich.print(response)
from rich.console import Console
from rich.markdown import Markdown
console = Console()
md = Markdown(response['raw_message']['content'])
console.print(md)Output of the reasoning which is stored in response['raw_message']['content']:
Task Description: Given a JSON schema, extract the named properties 'name', 'email', 'age' and 'gender' from the provided text and return a valid JSON response adhering to the schema.
Reasoning:
1 'name': The name is mentioned as "My name is Jane Smith".
2 'email': The email is mentioned as "My email is [email protected]".
3 'age': The age is mentioned as "I am 25 years old".
4 'gender': According to the text, 'Jane' is a female, so her gender would be represented as 'female' in the JSON response.
5 'friends': The friends are mentioned as "Her friends are Karen and Becky", but according to the schema, 'friends' should be an array of strings, not an object with a title. However, the schema does not explicitly state that each friend must have a unique name, so both 'Karen' and 'Becky' could be included in the 'friends' list.
JSON response:
´´´json
{
"name": "Jane Smith",
"email": "[email protected]",
"age": 25,
"gender": "female",
"friends": ["Karen", "Becky"]
}
´´´
Output of the extracted JSON within response['message']['content']:
{
"name": "Jane Smith",
"email": "[email protected]",
"age": 25,
"gender": "female",
"friends": ["Karen", "Becky"]
}
Note: This feature is currently not available for
chat_completion_with_stream. I try to make this happen.
Fixes
Important
Due to the last refactoring it turns out the retry...
v0.3.0
What is new?
This new version contains an updated ollama_instructor_client.py. The code was refactored to get rid of redundancies within the two clients OllamaInstructorClient and OllamaInstructorAsyncClient. The new OllamaBaseClient contains the redundant code for the derived clients. The usage of this library is not affected.
The version of ollama-instructor was bumped to 0.3.0 and supports the new version of ollama (python) which is 0.3.0.
A new folder "blog" was added to the respository. There will be blog articles about my thoughts and ideas regarding Ollama-instructor. The README.md was updated to display this.
List of announcements:
- Refactoring the clients of
ollama-instructorin ollama_instructor_client.py to make maintenance easier and reduce redundancies in code (see commit) - Supporting new ollama version 0.3.0 (see commit)
- New "blog" folder for publishing thoughts and ideas regarding ollama-instructor and its usage (see commit)
- Updated README.md file (see commit)
- Some smaller refactorings to docstrings of functions and tests (see commit)
Full Changelog: v0.2.0...v0.3.0
v0.2.0
New Feature
- new class
OllamaInstructorAsyncClientto support asynchronous operations (currently onlychat_completionis supported) - added tests for helper functions from
cleaner.py - added test for synchronous
chat_completionand asynchronouschat_completion - added description and example code for asynchronous
chat_completionin README.md - updated Roadmap.md
- updated version to 0.2.0
Full Changelog: v0.1.3...v0.2.0
v0.1.3
- README.md was updated to mention the dependency of running Ollama server when using
ollama-instructor - New markdown file Enhanced prompting within Pydantics BaseModel.md for enrich libraries documentation
- New markdown file Features of ollama-instructor.md to emphasize the libraries benefits.
- smaller refactoring on test collections for
chat_completionandchat_completion_with_stream.
Full Changelog: v0.1.2...v0.1.3
v0.1.2
- fix of init.py to provide usage of the library (was not working with the v0.1.1 release)
- Adjusted documentation OllamaInstructorClient and its methods chat_completion and chat_completion_with_stream
- Deleted method chat_completion_with_stream_old
- Adjusted documentation README.md and /docs/README.md for correct usage of library
- Adjusted validation_manger.py to correctly import json_parser.core.options
Full Changelog: v0.1.1...v0.1.2
Release version "0.1.1"
With this release the version "0.1.1" gets published.
- README.md: specifying arguments the
OllamaInstructorClientclass and of its methodschat_completionandchat_completion_with_streamas well. - init.py: providing the
OllamaInstructorClientclass and the methods ofcleaner.py - poetry: setting up poetry for python packaging and version control
Full Changelog: https://github.com/lennartpollvogt/ollama-instructor/commits/v0.1.1