Release 1.0.1 #133
Merged
Release 1.0.1 #133
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CLI tests using factgenie list datasets and factgenie list campaigns
fix parse_crowdsourcing_config default empty strings
fix factgenie launch; add ollama service to docker-compose
* add VLLMetric and OpenAIMetric supporting structured decoding * add example for VLLM inference Completely changed parsing LLM-eval annotations Now we use pydantic and enforce structure * rename type to annotation_type for parsing LLM outputs and in LLM prompts
and new functionality factgenie download -d DATASET_ID
Download dataset cli
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Version 1.0.1
First official release 🎉
Changes compared to the pre-release version:
Pip package
We release
factgenieas a PyPI package to make its installation easier.(Note that the recommended way to install factgenie is still as an editable Python package. That setup enables to write custom dataset classes.)
Resolves #3 .
File structure
We reorganized the file structure to make it more intuitive and sustainable. See #120 for more details.
Resolves #120 .
Multiple annotations within a campaign
We enable multiple annotations within a campaign, both during data collection and in the browsing interface.
Resolves #60.
Support for VLLM
We support VLLM as a backend for running LLMs, including constrained decoding for the annotations.
Resolves #75.
Prompt and instruction wizards
We support pre-filling prompts for LLMs and instructions for the crowdworkers based on the annotation span categories.
Resolves #108 .
Better reproducibility
We save more detailed information in the generated outputs to enhance reproducibility.
Resolves #130 .
Minor fixes
We implemented many other minor fixes.
Resolves #111, resolves #144, resolves #150.
Documentation and testing
We completely updated the documentation on wiki and tested factgenie in various setups.
Resolves #50.