Skip to content

v0.2.0 - Cache

Compare
Choose a tag to compare
@simedw simedw released this 13 Jul 15:44
· 4 commits to main since this release
7a1c240

Caching

Added two new Evaluators for handling caching, MemoryCache and FileCache.

Using the API:

evaluator = MemoryCache(SemanticEvaluator())

Using the CLI:

$ bench run --cache memory # or `file` or `none`

For commandline caching is on by default.

Match object

Changed the signature of evaluate_prediction from bool to return Optional[Evaluator.Match]
Match carries information about which of the expected values matched with the output from the tested model.
This will likely be extended in the next release with even more information.