Skip to content

v0.4.0

Compare
Choose a tag to compare
@mwlon mwlon released this 10 Nov 18:59
· 10 commits to main since this release
f48e7b8
  • New format version (3)
  • API breaking changes
    • All metadata types (e.g. ChunkMeta) now live in the pco::metadata:: module.
    • Metadata and ChunkCompressor types are no longer generic to latent type. Instead they contain dynamically-typed values, leveraging the dtype_dispatch crate.
    • Renamed NumberLike => Number.
    • Delta encoding is now configured via DeltaSpec enum of an Option<usize>, and similarly ChunkMeta contains a DeltaEncoding enum.
  • API additions
    • Added simple_compress_into function.
    • Added a Lookback delta encoding that improves compression ratio on some types of data.
  • Improvements
    • Improved bin optimization speed on Windows slightly.
    • Improved FloatQuant auto mode heuristics, improving the size/speed tradeoff.
    • Reduced binary size via fewer unnecessary specializations.
  • Degradations
    • Automatic detection of the new Lookback delta encoding is fairly slow, often increasing compression time ~10%. We're likely to try ameliorating this in future releases.