You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
New format version (3)
API breaking changes
All metadata types (e.g. ChunkMeta) now live in the pco::metadata:: module.
Metadata and ChunkCompressor types are no longer generic to latent type. Instead they contain dynamically-typed values, leveraging the dtype_dispatch crate.
Renamed NumberLike => Number.
Delta encoding is now configured via DeltaSpec enum of an Option<usize>, and similarly ChunkMeta contains a DeltaEncoding enum.
API additions
Added simple_compress_into function.
Added a Lookback delta encoding that improves compression ratio on some types of data.
Improvements
Improved bin optimization speed on Windows slightly.
Improved FloatQuant auto mode heuristics, improving the size/speed tradeoff.
Reduced binary size via fewer unnecessary specializations.
Degradations
Automatic detection of the new Lookback delta encoding is fairly slow, often increasing compression time ~10%. We're likely to try ameliorating this in future releases.