Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run knnPerfTest.py in nightly benchmarks #316

Open
mikemccand opened this issue Nov 13, 2024 · 0 comments
Open

Run knnPerfTest.py in nightly benchmarks #316

mikemccand opened this issue Nov 13, 2024 · 0 comments

Comments

@mikemccand
Copy link
Owner

We have this awesome tool to measure Lucene's KNN performance (indexing time, index size, searching time, recall vs brute force, etc.).

It's been really helpful in uncovering issues (e.g. broken int8 quantization) and measuring the tradeoff of different quantization approaches.

It has limitations (e.g. cannot yet test simple 1-bit quantization nor better binary quantization (BBQ), it doesn't report "effective hot searching RAM", etc.) but it's already very useful so let's just run it in nightly benchy (lay the spider web) and make pretty charts and hey maybe we catch a fly at some point (e.g. if recall surprisingly drops)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant