You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add support for byte vector storage in the OpenSearch vector store integration, leveraging OpenSearch 2.17's new byte vector capabilities with the Faiss engine. This would allow users to store vectors more efficiently by using 8-bit integers (-128 to 127) instead of floats, significantly reducing storage requirements while maintaining search quality.
Reason
Currently, the OpenSearch integration in LlamaIndex only supports float vectors. Adding byte vector support would require:
Adding a new data_type parameter to OpensearchVectorClient
Updating the index creation and query logic to handle byte vectors
Value of Feature
Storage Efficiency
Byte vectors require 1/4 the storage space of 32-bit float vectors
Reduced storage costs for large-scale deployments
Faster network transfer for distributed systems
Performance Benefits
Potentially faster similarity search due to reduced memory bandwidth requirements
More efficient cache utilisation
Better scalability for large vector databases
The text was updated successfully, but these errors were encountered:
Feature Description
Add support for byte vector storage in the OpenSearch vector store integration, leveraging OpenSearch 2.17's new byte vector capabilities with the Faiss engine. This would allow users to store vectors more efficiently by using 8-bit integers (-128 to 127) instead of floats, significantly reducing storage requirements while maintaining search quality.
Reason
Currently, the OpenSearch integration in LlamaIndex only supports float vectors. Adding byte vector support would require:
Value of Feature
Storage Efficiency
Performance Benefits
The text was updated successfully, but these errors were encountered: