Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Max key length is hardcoded to Byte.MAX_VALUE (127) #26

Closed
akhodakivskiy opened this issue Feb 17, 2019 · 2 comments
Closed

Max key length is hardcoded to Byte.MAX_VALUE (127) #26

akhodakivskiy opened this issue Feb 17, 2019 · 2 comments
Labels
duplicate This issue or pull request already exists

Comments

@akhodakivskiy
Copy link

HaloDBInternal.put has a hard check of the key length: https://github.com/yahoo/HaloDB/blob/master/src/main/java/com/oath/halodb/HaloDBInternal.java#L188

Is there a hard limit of 127 bytes on keys by design? Or is it intended to have longer keys using the fixedKeySize option? Currently it seems to be ignored altogether.

@scottcarey
Copy link

scottcarey commented Nov 14, 2019

It just throws an exception and breaks if you go over 127, I don't see any reason in the code other than inefficiency (keys are written to disk twice and must be resident in memory) that the limit exists.

I suppose at that size you could substitute a ~ 16 to 24 byte hash of the key instead. If all you need is lookup, a long enough hash should be fine. But I have run into this limit in cases where I also need to iterate over key/value pairs, and needed another indirection (map from hash to real key in another db, allowing slow scans and wasting memory). This was too slow so I went back to another storage engine for that use case.

@wangtao724 wangtao724 added the duplicate This issue or pull request already exists label Dec 26, 2019
@wangtao724
Copy link
Contributor

Mark this as duplicate to #42

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

3 participants