A few requests #109
andy-thomason
started this conversation in
Ideas
Replies: 1 comment
-
Sorry @andy-thomason, I somehow missed this. This is a great idea, and will see if it can be done! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have used arrow files and indexed databases in genetics research for many years.
It is especially useful to have schemas with nested List of Struct components which this derive
supports for serialization quite well, but doing random reads from record batches would also
be useful. Memory mapping makes arrow files very powerful and we used terabyte sized
files to store indexed genetic variant data.
It would be useful to have a third "ref" mode where we build structs of array references:
Dictionaries are also useful and the MutableDictionaryArray wrapper makes
it easy to build them. We can custom build ArrowSerialize wrappers for primitives which do this
but an attribute might be worth it.
Beta Was this translation helpful? Give feedback.
All reactions