Wonder if NATS KV and Obj store fits the pattern for Cayley ?

[gedw99]

Hey @dennwc

I was looking at Cayley and initially looked for an S3 store so that i don't have to mess around with scaling / HA.

I need to store a ton of schema info found and stored as JSON-LD and found this:
https://github.com/gleanerio/gleaner
https://github.com/gleanerio/nabu is the API to the Graph Store and they use a Java thing which i don't want to do to myself. Sure you know what i mean ...

But then i realised that NATS Jetstream probably has what is needed, and that scales out naturally.

NATS has these 2 stores.

https://docs.nats.io/using-nats/developer/develop_jetstream/kv
https://docs.nats.io/using-nats/developer/develop_jetstream/object

Under the hood they are just streams in reality and so they are reactive, which leads to other things that cayley likely does not need.

NATS also has some nice OPS tools to make life easy like https://github.com/nats-io/nack, so its a nice golang based eco system for cayley.

I don't know if you have usd NATS, but i would be happy to help out if you feel that its a worthy store for Cayley.
I have a look at the Cayley store API and its looks very flexible.

I suspect it might be an amazing thing because a ton of NATS users are always banging their heads on the brick wall of where to store state, and typically put a DB somewhere being fed by nats. Cayley being a Graph DB is flexible and is often a good fit for these Message Brokers where events go in and data is materialized into a store NUT the data shape is not fixed and so hence a graph is a really good match.

[dennwc]

Hmm, I must admit I never thought of NATS in the context of the store for Cayley. But quickly looking into the KV interface they provide - it seems OK-ish and I don't see any reasons why it wouldn't work.

For now my only concern is the Keys method - it assumes it can return all keys in one go, which won't be the case with large stores. But I guess they have things like "scan by prefix", or limit the maximal number of keys. In that case we can implement pagination and properly iterate over a large list of these.

I can't guarantee that I will look into it any time soon, but I could definitely review the new implementation.