Use LevelDB for storing events

[davidfokkema]

If the StorageManager needs to keep a lot of events in temporary storage, Redis will fill up all memory and more-or-less crash the Pi. I looked into a lot of Redis-like storage solutions, since I like the high-level methods (like incrementing the counters, popping from the front of a list, etc). I tried SSDB (does not support deleting hashes), Redis-NDS (very little activity), Edis (very little activity), Redis-LevelDB (does not know about hmset or hincrby) and Ardb (hangs on a simple storage test using more than one client).

I'll try keeping Redis, but storing the pickled events using LevelDB.

[davidfokkema]

Or ZODB - a native object database for Python?

[davidfokkema]

Or Vedis? Embeddable Redis, thread-safe, with Python bindings.

[davidfokkema]

Nope. No Vedis. Vedis does not have a rpush command, and does lpush in the wrong order. It also has some longstanding bugs...

[davidfokkema]

SSDB has its own protocol, which does allow for deleting hashes. Then, we have everything in one database. Otherwise, keep using Redis for the queue, and use leveldb (or rocksdb) for the events. Two stores...

[davidfokkema]

Hmmm... ArangoDB or MongoDB both have arrays. ArangoDB is not in Stretch, but MongoDB is.

[davidfokkema]

FaunaDB has arrays. It's from "the team that scaled Twitter".