Multiple versions of lmdb npm package

[wardpeet]

With the plugin ecosystem on Gatsby we want to ship some utils that use lmdb as well. The problem we see with the node ecosystem is that different plugins can have different versions of our utility and thus different versions of lmdb reading/writing to the same database. Would multiple versions of lmdb cause problems? And are different lmdb versions allowed to read/write?

[kriszyp]

There are a few different aspects to this. First, as far as the file format and format of the data, lmdb-js has used the same file format, entry structure, and MessagePack as the default encoding since 1.0 to the current version, so that should be interoperable. Of course it is always possible that something might warrant a change, but presumably would be accompanied by some adapter.

Perhaps a little more complicated as aspect of this is that LMDB only allows a single instance of the LMDB environment to be opened per database file from a process at once (see http://www.lmdb.tech/doc/index.html). lmdb-js actually has a mechanism to keep a list of opened LMDB environments to ensure the underlying LMDB environments are shared between threads when accessed from multiple threads. I think I would need to extend that mechanism to have a global reference to this list that could be shared between LMDB packages.

Anyway, I will look into this and do some testing to see if this would work out, but seems like it should.

[wardpeet]

Thank you but if it bloats the scope of the project and is a nieche feature. There are other ways to accomodate this. By exporting lmdb from gatsby itself and using that. I wanted to make sure it was necessary and it seems like it is so you can ignore this feature unless you have other use cases for this.

[kriszyp]

I think this is a reasonable thing to try to support; in general I think a library should be able to gracefully handle multiple versions running at once.