RocksDB is often used in large scale distributed storage systems. In such systems, many individual RocksDB instances are stored in many thousands of hosts across different data centers. There are RocksDB code/config/data compatibility issues in scenarios like following:
- Upgrade RocksDB code to newer version and reopen the DB.
- Downgrade RocksDB code to a lower version and reopen the DB. This also happens frequently, mostly because of reasons not related to RocksDB.
- Copying data of one DB to another host, which may run different version of RocksDB and/or get a different RocksDB configuration.
We don't think they are solely RocksDB users' problems and we always consider the potential compatibility scenarios when we develop new features in RocksDB.
Here is our goal of handling format compatibility. If users don't change options of the DB, RocksDB's compatibility guarantee when opening a DB generated by a different release is following:
- Backward compatible: newer version of RocksDB should be able to open DBs generated by all previous releases for normal configuration.
- Forward compatible: an older release of RocksDB should be able to open DB generated by releases at least several minor versions higher, if no new features are explicitly enabled by users which are not supported in older versions. The goal is to hold this forward compatibility as long as we can, and we can usually hold it longer than a year.
For example, release 5.6 can open DB generated by version at least 2.2 (4 years old) and version 3.10 (2 years old) can open DB generated using 5.6 if no new feature is used. We have continuous tests to verify that.
Format compatibility solves problem when users never introduce breaking change in their configuration to RocksDB. However, RocksDB may not be able to open a DB if opening with a different configuration, even using the same code. Think about a change of comparator or compaction style.
To solve this problem, RocksDB can materialize current DB options in [[RocksDB Options File]] under DB directory. Before opening a DB, users can choose to read back the options from the DB and use that option to open the DB, which will be compatible. Or, users can verify whether the new options they want to use are compatible with the current DB.
There is known limitations to this feature and we are addressing it. One major problem was that, the backward and forward compatibility guarantee mentioned in the previous section doesn't apply to the option verification tool. For example, if a newer version of RocksDB has a new feature of partitioning metadata blocks, there will be a new record “meta block size = ...” in the option file. Even if users use the default value, which is to turn the feature on, this record cannot be understood by previous RocksDB version and it leaves users from no choice but thinking it is not compatible. As a result downgrading RocksDB version may not work with this feature.
To address this issue, we are adding a new parameter of the option verification tool to skip unidentified options in release 5.6. With this feature, users can at lease open the option files generated using higher version. We are working on addressing other limitations.
Detecting option incompatibility is a big step forward. However, users sometimes need more. They are not always satisfied with RocksDB telling them no. They want RocksDB to make it yes ---- automatically migrate the data if they want to pay the costs.
We built a compaction setting migration tool to help users migrate DB to a different compaction setting. See tool definition. We may improve this tool to include more scenarios if there is a need from users. You are welcome to send pull requests for that, or tell us your request.