0.11.0

Release notes:

• Faster hashing of the meta tables thanks to @BrennenMM7!
  • Old databases are backwards compatible with the new meta page format for now. Be sure to upgrade to this version from 0.10 before moving on, in case backwards compatibility is removed in the future.

0.10.0

This release fixes pointer alignment issues that cause panics in debug builds with Rust 1.70+.

See #28 for more details!

0.9.0

This release adds some nice-to-have features to jammdb, which should make it easier to use and potentially more performant.

• Buckets now have a range method which allows you to iterate over a specific range of keys
• OpenOptions support two new flags to tune the database:
  • mmap_populate
  • direct_writes
  • You can read more about these ^ in the docs

I'll probably be spending my time for the next little while working on a new sister crate to this one, announcement coming soon!

0.8.0

TL;DR: this release fixes lots of bugs and is better in basically every way than previous releases. Read on for a fun tale.

First, here's a list of some of the most important changes:

• Rewrote the entire code base from scratch, getting rid of tons of unsafe code.
• Fixed lifetimes of public structs so they cannot outlive a transaction
• Renamed Transaction to Tx because I got sick of typing all those characters
• Fixed commit logic to minimize random seeks and fix a bug where data we needed to move to another page was overwritten before we could write it, leading to data corruption. This is accomplished by flushing all dirty pages to memory before writing anything to disk.
• Added an optional strict mode to ensure database is valid before finishing a write transaction
• Introduced a new Bytes enum and ToBytes trait that keeps track of your data’s lifetime so we can just keep an immutable reference to it. It can also take ownership the bytes if you pass in a Vec, a String, or a bytes::Bytes. So if your &[u8] doesn’t outlive the transaction, you can choose to make a copy of it yourself rather than the database always making a copy like it used to.
• Fixed so many bugs. Wrote intense randomized tests with a recording option so I could playback any failures and zero in on the issue. Node merging and splitting logic were especially infested, but we now have tests that play back over those specific edge cases that were causing trouble before.
• • more things that I can't remember at this point 😅

This release is so big I just had to skip 0.7.0 and go straight to 0.8.0! This release represents a complete rewrite of the entire codebase, and in order to explain why that was necessary I need to tell a quick story.

In late 2019 my second daughter was born and my sleep schedule got all messed up. I found myself staying up until 1 or 2 most nights trying to keep her asleep while my wife slept, and I decided to use this time to finally learn Rust. I had played around with it a little bit, but this was my first major project using the language. I also wanted to understand better how databases are organized on disk, so I figured writing a database in Rust would be the best way to do that!

I had used boltdb a few times, and found the code simple enough for me to pretty quickly wrap my head around how it works while also being powerful enough to be used in big projects like etcd. As I started porting the code to Rust, I quickly ran into issues trying to do things in Rust that are perfectly reasonable to do in Go, like keep pointers to parent structs in the children, and in my sleep-deprived state I devised a solution called the Ptr.

Previous releases of jammdb used this Ptr type to store raw pointers to various other types and dereference them wherever we want, basically replicating what we can do in other languages, but bypassing Rust's wonderful safety rules. I knew it was bad, but I didn't know Rust well enough to fix it. It worked fine in many cases, but as people started trying to use it in real-world situations I started getting issues that I could tell were caused by this very unsafe behavior. I didn't have a ton of time to fix the issues though, so it just sat on the back burner for a while.

Fast forward to the end of 2022, my third daughter was born and I found myself in a similar lack-of-sleep situation. I'm getting even more issues about the database and I decide to try and finally fix them. However, my more-intelligent-and-Rust-experienced self immediately recoiled to the Ptr type, and I decide that if I'm going to fix these issues I've got to get rid of it. Unfortunately it was so entangled into all of the logic, that the only way to fix it was to rewrite the whole thing. So I did! This allowed me to find and squash lots more bugs, and I am much more confident in the result than I was before. But just in case, if you want to use this for something important, just use the new strict mode to make sure any writes to the database won't corrupt everything.

I'm planning on writing a blog series that explains how jammdb works in detail, which should help anybody who wants to contribute understand things more quickly, so keep an eye out for that too.

0.6.0

bump version to 0.6.0

0.3.0

• Added get_or_create_bucket method 🎉
• Made BucketRef which wraps a bucket so you can have references to multiple buckets from a single transaction.

0.2.0

• put method returns the existing key / value pair
• added a get_kv method to get a key / value pair rather than a Data enum

0.1.0 First Release!

Initial release of jammdb!