Database Macro

[davidjohnbell]

Description

This pull request closes #364. The Serai code-base is full of repetitive database code used to namespace keys. The provided macro generates the database and item DSTs necessary for a new key. The macro supports zero or more scale encodable arguments in the generation of the key and will likewise automatically bincode serialize/de-serialize the database value resulting in better DX and hopefully less bugs.

Considerations

• The macro outputs an empty struct so that methods can be added using an impl block
• bincode was chosen over scale because it supports a wider range of types. It's still possible to store scale bytes in the form of a Vec<u8> and transform later as is the case now
• Many of the existing databases are generic (e.g., Network) which can't be auto generated by the macro. Moving forward, helper methods should receive those generic arguments instead.

Example

create_db!(
  KeyGenDb {
    ParamsDb: (vs: &ValidatorSet) -> ThresholdParams,
    CommitmentsDb: (key: &KeyGenId) -> HashMap<Participant, Vec<u8>>,
    GeneratedKeysDb: (set: &ValidatorSet, key_one: &[u8; 32], key_two: &[u8]) -> Vec<u8>,
    KeysDb: (key: &Vec<u8>) -> Vec<u8>
  }
);

In this case 3 databases are created, but let's focus on ParamsDB. ParamsDB gets a "KeyGenDB" database DST and a "ParamsDB" item DST, whose get and set become:

impl ParamsDB {
  pub fn get(getter: &impl Get, vs: &ValidatorSet) -> Option<ThresholdParams> {
    // brevity
  }
  pub fn set(txn: &mut DbTxn, vs: &ValidatorSet, value: ThresholdParams) {
    //brevity
  }
}

Future Work

The pull request is being opened now so that future work builds off of this macro while we convert old databases and pass tests.

[kayabaNerve]

Why not simply correct the missing import?

[davidjohnbell]

I believe the doc test is compiled in the current scope, but those types I use aren't going to be available, as is the case with the bincode dependency - so correcting the missing import would just lead to more errors 😅

[davidjohnbell]

sorry if I was spamming CI 😊

[kayabaNerve]

Ha, don't worry. We're not on any paid plan.