Handle meta events

[roman-khimov]

Is your feature request related to a problem? Please describe.

A part of #2878.

Describe the solution you'd like

Add meta event handling:

• create a new meta DB (single instance per node), reuse https://pkg.go.dev/github.com/nspcc-dev/[email protected]/pkg/core/storage for it
• forget about buckets, use prefixes for everything
• listen for events, catch and process them
• for each block open a new MemCachedStore, process in-block data immediately
• in-block data allows to build an index of OIDs and big/lock indexes
• these prefixes are added into https://pkg.go.dev/github.com/nspcc-dev/[email protected]/pkg/core/mpt when we're done with block processing (keep only latest state, we don't need historic)
• a full header index (see Implement searchv2 #3058) can be managed in a separate routine and not be a part of MPT
• mem cache flushing can be per-block currently

Describe alternatives you've considered

There can be some, but we need this data to be processed fast and we'll need MPT for subsequent synchronization.

[roman-khimov]

@AnnaShaleva and @AliceInHunterland can tell a bit more about the way storage is handled in NeoGo. What we're doing here is very similar to storeBlock except that we work with events and some data needs to be obtained additionally (but it's not a part of the state proper at the same time).

[carpawell]

@roman-khimov

1. So we have a new DB now for our indexes (not sure why it is a reuse of neo-go's one but ok, dont mind) and also have a MemCachedStore memory instance for every block, why? That is exactly "cached" version, not just a memory instance per block, right? Over what is it created, the original disk DB? What does the cache solve?
2. "listen for events", "for each block". What is this new DB subscribed for? I thought that it should receive only our new events from meta-on-chain TXs but it also needs to process every block?
3. "these prefixes are added into https://pkg.go.dev/github.com/nspcc-dev/[email protected]/pkg/core/mpt". What exact "these prefixes"? Why we need mpt here?
4. "full header index ... not be a part of MPT" what is the criterion of what mpt and what is not?

[roman-khimov]

1. It can be done without MemCachedStore using per-block Bolt transactions, but reusing NeoGo storage subsystem allows to simplify some interfaces and make DBs changeable. In future we can also optimize flushing from per-block to something smarter. Memory itself is irrelevant here.
2. Listen for new events, you need to watch for objects, that's the only thing you care about (epochs are interesting too, but maybe not immediately).
3. MPT is to identify and synchronize state. Prefixes are those created above, not the full index.
4. Reproducibility. I can fetch headers and recreate index using data stored in MPT, so adding everything into MPT is a resource waste.