Relayer attestation index

[sideninja]

Problem to Solve

Right now we have a high query load from the relayer to the attestation module to query for attestations at specific offset. We are aiding this load with some caching to lower the range we have to search but at the end of the day the problem is we don't have a mapping between attestation offset and consensus block height and we can only query attestation module at a specific snapshot defined by consensus height (due to attestations being pruned from db).
This means there's a lot of guess work (binary search etc) done and a lot of queries produced. Further more this adds a lot of complex bug prone code.
Workers each use this search on their own to find specific attestation offsets defined by stream, which makes the problem worse as it multiplies the queries.

Proposed Solution

An alternative design decission would be to divide areas of concerns into two. We leave workers relaying messages to on-chain the same, but we divide the source where the stream reads, instead of querying the attestation module we introduce a new componenet that follows consensus chain and loads all attestation as they happen (going through each consenssus block height and fetching all attestations), then that componenet writes the attestations in a local persistent index (storage).
The workers then use this as a source of attestations, and we make it so it's easy to query for next attestation offset. The streamer would stay the same it would just load attestations from this index.
So we would have a single process make a single query per consensus block height instead of multiplying queries by workers. We would also greatly simplify the design and make nice separation of concerns. Further more it would allow nicer testing of workers and attestation follower.

[kc1116]

So basically in your proposal you are proposing we build what seems to be a "light client" "consensus follower" type of sub system?

[sideninja]

yeah, one is following and filliing persistent index and others are streaming from that location only.