Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verify Indexes, offline rebuild of keystore #160

Open
martinsumner opened this issue Aug 9, 2018 · 2 comments
Open

Verify Indexes, offline rebuild of keystore #160

martinsumner opened this issue Aug 9, 2018 · 2 comments

Comments

@martinsumner
Copy link
Owner

See martinsumner/kv_index_tictactree#24

Need to make sure simple leveled functionality exists to verify indexes, and prompt a keystore to be rebuilt whilst a store is offline.

For the first part need to return a folder that will first fold objects (folding over the journal not the ledger) building an AAE tree for index entries of active objects. Then it should fold over all index entries in the Ledger providing an AAE tree to compare, and then compare the AAE trees - and log a warning if they don't compare. Use SnapPreFold = true for both folds, so they can be taken at a consistent point in time.

For the second part need to be able to call book_offlinerebuild(RootPath, ArchivePath), and have that archive the old Ledger, and then open the store with an empty Ledger, and then close again once it is rebuilt.

@martinsumner
Copy link
Owner Author

The object fold should fold over the ledger not the Journal, as:

  • The objects in the ledger are already eventually consistent through anti-entropy, so we can rely on their contents;

  • The contents of the HEAD object include the index entries (as part of the SibMetaBin).

@martinsumner
Copy link
Owner Author

It would be good if we could do a partial offline repair here. The index compare fold, would persist a list of broken segment IDs under a GUID, and output this as a log. The offline repair could then be passed the GUID, and fix the index entries that are broken by scanning all the indexes and objects and comparing only those that pass the segment ID filter - and then making only specific index changes.

This would have to be an offline repair, so that issues of concurrent changes to the ledger from the application do not need to be resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant