Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark: rocksdb concurrent reads/writes #1667

Closed
wants to merge 3 commits into from
Closed

Conversation

unnawut
Copy link
Contributor

@unnawut unnawut commented Jul 23, 2020

Opening this PR just for reference so we can copy & paste to do more benchmarking in the future.

This is done on my local machine so the numbers are much lower than what we're seeing on deployed environments. But some findings:

  1. Rocksdb reads and writes do interleave. But we route through OMG.DB.RocksDB.Server which handles one message at a time and so is probably the reason why our read/write operations are blocking.
    • Implement read/write as async tasks into this GenServer?
    • Reads could also be turned into a stream I think.
  2. Within OMG.DB.utxos(), the read iterator and decoder takes most of the time. The decoder takes most of the memory.
  3. Tried rocksdb snapshots, with or without, doesn't significantly impact concurrent reads/writes (see results)
    • Ignore rocksdb snapshots for now
  4. Doesn't look like any caller needs the reversed utxo list this would shave off a few milliseconds

@unnawut
Copy link
Contributor Author

unnawut commented Jul 23, 2020

Not intending to merge this but want to keep the files for the record.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant