Kv store stats #93
Kv store stats #93
Conversation
|
Looks good, thanks! Just those two things I've noticed and noted above... |
At the risk of showing my github ignorance, I don't see where you noted two things above. There don't appear to be any comments in the code. Am I missing something? |
Sorry, my fault :) There's the "submit review" button (that I feel is a bit hidden) that I always forget to push... |
The "sidctl stats" command prints various memory usage stats from the key-value store. It shows total size of the keys, the hash metadata (including the keys), the internal and external values, and the number of key value pairs. The value stats show both the total size of the memory used to hold the values, including metadata, and the size of just the value data itself.
bmarzins
force-pushed
the
kv-store-stats
branch
from
0effde9 to
e52afb1
Compare
|
OK, thanks, will merge this! Though, now I recall that I wanted the stats for #87 at first, but making the stats possible on demand for the whole DB through As for the optimization mentioned in #87 - the point there was to avoid iterating over all records to be able to tell how much shared memory to allocate for us to send the part of the database - in that exact case, the diffs for syncing with main db. Instead, we'd now instantaneously how much memory we're going to use for syncing. To do that, we'd need to track the memory consumption directly in the low-level But this won't be probably that straightforward, because we'd need to be able to filter just the records which have cetrain combination of flags/properties set. These flags/properties are the ones in the "wrapping" layer, not the "low-level" layer, but still we need to be able to tell the overall memory consumption, including both the "low-level" and "wrapper". So I think this is more about indexing then - indexing all records which have XYZ flag/property set and then being able to get the subset quickly. And then using the iterator on top of the results, counting the memory use - simply, the goal would be to get certain subset of records based on defined filter quickly, not iterating over all existing records in the whole db. That filter might be defined even in advance, like in the case with sending db diffs from worker process to main process ( |
Here is the sidctl stats command, and other minor cleanups. If you want different stats reported, just let met know.