Skip to content

Commit

Permalink
dbwrapper: Bump max file size to 32 MiB
Browse files Browse the repository at this point in the history
The default max file size for LevelDB is 2 MiB, which results in the
LevelDB compaction code generating ~4 disk cache flushes per second when
syncing with the Bitcoin network.
These disk cache flushes are triggered by fdatasync() syscall issued by the
LevelDB compaction code when reaching the max file size.

If the database is on a HDD this flush rate brings the whole system to a
crawl.
It also results in very slow throughput since 2 MiB * 4 flushes per second
is about 8 MiB / second max throughput, while even an old HDD can pull
100 - 200 MiB / second streaming throughput.

Increase the max file size for LevelDB to 32 MiB instead so the flush rate
drops significantly and the system no longer gets so sluggish.

The new max file size value chosen is a compromise between the one that
works best for HDD and SSD performance, as determined by benchmarks done by
various people.
  • Loading branch information
maciejsszmigiero committed Nov 30, 2024
1 parent dbc8ba1 commit b73d331
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 0 deletions.
1 change: 1 addition & 0 deletions src/dbwrapper.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ static leveldb::Options GetOptions(size_t nCacheSize)
// on corruption in later versions.
options.paranoid_checks = true;
}
options.max_file_size = std::max(options.max_file_size, DBWRAPPER_MAX_FILE_SIZE);
SetMaxOpenFiles(&options);
return options;
}
Expand Down
1 change: 1 addition & 0 deletions src/dbwrapper.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@

static const size_t DBWRAPPER_PREALLOC_KEY_SIZE = 64;
static const size_t DBWRAPPER_PREALLOC_VALUE_SIZE = 1024;
static const size_t DBWRAPPER_MAX_FILE_SIZE = 32 << 20; // 32 MiB

//! User-controlled performance and debug options.
struct DBOptions {
Expand Down

0 comments on commit b73d331

Please sign in to comment.