Skip to content

How Plugins Work

Jump to bottom
shahar z edited this page Aug 18, 2022 · 12 revisions

NDBench Plugins are implemented by leveraging Guice BindingAnnotations and Guice Extensions

Plugins are registered in NdBench-Core at startup with the help of Guice MapBinder implementation.

All classes which are annotated with @NdBenchClientPlugin are installed/ registered as plugins at startup time.

At runtime, one of the installed/ registered plugins are loaded based on the REST call (/REST/ndbench/driver/init/<PluginName>) for initialization.

As these plugins are loaded, NdBenchDriver uses NdBenchClient interface to operate on underlying plugin implementation.

Plugin development tips

  1. Always map key to your workload's primary key. This will allow you to take full advantage of NDBench's key set size, load pattern, and backfill capabilities. If you don't map key to your primary key (eg. generate primary key randomly), then NDBench's backfill will only backfill a fraction of keys, and you will have to re-implement key set size and load pattern in your plugin.

How keys are generated

NDBench plugins implement at least readSingle(String key) and writeSingle(String key) methods. The parameters that affect key generation are:

  1. numKeys configuration value.
  2. Whether we are backfilling (ie. /backfill) or load testing (ie. /start, /startReads, or /startWrites).
  3. If load testing, which load pattern is used (eg. random, sliding_window, zipfian).

numKeys is the size of key set eligible for selection. For example, numKeys=100 may generate String keys from "T0" to "T99".

When backfilling, every key in the set is generate exactly once and sent to the plugin's writeSingle() method.

When load testing, keys are generated differently based on which load pattern is used:

  • random: pick from the key set randomly (ie. uniform distribution).
  • sliding_window: given a window size and test duration (in seconds), picks keys only from the current "window". The window slides at a constant rate throughout the test duration.
  • zipfian: pick from the key set using a Zipf distribution.

How values are generated

Plugins must implement the init(DataGenerator) method which receives a DefaultDataGenerator object whose getRandomValue() method can then be called to generate random String values. The following configurations affect the returned values:

  1. dataSize: the size in bytes.
  2. useVariableDataSize: if true, the size will instead by picked randomly (ie. uniform distribution) in [dataSizeLowerBound, dataSizeUpperBound).
  3. numValues: NDBench caches this many values initially, and every 10ms changes one of them to a new value. Therefore, the higher this number, the more likely to get a unique value (and the more JVM memory used).
Clone this wiki locally