Passing Options to Trace Formats

Libtrace has a number of options that can be configured per format that determine how it uses resources which may impact performance. For example, the number of threads to use when processing the trace (find the full list of options at the bottom of this page). Often a libtrace application may provide a limited set of commonly used options via its command-line options. However, requiring an application to provide access to all of libtrace's options has limitations: it is a lot of boilerplate code to add all possible options, old applications won't support new options, and passing different options to multiple format URIs is difficult.

As such, Libtrace provides two other mechanisms to configure options, 1) prepending options to the format URI, and 2) defining options in environment variables.

Providing Options via the Format URI

Available in libtrace 4.0.16 and newer

A user can prepend the format URI with options for that format. This takes the format: <key>=<value>,...,<key>=<value>:<format URI>. Where the format URI normally consists of <format>:<path>

Take for example tracestats, which accepts a list of format URIs and processes them sequentially. We can ask tracestats to use a single thread when processing the file erf:capture.erf, but four threads when processing the live interface ring:eth0 by setting the perpkt_threads option on each URI as follows:
./tracestats perpkt_threads=1:erf:capture.erf perpkt_threads=4:ring:eth0

Sometimes a value you pass to an option needs to include commas (,), an equals sign (=), or colons(:). You can escape such by placing the entire value in square brackets.

For example, coremap is an option which uses commas to seperate its arguments:
./tracestats perpkt_threads=4,coremap=[0,2,4,6]:dpdk:0000:42:0.0
As we want to pass the value 0,2,4,6, which contains commas, we must wrap the value in square brackets.

Providing Options Using Environment Variables

Available since libtrace 4

Environments variables are an alternative way to provide options to libtrace. Libtrace searches three environment variables when opening a trace. Libtrace will search for three environment variables and apply them to the configuration in the following order, where the first has the lowest priority.

1. LIBTRACE_CONF - Apply options to all traces started
2. LIBTRACE_CONF_<FORMAT> - Apply options to all traces of a given format
3. LIBTRACE_CONF_<FORMAT_URI> - Apply options to a specific URI

All environment variables names MUST only contain A-Z, 0-9 and _ (underscore). Any characters outside of this range should be capitalised if possible or replaced with an underscore.

For example:

• int:eth0 would match LIBTRACE_CONF, LIBTRACE_CONF_INT, LIBTRACE_CONF_INT_ETH0
• dag:/dev/dag0,0 would match LIBTRACE_CONF, LIBTRACE_CONF_DAG, LIBTRACE_CONF_DAG__DEV_DAG0_0
• test.erf would match LIBTRACE_CONF, LIBTRACE_CONF_ERF, LIBTRACE_CONF_ERF_TEST_ERF

Again, as with prepending options, the format accepted is a comma separated list of keys and values. Take our tracestats example from earlier, ./tracestats perpkt_threads=1:erf:capture.erf perpkt_threads=4:ring:eth0. Using environment variables this would be written as:

export LIBTRACE_CONF_ERF="perpkt_threads=1"
export LIBTRACE_CONF_RING="perpkt_threads=4"
./tracestats erf:capture.erf ring:eth0

Commonly Used Options

Option Name & Equivalent Function	Type	Default	Brief Descripton
perpkt_threads, pt trace_set_perpkt_threads()	int	Varies	The maximum number of per-packet threads to use when reading from a format. Note some applications may set this value and expect it to remain unchanged.
burst_size, bs trace_set_burst_size()	size_t	32*	The maximum number of packets libtrace may batch together when reading from a format
hasher_queue_size, hqs trace_set_hasher_queue_size()	size_t	1000*	The queue size, in packets, between the hasher thread and each per-packet thread
coremap trace_set_coremap()	string	None	Bind per-packet threads to specific CPU cores. E.g. coremap=[4,6] map the first thread to core 4 and second thread to core 6.

See the API documentation on the corresponding trace_set_* functions for a more complete description of these options.
* Likely to be updated with new releases

Advanced Options

These settings control the internal sizes of buffers or other behaviour within libtrace. We recommend leaving these set to the default values. Note, some of these settings only apply in very particular circumstances; for example only when the format is live, single-threaded, using a hasher function, or using reporter function.

Option Name & Equivalent Function	Type	Default	Brief Descripton
cache_size, cs trace_set_cache_size()	size_t	Varies	The size of the shared packet cache used to store empty packets
thread_cache_size, tcs trace_set_thread_cache_size()	size_t	64*	The size of each per thread packet cache used to store empty packets
fixed_count, fc trace_set_fixed_count()	bool	False	Limits the number of internal packet buffers to the packet cache size
tick_interval, ti trace_set_tick_interval()	size_t	0	The interval between tick messages in milliseconds
tick_count, tc trace_set_tick_count()	size_t	0	The number of packets between tick messages
hasher_polling, hp trace_set_hasher_polling()	bool	False	Enables per-packet threads polling on the hasher queues. Likely to decrease performance.
reporter_polling, rp trace_set_reporter_polling()	bool	False	Enable the reporter thread to poll on the result queues. Likely to decrease performance.
reporter_thold, rt trace_set_reporter_thold()	size_t	100*	The maximum number of messages a combiner may queue before publishing to the reporter thread
debug_state, ds trace_set_debug_state()	bool	False	Enables debugging information about libtrace threads

See the API documentation on the corresponding trace_set_* functions for a more complete description of these options.
* Likely to be updated with new releases