SPEC Chop

Support scripts to apply ChopStiX on SPEC benchmarks.

These scripts allow you to profile SPEC benchmarks, trace the region of interest and generate (extract) the corresponding MPT files (see Microprobe documentation) to later generate standalone test cases.

Installation

To download and setup SPEC Chop for installation follow these steps:

git clone https://github.com/IBM/spec-chop
cd spec-chop

Requirements

SPEC Chop is just a bunch of shell scripts to integrate/automate the different processes.

Please install these packages on your system:

• SPEC CPU 2017 benchmark suite
• ChopStiX (https://github.com/IBM/chopstix.git)
• Microprobe (https://ibm.github.io/microprobe/)

Configuration

Edit the spec-chop.config file settings to setup your installation paths and the run options. Most commands do not have any parameters and they use the settings in the configuration files. Most settings in the configuration files can be overridden via environment variables (check the actual scripts for details).

Scripts

• spec-chop-lsf-submit.sh: submit the configured SPEC benchmarks for execution using the bsub LSF command.
• spec-chop-runspec.sh <spec_name>: run benchmark <spec_name> on the current machine.
• spec-chop-wrapper-sample.sh: wrapper support script to sample (profile) a benchmark execution (see example below).
• spec-chop-wrapper-trace.sh: wrapper support script to trace a particular region of interest of a benchmark execution (see example below).
• spec-chop-clean-spec.sh: helper script to cleanup temporary files in the SPEC installation tree.
• spec-chop-tracer.sh: Check results directory (generated by the sampling process) and submits LSF jobs to trace the top N functions according according to the configuration options in spec-chop.config. Microprobe ready MPT/MPS files are automatically generated in the process.
• spec-chop-wrapper-trace.sh: wrapper support script to trace a particular region (e.g. a function) of benchmark execution (see example below).
• spec-chop-mpt2tst.sh: Converts the MPT/MPS found in the results directory to TSTs (ready for MESA execution).
• spec-chop-lsf-mpt2tst.sh: Converts the MPT/MPS found in the results directory to TSTs (ready for MESA execution). Convertion is parallelized on multiple LSF jobs.
• spec-chop-mpt2tst-helper.sh: helper script to parallelize the mpt2tst generation process.

Examples of usage

Results and execution logs of the command below will be stored in your configured SPEC_CHOP_DB_BASEPATH.

Running SPEC

Execute the default configured SPEC benchmarks on LSF:

spec-chop-lsf-submit.sh  

Execute a particular SPEC benchmark on LSF:

SPEC_CHOP_BENCHMARKS="<bench1> <bench2>" spec-chop-lsf-submit.sh

Sampling SPEC

Execute the default configured SPEC benchmarks on LSF using the sampling wrapper with the default options:

spec-chop-lsf-submit.sh WRAPPER spec-chop-wrapper-sample.sh

Execute the default configured SPEC benchmarks on LSF using the sampling wrapper with custom sampling options:

spec-chop-lsf-submit.sh WRAPPER spec-chop-wrapper-sample.sh SAMPLE_EVENT cycles SAMPLE_PERIOD 1500000

Execute a SPEC benchmark locally using the sampling wrapper with the custom options:

SPEC_CHOP_WRAPPER=spec-chop-wrapper-sample.sh SPEC_CHOP_SAMPLE_EVENT=cycles SPEC_CHOP_SAMPLE_PERIOD=1500000 spec-chop-runspec.sh <benchmark_name>

Tracing SPEC

Trace the execution of the default configured SPEC benchmark on LSF:

spec-chop-tracer.sh

Trace a particular SPEC benchmark on LSF:

SPEC_CHOP_BENCHMARKS="<bench1> <bench2>" spec-chop-tracer.sh

The command above will look for previously generated sampling data (check Sampling SPEC section) to select the functions to trace. The following configuration variables affect the selection process:

• SPEC_CHOP_MAX_FUNCTIONS: Maximum number of functions to trace.
• SPEC_CHOP_MIN_FUNCTION_SIZE: Minimum size (in instructions) to consider a function for tracing.
• SPEC_CHOP_MIN_COVERAGE: Minimum recommended coverage. If, the total coverage of selected functions is below this threshold, a warning is raised. Note that this does not effect the selection process. The warning is a suggestion that other tracing should be required to increase the execution coverage of the benchmark.
• SPEC_CHOP_OPTIONS_TRACE: Options to be use by the chop trace command. Check ChopStiX documentation for details.

Once the set of functions are selected, a new LSF job is submitted for each of them. Once the jobs are finished Microprobe-ready MPT/MPS will be present in the configured results directory.

Generating TSTs from MPTs

Convert TSTs to MPTs locally:

spec-chop-mpt2tst.sh

Convert TSTs to MPTs on LSF:

spec-chop-lsf-mpt2tst.sh