SML/NJ Benchmark Suite

This repository contains benchmark programs and supporting script for measuring the performance of the SML/NJ system. These include the "classic" benchmarks that have frequently been used to evaluate the performance of SML/NJ (and other SML systems) in the published literature. The original version of this suite was assembled by Lal George at Bell Laboratories in the late 1990s.

This version of the benchmark suite adds many additional programs, increases the problem sizes for some programs, and modernizes the code for others.

Running the Benchmarks

Each directory contains a script called runit. To invoke it, the shell variable SML must be set to the path for the SML/NJ compiler. 'runit' executed at the top level will create a file called LOG that contains the results of compiling and running each benchmark.

Executing

bin/runit.sh -help

will show command line options.

Benchmark Directory Structure

Each benchmark directory contains the following files (not all of which may be present):

• main.sml -- a SML file that defines the Main structure.

• sources.cm -- a CM file for building the benchmark program.

• additional SML source files for the benchmark.

• FILES -- an optional list of additional source files (other than main.sml) in compilation order. The bin/make-single-file.sh script uses this list to construct a single-file version of the program. This file is omitted for benchmarks that have main.sml as their only source file

• ANSWER -- an optional text file containing the correct output from running the Main.testit function. Some benchmarks currently do not have test output; for these benchmarks, the ANSWER file is omitted.

• DATA -- an optional directory containing any input data files required by the benchmark program.

The Benchmark Programs

The following is a brief description of each benchmark program. The programs are listed by directory name in alphabetical order; for more detailed information, see the README.md file in each directory.

Directory	LOC	Short Description
barnes-hut	820	Hierarchical N-body solver
boyer	912
count-graphs	399
delta-blue	703	Delta-Blue incremental constraint solver
dlx	1,887
fft	166
id-ray	xxx	Ray tracer ported from Id
knuth-bendix	430	Knuth-Bendix completion algorithm
lexgen	1,036	lexical-analyzer generator
life	118	Conway's game of life
logic	317	Knuth-Bendix completion algorithm
mandelbrot	49	Mandelbrot-set computation
mandelbrot-rat	xxx	Mandelbrot-set computation using rationals
mazefun	185	Maze generator
mc-ray	736	Monte-Carlo ray tracer
mlyacc	5,566
nucleic	2,920
pia	xxx
plclub-ray	2,026	Another ray tracer
ratio-regions	xxx
ray	352
simple	760	A spherical fluid-dynamics program
smith-nf	371	Computes the Smith Normal Form
tsp	304	A Traveling-Sales-Person solver
vliw	2,934	A Very-Long-Instruction-Word instruction scheduler

Scripts

cloc.sh

The cloc.sh script can be used to count the number of source lines in the benchmark programs. It uses the cloc program and reports the number of blank, comment, and code lines. It applies cloc to the result of make-single-file.sh, so the result includes the BMARK signature and three extra lines of code for the local declaration.

make-single-file.sh

The make-single-file.sh script is used to create a single compilation unit for the benchmark sources.

runit.sh

The runit.sh script is used to run one or more benchmark programs.