Built in Functionality

Plugins

There is one test plugin already built-in, which simply runs tests and outputs the results to a JSON file. It's not terribly interesting on it's own, but it demonstrates a number of features and may be a helpful guide to follow. You can see it's source code here.

Runners

There are several ways to run tests. The easiest is to simply use the RunnerFactory to create a runner for you. To do so, simply write:

val runner = RunnerFactory.from(project)

Then you can run an arbitrary order of tests like so:

runner.run(List("test1", "test2"))

There is also a runList method provided for ease of use with Java code. Note that the test names passed to the runner should be fully qualified names.

The two following runners are available:

• FixedOrderRunner: The FixedOrderRunner will simply run the specified list of tests in a given order.
• SmartRunner: The SmartRunner will run tests in a given order, but will also automatically timeout tests using knowledge from prior runs, detect flaky tests, and do some basic sanity checking of results.

Runners will all return an Try[TestRunResult], where TestRunResult contains the following methods for accessing data:

public java.util.List<java.lang.String> testOrder();
public java.util.Map<java.lang.String,TestResult> results();
public java.util.Map<java.lang.String,DiffContainer> diffs(); // This will be empty if the capture_state property (above) is not set to true.

The SmartRunner is used by default.

Utility

Test discovery

If you do not know the list of tests in your project, you may obtain them by using the TestLocator:

val testOrder = TestLocator.tests(project)

final scala.collection.immutable.Stream<String> testStream = TestLocator.tests(project);

// or

final List<String> tests = scala.collection.JavaConverters.bufferAsJavaList(TestLocator.tests(project).toList().toBuffer());

This will return a list of fully qualified test names that can be passed to a runner in the same order that surefire would return.