zUnit = goodbits(tape) + goodbits(mocha) - dependencies;
zUnit is a zero dependency1, non-polluting2 test harness for Node.js that you can execute like any other JavaScript program. I wrote it because mocha, my preferred test harness, is the number one culprit for vulnerabilities in my open source projects and I'm tired of updating them just because mocha, or one of its dependencies triggered an audit warning. While zUnit does lack some of the advanced features, such as concurrent tests, automatic retries and true file globbing3, most of the day-to-day features are present.
zUnit has no production dependencies, but does depend on a few development dependencies such as eslint and prettier.
You can add test functions (describe, it, etc) to the global namespace via the pollute config option.
Since writing zUnit I've begun to wonder whether some of Mocha's advanced features are universally beneficial. Many test suites are too small to warrant concurrency, and others (e.g. persistence tests) may require a great deal of effort to isolate. Concurrent testing also has drawbacks - the test harness and reporters become more complex and the output must be buffered, delaying feedback. I'm also unconvinced about automaticaly retrying tests, I think it better to fix any that are flakey, and take a statistical approach when results are naturally unpredictable.
-
Install zUnit
npm i zunit --save-dev -
Add the zUnit script to package.json
{ "scripts": { "test": "zUnit" } } -
Create a test suite, e.g.
test/user-db.test.jsconst { describe, it, xit, beforeEach } = require('zunit'); const assert = require('assert'); const userDb = require('../lib/user-db'); describe('User DB', () => { beforeEach(async () => { await userDb.flush(); }); describe('List Users', () => { it('should list all users', async () => { await userDb.create({ name: 'John' }); await userDb.create({ name: 'Julie' }); const users = await userDb.list(); assert.strictEqual(users.length, 2); assert.strictEqual(users[0].name, 'John'); assert.strictEqual(users[1].name, 'Julie'); }); xit('should list matching users', async () => {}); }); });
-
Run the tests
npm test User DB List Users should list all users - PASSED (2ms) should list matching use - SKIPPED (0ms) Summary Tests: 2, Passed: 1, Skipped: 1, Failed: 0, Duration: 2ms
Suite.discover() was made asynchronous to support dynmically importing ECMAScript modules. If you used a custom launch script that automatically discovers tests you will need to update it to wait for discover to resolve. Another breaking side-effect of this change is that it is no longer possible to implicitly export test suites. Previously you could require test suites that were defined with the describe syntax as follows...
describe('Database Tests', () => {
// ...
});const databaseTests = require('./databaseTests.test.js');
const allTests = new Suite('All Tests').add(databaseTests);Now if you manually compose test suites rather than discovering them, they must be explicitly exported, e.g.
module.exports = describe('Database Tests', () => {
// ...
});You can configure zUnit's launch script by:
- Specifying a configuration file when invoking the script, e.g.
{ "scripts": { "test": "zUnit test/zUnit.json" } } - Adding a
zUnitsubdocument to package.json, e.g.{ "zUnit": { "exit": true, "pollute": true, "require": ["test/setup.js"] } } - Creating a file called
.zUnit.jsonor.zUnit.jsin the project root
| Name | Type | Default | Notes |
|---|---|---|---|
| name | String | package.name |
The top level suite name. |
| directory | String | path.resolve('test') |
The initial directory to recurse when requiring tests. |
| pattern | String or RegExp | /^[\w-]+\.test\.js$/ |
The regular expression to use for matching test files. Omit the start and end slashes when using json |
| require | Array | [] |
A list of CommonJS or ECMAScript modules to load before discovering tests. |
| pollute | Boolean | false |
Control whether to pollute the global namespace with test functions so you don't have to require them. |
| exit | Boolean | false |
For the node process to exit after tests are complete. |
Sometimes the code under test uses callbacks, making it easier if the test is callback based too. If you define your test functions to take two arguments, the second argument will be passed a callback which you should invoke to signify that the test is done. e.g.
const { describe, it } = require('zunit');
it('should do something wonderful', (test, done) => {
callbackApi((err, items) => {
if (err) return done(err);
assert.strictEqual(items.length, 0);
done();
});
});Unlike with mocha, you can make the test function asynchronous, allowing you to use await when you have a mixture of callback and promise based code in your test.
You can define pending tests / skip tests in the following ways...
-
Using
xitconst { describe, xit } = require('zunit'); describe('My Suite', () => { xit('should do something wonderful', async () => { // ... }); });
-
Passing an option to
itconst { describe, it } = require('zunit'); describe('My Suite', () => { it( 'should do something wonderful', async () => { // ... }, { skip: true, reason: 'Optional Reason' } ); });
-
Using
xdescribeconst { xdescribe, it } = require('zunit'); xdescribe('My Suite', () => { it('should do something wonderful', async () => { // ... }); });
-
Passing an option to
describeconst { describe, it } = require('zunit'); describe( 'My Suite', () => { it('should do something wonderful', async () => { // ... }); }, { skip: true, reason: 'Optional Reason' } );
-
Defining a test without a test function
const { describe, it } = require('zunit'); describe('My Suite', () => { it('should do something wonderful'); });
-
Returning
test.skip()from within a test functionconst { describe, it } = require('zunit'); describe('My Suite', () => { it('should do something wonderful', async (test) => { return test.skip('Optional Reason'); }); });
-
In a beforeEach hook
const { describe, it, beforeEach } = require('zunit'); describe('My Suite', () => { beforeEach(async (hook) => { return hook.test.skip('Optional Reason'); }); it('should do something wonderful', async (test) => { // ... }); });
-
In a before hook
const { describe, it, before } = require('zunit'); describe('My Suite', () => { before(async (hook) => { return hook.suite.skip('Optional Reason'); }); it('should do something wonderful', async (test) => { // ... }); });
You can selectively run tests or suites as follows...
-
Using
oitconst { describe, oit } = require('zunit'); describe('My Suite', () => { oit('should do something wonderful', async () => { // ... }); });
-
Passing an option to
itconst { describe, it } = require('zunit'); describe('My Suite', () => { it( 'should do something wonderful', async () => { // ... }, { exclusive: true } ); });
-
Using
odescribeconst { odescribe, it } = require('zunit'); odescribe('My Suite', () => { it('should do something wonderful', async () => { // ... }); });
-
Passing an option to
describe(affects all tests in the enclosing and included suites)const { describe, it } = require('zunit'); describe( 'My Suite', () => { it('should do something wonderful', async () => { // ... }); }, { exclusive: true } );
Tests default to timing out after 5 seconds. You can override this as follows...
-
Passing a timeout option when running the main suite
runnable.run(reporter, { timeout: 10000 }).then(() => { if (runnable.failed) process.exit(1); });
-
Passing a timeout option to
itconst { describe, it } = require('zunit'); describe('My Suite', () => { it( 'should do something wonderful', async () => { // ... }, { timeout: 10000 } ); });
-
Passing a timeout option to
describe(affects all tests in the suite)const { describe, it } = require('zunit'); describe( 'My Suite', () => { it('should do something wonderful', async () => { // ... }); }, { timeout: 10000 } );
The timeout includes the duration of beforeEach/afterEach lifecycle hooks, although these may also have their own timeouts.
Test suites continue running tests after failure by default. You can override this in the following ways...
-
Passing an abort option when running the main suite
runnable.run(reporter, { abort: true }).then(() => { if (runnable.failed) process.exit(1); });
-
Passing an option to
describeconst { describe, it } = require('zunit'); describe( 'My Suite', () => { it('should do something wonderful', async () => { // ... }); }, { abort: true } );
- before - runs once before the first test in the enclosing and included suites
- after - runs once after the last test in the enclosing and included suites
- beforeEach - runs before each test in the enclosing and included suites
- afterEach - runs after each test in the enclosing and included suites
This is best demonstrated with an example
const { describe, before, after, beforeEach, afterEach, it } = require('zunit');
describe('Suite', () => {
before(async (hook) => {
console.log(hook.name);
});
beforeEach(async (hook) => {
console.log(hook.name);
});
after(async (hook) => {
console.log(hook.name);
});
afterEach(async (hook) => {
console.log(hook.name);
});
it('Test 1', async () => {});
it('Test 2', async () => {});
describe('Nested Suite', () => {
before(async (hook) => {
console.log(hook.name);
});
beforeEach(async (hook) => {
console.log(hook.name);
});
after(async (hook) => {
console.log(hook.name);
});
afterEach(async (hook) => {
console.log(hook.name);
});
it('Nested Test 1', async () => {});
it('Nested Test 2', async () => {});
});
});When a Before hook fails, the tests are not run, and therefore denied opportunity to pass or fail. This means there will be a discrepancy in the stats (i.e. tests != passed + failed). In this case the harness report will be marked as incomplete and failed.
When an After hook fails, the tests have run, so there will be no discrepancy in the test stats, and the harness report will not be marked as incomplete, but will still be failed.
Some report specifications such as TAP and Surefire have no concept of hooks, and therefore do not have a sensible mechanism for reporting their failure. It is therefore important to always check the result of the harness report, i.e.
const reporter = new TapReporter();
harness.run(reporter).then((report) => {
if (report.failed) process.exit(1);
if (report.incomplete) {
console.log('One or more tests were not run!');
process.exit(2);
}
});You can explicitly name hooks by passing a string as the first parameter. You can also skip a suite from a before hook, and a test from a beforeEach hook. e.g.
before('Suite Setup', (hook) => {
hook.suite.skip('optional reason');
});
beforeEach('Test Setup', (hook) => {
hook.test.skip('optional reason');
});As with tests you can enable callback mode by adding a second paramter to any lifecycle's hook function, e.g.
before((hook, done) => {
callbackApi((err) => {
if (err) return done(err);
done();
});
});The function may still be async if you need to mix and match promises and callbacks.
Finally you can specify a timeout for any lifecycle hook af follows...
before(
async (hook) => {
// ...
},
{ timeout: 1000 }
);Timeouts for before/after hooks are independent of test timeouts, but timeouts for beforeEach/afterEach operate within the test's timeout and so must be shorter if they are to be of any use.
It is sometimes necessary to initialise a variable in a before or beforeEach function, which is subsequently used from your tests. The typical approach is as follows...
describe('Database Tests', () => {
let db;
before(async () => {
db = await Databaes.connect();
});
it('should find no records when empty', async () => {
const records = await db.findAll();
assert.strictEqual(records.length, 0);
});
});This is fine providing your tests are declared within the same module as the variable, but occasionally they may not be. You could for example have a shared set of compliance tests for different database implementations, and therefore need a way to inject the database client into those tests. This is where Locals come in. In zUnit, locals is an object with three public methods
- locals.get('name');
- locals.set('name', value);
- locals.del('name');
It is a property of both suite and test objects available at runtime, i.e.
before((hook) => {
hook.suite.locals.set('a', 1);
});
beforeEach((hook) => {
hook.test.locals.set('b', 2);
});
it('some test', (test) => {
assert.strictEqual(test.locals.get('a'), 1);
assert.strictEqual(test.locals.get('b'), 2);
});Using locals, the previous database test can be rewritten as...
describe('Database Tests', () => {
before(async (hook) => {
const db = await Databaes.connect();
hook.suite.locals.set('db', db);
});
it('should find no records when empty', async (test) => {
const db = test.locals.get('db');
const records = await db.findAll();
assert.strictEqual(records.length, 0);
});
});Locals also support nesting...
describe('Outer Suite', () => {
before((hook) => {
hook.suite.locals.set('a', 1);
hook.suite.locals.set('b', 1);
});
describe('Nested Suite', () => {
before((hook) => {
hook.suite.locals.set('b', 2);
hook.suite.locals.set('c', 3);
});
it('should access outer locals', async (test) => {
assert.strictEqual(test.locals.get('a'), 1);
});
it('should mask outer locals', async (test) => {
assert.strictEqual(test.locals.get('b'), 2);
});
it('should access nested locals', async (test) => {
assert.strictEqual(test.locals.get('c'), 3);
});
});
});Nested locals only mask values in upper scopes. They do not replace or delete them.
There's no need to use describe and it if you prefer not to. You can just as easily create test suites programmatically using the API...
const assert = require('assert');
const { Hook, Suite, Test } = require('zunit');
const reset = new Hook('Reset Environment', () => {
// ...
});
const suite = new Suite('Test Suite').beforeEach(reset);
const test1 = new Test('Test 1', async () => {
assert.strictEqual(1, 2);
});
const test2 = new Test('Test 2', async () => {
assert.strictEqual(1, 2);
});
suite.add(test1, test2);
module.exports = suite;Both the Suite and Test constructors accept an optional options object which can be used for aborting early, skipping tests or making them exclusive. e.g.
const suite = new Suite('Test Suite', { abort: true, skip: true });
const test = new Test('Test 1', { exclusive: true });If the packaged launch script doesn't meet your needs you can create your own. For example, you may want to use a different reporter...
const { EOL } = require('os');
const { Harness, Suite, TapReporter } = require('zunit');
new Suite('zUnit').discover().then((suite) => {
const harness = new Harness(suite);
const reporter = new TapReporter();
harness.run(reporter).then((report) => {
if (report.failed) process.exit(1);
if (report.incomplete) {
console.log(`One or more tests were not run!${EOL}`);
process.exit(2);
}
process.exit();
});
});The launch script can automatically discover test suites using the discover function. e.g.
new Suite('zUnit').discover().then((suite) => {
const harness = new Harness(suite);
// ...
});By default, the discover function will recursively descended into the 'test' directory looking files which end in '.test.js', '.test.cjs' and '.test.mjs'. You can override this behaviour through the following options.
| Name | Type | Default | Notes |
|---|---|---|---|
| directory | String | path.resolve('test') |
The initial directory to recurse when requiring tests. |
| pattern | Regular Expression | /^[\w-]+\.test\.(?:js|cjs|mjs)$/ |
The regular expression to use for matching test files. |
| filter | Function() : Boolean | Indicates whether a directory should be recursed or a file should be included. Override this if you have directories you want to ignore |
For example:
const options = { directory: __dirname, pattern: /^\w+Test.(?:js|cjs|mjs)$/ };
new Suite('zUnit').discover(options).then((suite) => {
const harness = new Harness(suite);
// ...
});The launch script can also manually compose tests suites, but this does necessitate that the suites are exported, e.g.
const databaseSuite = require('./database.test.js');
const apiSuite = require('./api.test.js');
const suite = new Suite('zUnit').add(databaseSuite, apiSuite)
const harness = new Harness(suite);
module.exports = describe('Database Suite', () => {
// ...
});module.exports = describe('API Suite', () => {
// ...
});zUnit ships with the following reporters
This reporter builds up a graph of test results for subsequent interogation.
const reporter = new GraphReporter();
await harness.run(reporter);
const graph = reporter.toGraph();Each node in the graph has the following properties
| Name | Type | Notes |
|---|---|---|
| name | String | The node name |
| type | String | The node type (test or suite) |
| isTest | Function() : Boolean | Indicates whether the node is a test |
| isSuite | Function() : Boolean | Indicates whether the node is a suite |
| point | Number | The test point number (undefined for suites) |
| result | String | One of Outcomes |
| passed | Boolean | Indicates wither the node passed |
| failed | Boolean | Indicates wither the node failed |
| skipped | Boolean | Indicates wither the node skipped |
| error | Error | Populated if the test fails |
| duration | Number | Milliseconds |
| tests | Number | Number of tests |
| failures | Number | Number of failures |
| skips | Number | Number of skipped tests |
| children | Array<GraphNode> | Array of child nodes |
| parent | GraphNode | Parent node |
| resolve | Function(...Number) : GraphNode | Resolves the specified child, e.g. .resolve(1, 2, 3) |
A Surefire reporter which is compatible with the Jenkins xUnit plugin
const reporter = new SurefireReporter();
await harness.run(reporter);| Option | Type | Default | Notes |
|---|---|---|---|
| stream | stream.Writable | stdout | Override to redirect output |
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite name="Suite" tests="2" failures="1" errors="0" skipped="0" time="10.023">
<testcase name="Suite / should pass" time="5">
</testcase>
<testcase name="Suite / should fail" time="5.023">
<failure message="Oh Noes!" type="AssertionError">
<![CDATA[
AssertionError [ERR_ASSERTION]: 1 == 2
at Test._fn (/Users/example/zunit/test/Test.test.js:273:14)
at async Promise.all (index 0)
at async Test._runAll (/Users/example/zunit/lib/Test.js:80:7)
at async Test.run (/Users/example/zunit/lib/Test.js:64:7)
at async Suite._runTestable (/Users/example/zunit/lib/Suite.js:136:5)
at async Suite._runAll (/Users/example/zunit/lib/Suite.js:127:9)
at async Suite.run (/Users/example/zunit/lib/Suite.js:114:7)
at async Suite._runTestable (/Users/example/zunit/lib/Suite.js:136:5)
at async Suite._runAll (/Users/example/zunit/lib/Suite.js:127:9)
at async Suite.run (/Users/example/zunit/lib/Suite.js:114:7)
]]>
</failure>
</testcase>
</testsuite>It is necessary to take some liberties with the Surefire format since:
- it does not support nested test suites
- it has no concept of Before/After hooks
- it differentiates between assertion failures and errors
Pipes test events to multiple reporters
const specReporter = new SpecReporter();
const tapReporter = new TapReporter({ stream: fileStream });
const multiReporter = new MultiReporter().add(specReporter, tapReporter);
await harness.run(reporter);Similar to mocha's spec reporter
const reporter = new SpecReporter(options);
await harness.run(reporter);zUnit
Harnesses
should run a test suite
- PASSED (4ms)
should run an individual test
- PASSED (2ms)
Summary
Tests: 2, Passed: 2, Failed: 0, Skipped: 0, Duration: 6ms
| Option | Type | Default | Notes |
|---|---|---|---|
| stream | stream.Writable | stdout | Override to redirect output |
| colours | Boolean | true | Toggles colours |
| colors | Boolean | true | Alias for colours |
A TAP Reporter
const reporter = new TapReporter();
await harness.run(reporter);| Option | Type | Default | Notes |
|---|---|---|---|
| stream | stream.Writable | stdout | Override to redirect output |
TAP version 13
1..2
ok 1 - Harnesses / should run a test suite
ok 2 - Harnesses / should run an individual test
The Rick Reporter is for when you need persistence and integrity.
It can be annoying to repeatedly add and remove syntax related imports in your tests. You can exclude these from eslint's no-unused-vars rule with the following config...
{
"rules": {
"no-unused-vars": [
"error",
{
"varsIgnorePattern": "it|xit|oit|describe|xdescribe|odescribe|before|beforeEach|after|afterEach|include"
}
]
}
}Alternatively, if you are using globals then you should tell eslint to ignore them...
{
"globals": {
"describe": "readonly",
"xdescribe": "readonly",
"odescribe": "readonly",
"it": "readonly",
"xit": "readonly",
"oit": "readonly",
"before": "readonly",
"beforeEach": "readonly",
"after": "readonly",
"afterEach": "readonly",
"include": "readonly"
}
}Migrating from Mocha can be extremely quick, depending on the features and api style you use.
Mocha
describe('foo', () => {
it('bar', (done) => {});
});zUnit
describe('foo', () => {
it('bar', (test, done) => {});
});Mocha
describe('foo', () => {
it('bar', function () {
this.timeout(1000);
this.slow(500); // No zUnit equivalent
this.skip();
});
});zUnit
describe('foo', () => {
it(
'bar',
(t) => {
t.skip('optional reason');
},
{ timeout: 1000 }
);
});Mocha conveniently makes its test functions global. If you want to achieve the same with zUnit, you can set pollute to true in the launch script config, or if you have written your own launch script, add the following code.
const { syntax } = require('zunit');
Object.entries(syntax).forEach(([keyword, fn]) => (global[keyword] = fn));Mocha
describe('foo', () => {
it.only('bar', () => {});
it.skip('baz', () => {});
});zUnit
describe('foo', () => {
oit('bar', () => {});
xit('baz', () => {});
});- Frank Carver for schooling me on the things a test harness should and shouldn't do.
- geopic for contributing zUnit's TypeScript defintions.