SeGLiR is a javascript library for rapid A/B-testing with Sequential Generalized Likelihood Ratio Tests.
Sequential GLR tests are a family of sequential hypothesis tests, i.e. tests that stop as soon as a significant result has been detected. Compared to classical fixed samplesize tests, sequential GLR tests may give a significant decrease in the needed samplesize, while keeping the same type-1 and type-2 error guarantees. Below is a comparison of the expected samplesize for fixed samplesize and sequential GLR tests at the same levels. For a more detailed explanation of this family of tests (as well as the graph below), see the reference.
Try out this demo to compare samplesize from the fixed samplesize test versus the sequential GLR test yourself.
SeGLiR currently contain these tests:
- comparing two bernoulli proportions (for instance for comparing conversion on a website)
- comparing two normal means (with equal, known or unknown variance)
- choosing the best arm in a multi-armed bandit setting (with δ-PAC guarantees)
Improvements and additions are welcome, take a look at issues for suggested improvements.
The recommended way is to install SeGLiR via node package manager (install node.js if you don't already have it):
npm install seglir
If you're not running SeGLiR in node, note that SeGLiR requires jStat.js available.
var glr = require("seglir");
// create an instance of a two-sided test comparing bernoulli proportions, with indifference region with size 0.01, alpha-level = 0.05, beta-level = 0.10
var test = new glr.test("bernoulli", "two-sided", 0.01, 0.05, 0.10);
// add data as it comes in
...
test.addData({x : 0});
test.addData({x : 0, y : 0});
test.addData({y : 1});
...
// when addData() returns string 'true' or 'false', the test is concluded
// 'true' means the null hypothesis was accepted
// 'false' means the null hypothesis was rejected (and the alternative hypothesis accepted)
// get bias-adjusted estimates
test.estimate()For a complete function reference, see the reference.
Make sure you have grunt and node installed.
To install the development dependencies run npm install and to build it run grunt in the root directory.
SeGLiR is distributed under the MIT License.