QUnit - Tech Learning

This post is part of my tech learning series, where I take a few hours each week to try out a piece of technology that I'd like to learn.

This time I'm back to using JavaScript but instead of using a framework or library to build an application, I decided to try out the testing framework called QUnit.

QUnit came from the jQuery project and is now a general purpose JavaScript testing framework. I've used Jasmine for my JavaScript testing in the past but I'm not a fan of the BDD/spec style it uses. Maybe that's because I learned unit testing on Ruby's Test::Unit but I've always preferred assert style tests over expect.

Testing with QUnit

Since I don't want to build an application just to use QUnit, I decided to port an existing test suite from Jasmine to QUnit. My A/B Test Calculator was the perfect candidate:

• it's small

• it's self-contained

• it has an existing, high coverage test suite

Getting started with QUnit was easy. Their documentation was pretty through and all that I needed was to create an HTML file that did a few things:

• loaded the QUnit css

• had a div with the id of qunit

• had a div with the id of qunit-fixture (though I didn't use it in these tests)

• loaded the QUnit JavaScript

• loaded my test suite

Autostart

One thing that threw me for a loop was I was trying to use grunt to run the tests on the commandline. Normally QUnit runs automatically on the page load, but in Grunt that wouldn't work.

Instead, you need to stop QUnit from starting automatically, load your tests, and then start QUnit manually.

<script src="qunit-1.14.0.js"></script>
<script>QUnit.config.autostart = false;</script>
<script src="tests.js"></script>
<script>QUnit.start();</script>

Test layout

Instead of using nested functions or methods like Jasmine and Ruby to separate contexts, QUnit takes a simpler approach. Making a function call to QUnit.module("name") will end your current context and start a new one.

At first I thought this was confusing but it actually prevented my tests from getting deeply nested. I could see this being an advantage when refactoring tests because you wouldn't need a dummy commit to re-indent everything after moving code around.

Each module can also have setup and teardown functions that run but none of my tests needed them.

Tests

Each test is started using QUnit.test("test name"), function(assert) with the function acting as your test body. The assert object is where your assertions are called on. Though I wouldn't do it, you could use a different name like expect and make it spec-like:

QUnit.test("test name"), function(expect) {
  expect.equal(a, b)
}

Calling it should might be interesting though:

QUnit.test("test name"), function(should) {
  should.equal(a, b)
}

Running QUnit tests

Running the tests couldn't have been easier. I was using grunt on the commandline and they'd work exactly how you think.

I was also able to open the test HTML file in a browser using file:/// and QUnit worked exactly how you'd expect.

Summary

I really enjoyed using QUnit. It feels more direct and simple than Jasmine though I don't know how many third party libraries are out there for it.

I'd also be curious to see how to integrate it with Rails and the asset pipeline. That's one of the main reasons why I started with Jasmine, it had some great Ruby integration and it felt like a good fit there.

I'll definitely be adding QUnit to my standard toolkit. It won't be replacing Jasmine yet, at least until I get an opportunity to try QUnit in Rails and in a production application.

Code

The HTML test runner which loads the tests and qunit.

QUnit Tech Learning

<script src="../src/qunit-tech-learning.js"></script>
<script src="qunit-1.14.0.js"></script>
<script>QUnit.config.autostart = false;</script>
<script src="tests.js"></script>
<script>QUnit.start();</script>

The test suite which has two modules.

QUnit.module("Sanity");
QUnit.test("sanity test", function(assert) {
  assert.ok(1 == "1");
});
QUnit.test("SplitReview loaded", function(assert) {
  assert.ok(SplitReview);
});

// Based on abingo and http://20bits.com/article/statistical-analysis-and-ab-testing
var treatmentA = \[
  { impressions: 182, conversions: 35},
  { impressions: 180, conversions: 45}
\];
var treatmentB = \[
  { impressions: 182, conversions: 35},
  { impressions: 189, conversions: 28}
\];
var treatmentC = \[
  { impressions: 182, conversions: 35},
  { impressions: 188, conversions: 61}
\];
var treatmentD = \[
  { impressions: 182, conversions: 35},
  { impressions: 189, conversions: 14}
\];
var treatmentE = \[
  { impressions: 182, conversions: 282},
  { impressions: 189, conversions: 114}
\];

QUnit.module("SplitReview#score");
QUnit.test("should return a scoring object", function(assert) {
  var result = SplitReview.score(treatmentA);

  assert.equal(typeof(result), "object");
  assert.ok(result.z, "z should be defined");
  assert.ok(result.p, "p should be defined");
  assert.ok(result.winner, "winner should be defined");

  assert.ok(result.a\_rate)
  assert.ok(result.b\_rate)
});

QUnit.test("should show the experiment as a winner against treatment A at 90%", function(assert) {
  var result = SplitReview.score(treatmentA);

  assert.equal(result.a\_rate, 19.23);
  assert.equal(result.b\_rate, 25.0);
  assert.equal(result.winner, 'b');
  assert.equal(result.z, 1.33);
  assert.equal(result.p, 0.10);
});

QUnit.test("should show no winner against treatment B", function(assert) {
  var result = SplitReview.score(treatmentB);

  assert.equal(result.a\_rate, 19.23);
  assert.equal(result.b\_rate, 14.81);
  assert.equal(result.winner, null);
  assert.equal(result.z, -1.13);
  assert.equal(result.p, null);
});

QUnit.test("should show the experiment as a winner against treatment C at 99%", function(assert) {
  var result = SplitReview.score(treatmentC);

  assert.equal(result.a\_rate, 19.23);
  assert.equal(result.b\_rate, 32.45);
  assert.equal(result.winner, "b");
  assert.equal(result.z, 2.94);
  assert.equal(result.p, 0.01);
});

QUnit.test("should show the control as a winner against treatment D at 99.9%", function(assert) {
  var result = SplitReview.score(treatmentD);

  assert.equal(result.a\_rate, 19.23);
  assert.equal(result.b\_rate, 7.41);
  assert.equal(result.winner, "a");
  assert.equal(result.z, -3.39);
  assert.equal(result.p, 0.001);

});

QUnit.test("should show the control as a winner against treatment E at 99.9% (over 100% conversion rate based on unique counting)", function(assert) {
  var result = SplitReview.score(treatmentE);

  assert.equal(result.a\_rate, 154.95);
  assert.equal(result.b\_rate, 60.32);
  assert.equal(result.winner, "a");
  assert.equal(result.z, -12.27);
  assert.equal(result.p, 0.001);

});