Unit Testing Emscripten Library in Browser Using CMake and Nightwatch.JS

logo-nightwatchIn a previous blog post, I described how I took Emscripten-created JS and turned it into an UMD module.  One of the reasons I did this was because I wanted more control over the generated JavaScript and for it to be usable in more contexts, such as with the RequireJS module loader.

As I am a responsible developer, I desired to create a number of automated unit tests to ensure that the client-visible API for my Emscripten module works as I intended.  I began by searching for a browser automated test framework and settled upon Nightwatch.js.  Now I just had to figure out how to get Nightwatch.js tests running in my existing, CMake-based build system.  Here’s how I did it.

Configurig Nightwatch.JS

In order to use Nightwatch.JS, you must first configure it by creating a file called nightwatch.json. The first major decision you need to make is which WebDriver-implementing system you wish to use. Most users use Selenium, but you can also run an individual browser driver directly.

As I was not concerned with cross-browser compatibility — I assume that if the test works on one browser it will work on all major browsers — and I was looking for a system with a minimum number of build-time dependencies, I decided to use ChromeDriver automatically as my WebDriver implementation.

To make everything work, I did the following:

1. To automatically download chromedriver, add the following to CMakeLists.txt:

# Install chromedriver
add_custom_command(
  OUTPUT node_modules/chromedriver/package.json
  COMMAND npm install chromedriver
  )
add_custom_target(
  chromedriver ALL
  DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/node_modules/chromedriver/package.json
  )

2. To configure Nightwatch.JS to use chromedriver, create a nightwatch.json which looks like this (the purpose of nightwatch-globals.js will become clear shortly):

{
    "globals_path": "nightwatch-globals.js",
    "selenium" : {
        "start_process" : false
    },
    "test_settings" : {
        "default" : {
            "selenium_host": "localhost",
            "selenium_port": 9515,
            "default_path_prefix": "",
            "desiredCapabilities": {
                "browserName": "chrome",
                "chromeOptions" : {
                    "args" : ["--no-sandbox"]
                },
                "acceptSslCerts": true
            }
        }
    }
}

3. To start and stop chromedriver when running tests, create a nightwatch-globals.js which looks like this:

var chromedriver = require('chromedriver');

module.exports = {
    before: function(done) {
        chromedriver.start();
        done();
    },
    after: function(done) {
        chromedriver.stop();
        done();
    }
};

4. CMake will run the unit tests from ${CMAKE_CURRENT_BINARY_DIR}, so we’ll need to copy the above config files to ${CMAKE_CURRENT_BINARY_DIR}. Here’s how to do that:

# Copy nightwatch config files to target directory
add_custom_command(
  OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/nightwatch.json
  COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/nightwatch.json ${CMAKE_CURRENT_BINARY_DIR}/nightwatch.json
  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/nightwatch.json
  )
add_custom_target(
  nightwatch.json ALL
  DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/nightwatch.json
  )

add_custom_command(
  OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/nightwatch-globals.js
  COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/nightwatch-globals.js ${CMAKE_CURRENT_BINARY_DIR}/nightwatch-globals.js
  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/nightwatch-globals.js
  )
add_custom_target(
  nightwatch-globals.js ALL
  DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/nightwatch-globals.js
  )

Automatically Download Nightwatch.JS and Run Unit Tests

1. First, we need to create a Nightwatch.JS unit test. Here’s a sample test case from the Nightwatch.JS home page:

// google.js
module.exports = {
  'Demo test Google' : function (browser) {
    browser
      .url('http://www.google.com')
      .waitForElementVisible('body', 1000)
      .setValue('input[type=text]', 'nightwatch')
      .waitForElementVisible('button[name=btnG]', 1000)
      .click('button[name=btnG]')
      .pause(1000)
      .assert.containsText('#main', 'Night Watch')
      .end();
  }
};

2. To automatically download the Nightwatch.JS library, add the following lines to CMakeLists.txt:

# Install nightwatch
add_custom_command(
  OUTPUT node_modules/nightwatch/package.json
  COMMAND npm install nightwatch
  )
add_custom_target(
  nightwatch ALL
  DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/node_modules/nightwatch/package.json
  )

3. To run the above unit test as a CMake unit test, add the following lines to CMakeLists.txt:

add_test(
    NAME nightwatch_test
    COMMAND ./node_modules/nightwatch/bin/nightwatch -t ${CMAKE_CURRENT_SOURCE_DIR}/google.js
  )

You may want to separate your tests into multiple JavaScript files and execute them independently. Here’s one way to do that from CMake:

file(GLOB TESTCASE_SRC tests/*.js)
foreach (testPath ${TESTCASE_SRC})
  get_filename_component(testName ${testPath} NAME_WE)

  # Test all unit tests
  add_test(
    NAME browser_${testName}
    COMMAND ./node_modules/nightwatch/bin/nightwatch -t ${testPath}
  )
endforeach()

Using Local Web Server when Running Test Cases

In certain cases, your unit tests be able to refer to local file: URLs, but things tend to be a lot easier if your unit tests reference URLs from a local web server. It’s really easy to get one up and running:

1. Download Node’s http-server module by adding the following to your CMakeLists.txt

# Install http-server
add_custom_command(
  OUTPUT node_modules/http-server/package.json
  COMMAND npm install http-server
  )
add_custom_target(
  http-server ALL
  DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/node_modules/http-server/package.json
  )

2. Modify nightwatch-globals.js to start and stop the web server as part of the tests:

var chromedriver = require('chromedriver');
var http = require('http-server');

module.exports = {
    before: function(done) {
        this.server = http.createServer();
        this.server.listen(8080);
        chromedriver.start();
        done();
    },
    after: function(done) {
        this.server.close();
        chromedriver.stop();
        done();
    }
};

Once this is done, your tests can refer to http://localhost:8080.

Note that http-server reads files from the current working directory, and CMake runs unit tests from ${CMAKE_CURRENT_BINARY_DIR}, so you may need to copy your test HTML and JavaScript to ${CMAKE_CURRENT_BINARY_DIR}. Here’s some CMake code which you might find helpful:

# Copy all .HTML files to binary directory
file(GLOB HTML_SRC *.html)
foreach (htmlPath ${HTML_SRC})
  get_filename_component(htmlFileName ${htmlPath} NAME)

  # Copy HTML to binary folder so they can be referred to by the test
  add_custom_command(
    OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${htmlFileName}
    COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/${htmlFileName} ${CMAKE_CURRENT_BINARY_DIR}/${htmlFileName}
    DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/${htmlFileName}
  )
  add_custom_target(
    browser_copy_${htmlFileName} ALL
    DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${htmlFileName}
    )
endforeach()

For a real-life, working example of all this in action, see the source code for my streaming percentiles library.