Code coverage reports with Puppeteer and Istanbul


The project I’m currently working on includes a set of Puppeteer scripts to test various UI/UX scenarios. Our project generates full stack code coverage reports of unit, integration, and e2e tests, so I spent some time looking into how to integrate Puppeteer into this process in a generic way.

This article has been tested against Puppeteer 1.13.0 and nyc 14.1.1

nyc (Istanbul’s official CLI tool) supports a set of lower-level commands that we will tap into. The idea is to:

The nyc instrument CLI tool takes a source and a destination directory. If you have lib/sdk, lib/components, and lib/ui, then you can do:

mkdir lib-nyc # Should probably be in .gitignore
nyc instrument --extension .js --extension .jsx --compact=false lib/sdk lib-nyc/sdk
nyc instrument --extension .js --extension .jsx --compact=false lib/components lib-nyc/components
nyc instrument --extension .js --extension .jsx --compact=false lib/ui lib-nyc/ui

Then we can instruct something like Webpack to take the source files from our instrumented directory, potentially through an environment variable that the configuration file understands:

BASE_DIRECTORY=lib-nyc webpack --config=webpack.config.js

When running instrumented code, the code coverage runtime information will be stored in window.__coverage__. We want to get that object out of the Puppeteer page and store it in .nyc_output/<uuid>.json, where the file name is a random UUID:

const _ = require('lodash')
const uuid = require('uuid/v4')
const puppeteer = require('puppeteer')
const fs = require('fs')
const path = require('path')
const mkdirp = require('mkdirp')

const ROOT = 'path/to/project/root'

...

const browser = await puppeteer.launch({ ... })
const page = await browser.newPage()

...

const coverageReport = page.evaluate(() => {
  return window.__coverage__
})

if (coverageReport && !_.isEmpty(coverageReport)) {
  const NYC_OUTPUT_BASE = path.resolve(ROOT, '.nyc_output')
  mkdirp.sync(NYC_OUTPUT_BASE)
  const NYC_OUTPUT_DEST = path.resolve(NYC_OUTPUT_BASE, `${uuid()}.json`)
  console.log(`Storing code coverage results at ${NYC_OUTPUT_DEST}`)
  fs.writeFileSync(NYC_OUTPUT_DEST, JSON.stringify(coverageReport), {
    encoding: 'utf8'
  })
}

The .nyc_output will be populated after running the Puppeteer scripts:

$ ls -l .nyc_output
total 8112
-rw-r--r--  1 jviotti  staff  688944 Sep  9 15:11 08584aa2-9fe0-48ec-845c-e7962fe80310.json
-rw-r--r--  1 jviotti  staff  689136 Sep  9 15:10 6005bc48-ce3e-4a30-959a-fa94ba344ad2.json
-rw-r--r--  1 jviotti  staff  689673 Sep  9 15:09 d71c8147-79ea-4b9e-b8bb-f83a2c8b7740.json
-rw-r--r--  1 jviotti  staff  688883 Sep  9 15:10 f85c5916-d4b7-49b8-8db9-a402d4cd3dbb.json
-rw-r--r--  1 jviotti  staff  689500 Sep  9 15:11 f8ac1092-ab68-41a4-9bad-8702a43b9d11.json
-rw-r--r--  1 jviotti  staff  688915 Sep  9 15:09 fcd64511-3b22-44aa-a24c-e65359b26953.json

Finally, we can generate an HTML joint report using nyc:

nyc --extension .js --extension .jsx --compact=false --reporter=html report

Then, we can open coverage/index.html, and inspect the visual report:

nyc report