Highlevel Testing With CasperJS

If you've written a webapp and you want to ensure that critical parts such as the signup process stay working, the best would be to have an actual user go through that process every time you change your codebase. But since that's is both tedious & expensive, the second best thing is to automate a chrome browser (webkit engine anyway) to do this for you, and upload screenshots if anything unexpected happens.

Welcome to CasperJS!

From the CasperJS website:

"Casperjs is an open source navigation scripting & testing utility written in Javascript and based on PhantomJS — the scriptable headless WebKit engine. It eases the process of defining a full navigation scenario and provides useful high-level functions, methods & syntactic sugar".

Install

OSX

If you use Homebrew, you can install both CasperJS and PhantomJS using this command:

$ brew install casperjs

Ubuntu

To install both PhantomJS and CasperJS into /usr/local with this layout:

/usr/local/bin/casperjs  -> /usr/local/casperjs-1.0.0/bin/casperjs
/usr/local/bin/phantomjs -> /usr/local/phantomjs-1.8.0-linux-x86_64/bin/phantomjs

Please execute the following (Warning: purges all previous versions of CasperJS inside /usr/local/)

PHANTOM_VERSION="1.8.0"
PHANTOM_BASE="phantomjs-${PHANTOM_VERSION}-linux-x86_64"
if [ "$(/usr/local/bin/phantomjs --version 2>/dev/null)" != "${PHANTOM_VERSION}" ]; then
  DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes install libfontconfig1-dev
  pushd /usr/local/
    wget --quiet https://phantomjs.googlecode.com/files/${PHANTOM_BASE}.tar.bz2
    tar -jxvf ${PHANTOM_BASE}.tar.bz2 && rm ${PHANTOM_BASE}.tar.bz2*
    ln -nfs /usr/local/${PHANTOM_BASE}/bin/phantomjs /usr/local/bin/phantomjs
  popd
fi

CASPER_VERSION="1.0.0"
CASPER_BASE="casperjs-${CASPER_VERSION}"
if [ "$(/usr/local/bin/casperjs --version 2>/dev/null)" != "${CASPER_VERSION}" ]; then
  pushd /usr/local/
    rm -rf *casperjs*
    wget --quiet https://github.com/n1k0/casperjs/tarball/${CASPER_VERSION}
    tar -zxvf ${CASPER_VERSION} && rm ${CASPER_VERSION}*
    mv *casperjs* /usr/local/${CASPER_BASE} # n1k0-casperjs-e629586
    ln -nfs /usr/local/${CASPER_BASE}/bin/casperjs /usr/local/bin/casperjs
  popd
fi

$ phantomjs --version
$ casperjs --version

Use

Notes

You can write CaspjerJS scripts in Javascript or Coffeescript. CasperJS will just switch interpreters based on the extension of the script you feed it. For small projects like these I prefer Coffeescript.

Note that CasperJS is not node.js and though compatible with require, you cannot use any npm modules.

To check if your .coffee files are valid, I recommend running them through coffeelint (npm install -g coffeelint).

Make sure you run at least RC3 if you want to capture screenshots of timeouts as well.

Example

Here's an example script that shows some different tricks, I've commented along the way. Some gems:

Anytime a testcase fails, a .png is saved. CasperJS will exit with code 1 so it's really easy to detect a fail then upload this screenshot to campfire for example. This is possible using just curl and your campfire api keys.
Anytime a page contains: Error or Exception, a fail is automatically triggered without the need to write additional asserts for this. It can be disabled on a URL basis (in this case /nonexistent is allowed to throw these texts).

## Setup
##########################################################################

utils  = require("utils")
casper = require("casper").create
  verbose: true
  logLevel: "warning"
  exitOnError: true
  safeLogs: true
  viewportSize:
    width: 1024
    height: 768

testhost   = casper.cli.get "testhost"
screenshot = casper.cli.get "screenfile"

casper
  .log("Using testhost: #{testhost}", "info")
  .log("Using screenshot: #{screenshot}", "info")

if not testhost or not screenshot or not /\.(png)$/i.test screenshot
  casper
    .echo("Usage: $ casperjs test project.coffee --ignore-ssl-errors=yes --testhost=<testhost> --screenfile=<screenshot.png>")
    .exit(1)

## Hooks
##########################################################################

# Capture screens from all fails
casper.test.on "fail", (failure) ->
  casper.capture(screenshot)
  casper.exit 1

# Capture screens from timeouts from e.g. @waitUntilVisible
# Requires RC3 or higher.
casper.options.onWaitTimeout = ->
  @capture(screenshot)
  @exit 1

# Scan for the word notice|warning|error|exception by default
casper.on "step.complete", (page) ->
  # Skip urls that can contain 'error'/'exception'
  u = casper.getCurrentUrl()
  if (u == "https://#{testhost}/nonexistent")
    return

  @test.assertEval ->
    !$('div#content').text().match(/(notice|warning|error|exception)/i)
  , "no notices, warnings, errors or exceptions in #{u}"

## Testcases
##########################################################################

# This is an app that has everything (even the /news page) behind a login.

# try to access nonexistent when logged in (don't 404, we only tell customers what exists and what not)
casper.start "https://#{testhost}/nonexistent", ->
  @test.assertHttpStatus(302, "nonexistent should 302 when logged not in (can't show guests what exists)")
  @test.assertUrlMatch /\/customers\/login/, "redirect to login"

# open /news/ without login, errors out, should go to login,
casper.thenOpen "https://#{testhost}/news/", ->
  @test.assertTextExists "I could not give you access to", "cannot access news without login"
  @test.assertUrlMatch /\/customers\/login/, "redirect to login"
  @test.assertTitle "Please login", "login page title is the one expected"
  @test.assertExists "form[action=\"/customers/login/\"]", "login page must have a form with customer/login action"
  @fill "form[action=\"/customers/login/\"]", { "data[Customer][username]": "janedoe", "data[Customer][password]": "jsdi32ld!" }, true

# redirect to landing page /news/
casper.then ->
  @test.assertUrlMatch /\/news/, "redirected to landing page after login"

# notice login twice
casper.thenOpen "https://#{testhost}/customers/login", ->
  @test.assertTextExists "You are already logged in", "notice already logged in"

# try to access admin page
casper.thenOpen "https://#{testhost}/admin/tickets", ->
  @test.assertTextExists "I could not give you access to ", "prohibit to access admin page"

# try to access nonexistent when logged in
casper.thenOpen "https://#{testhost}/nonexistent", ->
  @test.assertHttpStatus 404, "nonexistent should 404 when logged in"

# dashboard has panels
casper.thenOpen "https://#{testhost}/customers/dashboard", ->
  @test.assertTitle "Dashboard", "customer dashboard title is ok"
  @test.assertEvalEquals ->
    __utils__.findAll(".user-dashboard div.accordion-heading").length
  , 8, "found 8 customer dashboard panels"

# calculate storage price
casper.thenOpen "https://#{testhost}/storage_accounts/add", ->
  @evaluate ->
    $("#StorageAccountBytesMax").val("10737418240")
    $("#StorageAccountPassword").val("dlfksfag!1")
    $("#StorageAccountEmail").val("janedoe@example.com")
    $("#StorageAccountBytesMax").change()

  @waitFor ->
    @evaluate ->
      $("#billabe_buy").text() != "Calculating..."
  , ->
    @test.assertSelectorHasText "#billabe_buy", "45.00", "10gb is 45.00 euros for janedoe"

# unowned invoice: prohibit
casper.thenOpen "https://#{testhost}/invoices/view/201100493", ->
  @test.assertTextExists "Invoice not found", "prohibit access to invoice of another customer"

# owned invoice: allow and check it's price is 12 cents
casper.thenOpen "https://#{testhost}/invoices/view/201100975", ->
  @test.assertTextExists "Subtotal", "my invoice has a subtotal"
  @test.assertEval ->
    $("td.total").text().indexOf("0.12") > -1
  , "invoice 201100975 total is 12 cents"


## Bombs away
##########################################################################

casper.run ->
  @test.renderResults true

See what we did? In just ~100 LoC we make sure this app deals correct prices for new products, protects people's invoices and admin pages from unauthorized access, makes sure the login system functions & redirects correctly, and that no page except /nonexistent has any errors on it. If any of these conditions aren't met, a screenshot is made.

Run

To run it, type something like:

$ casperjs \
  test \
  ./tests/project.coffee \
  --ignore-ssl-errors=yes \
  --testhost=staging.exampleproject.com \
  --screenfile=./webroot/fails/screenshot.png # || script to upload screenshot.png to campfire.

Ideally, you'd wrap this up in a script and plug it into your Continuous Integration server so that it gets run on every change.

Alternatively

While still developing, it's really pleasant to have your Mac open the screenshot automatically after any fail:

$ rm -f ~/Desktop/screen.png \
 ; casperjs test ./tests/main.coffee --ignore-ssl-errors=true --testhost=www.example.local --screenfile=~/Desktop/screen.png \
|| open ~/Desktop/screen.png

Conclusion

There are also paid services you can outsource this to. Most of them offer a lot more features such as also testing against FF, IE, Opera, Mobile, etc. so it may make sense for you to use one of those. Some I know in no particular order:

As for some advantages of rolling this out yourself:

customize to your needs, run on your own CI server
the tests & actual code are stored in the same repository, hack on your code, hack on your tests, it's all versioned and coupled, this makes it easy and fun to update your tests.
no monthly fees
and as you've noticed it's actually not hard to do anymore, thanks to PhantomJS & CasperJS