expectations

adding signal, removing noise

Introduction

expectations is built with the idea that unit tests should contain one assertion per test. A result of this design choice is that expectations has very minimal syntax.

For example, if you want to verify the result of a function call, all you need to do is specify what return value you expect from the function call.

(expect 2 (+ 1 1)

note: you'll want to (:require [expectations :refer :all]); however, no other setup is required for using expectations. I created a sample project for this blog post and the entire test file looks like this (at this point):

(ns sample.test.core
  (:require [expectations :refer :all]))

(expect 2 (+ 1 1))

I'm using 'lein expectations' to run my tests. For more information on 'lein expectations', check out the Installing page.

jfields$ lein expectations
Ran 1 tests containing 1 assertions in 2 msecs
0 failures, 0 errors.

That's the simplest, and most often used expectation - an equality comparison. The equality comparison works across all Clojure types - vectors, sets, maps, etc and any Java instances that return true when given to Clojure's = function.

(ns sample.test.core
  (:require [expectations :refer :all]))

(expect 2 (+ 1 1))
(expect [1 2] (conj [] 1 2))
(expect #{1 2} (conj #{} 1 2))
(expect {1 2} (assoc {} 1 2))

Running the previous expectations produces similar output as before.

jfields$ lein expectations
Ran 4 tests containing 4 assertions in 26 msecs
0 failures, 0 errors.

Successful equality comparison isn't very exciting; however, expectations really begins to prove its worth with its failure messages. When comparing two numbers there's not much additional information that expectations can provide. Therefore, the following output is what you would expect when your expectation fails.

(expect 2 (+ 1 3))

jfields$ lein expectations
failure in (core.clj:4) : sample.test.core
           (expect 2 (+ 1 3))
           expected: 2
                was: 4

expectations gives you the namespace, file name, and line number along with the expectation you specified, the expected value, and the actual value. Again, nothing surprising. However, when you compare vectors, sets, and maps expectations does a bit of additional work to give you clues on what the problem might be.

The following 3 expectations using vectors will all fail, and expectations provides detailed information on what exactly failed.

(expect [1 2] (conj [] 1))
(expect [1 2] (conj [] 2 1))
(expect [1 2] (conj [] 1 3))

jfields$ lein expectations
failure in (core.clj:5) : sample.test.core
           (expect [1 2] (conj [] 1))
           expected: [1 2]
                was: [1]
           2 are in expected, but not in actual
           expected is larger than actual
failure in (core.clj:6) : sample.test.core
           (expect [1 2] (conj [] 2 1))
           expected: [1 2]
                was: [2 1]
           lists appears to contain the same items with different ordering
failure in (core.clj:7) : sample.test.core
           (expect [1 2] (conj [] 1 3))
           expected: [1 2]
                was: [1 3]
           3 are in actual, but not in expected
           2 are in expected, but not in actual
Ran 3 tests containing 3 assertions in 22 msecs
3 failures, 0 errors.

In these simple examples it's easy to see what the issue is; however, when working with larger lists expectations can save you a lot of time by telling you which specific elements in the list are causing the equality to fail.

Failure reporting on sets looks very similar:

(expect #{1 2} (conj #{} 1))
(expect #{1 2} (conj #{} 1 3))

jfields$ lein expectations
failure in (core.clj:9) : sample.test.core
           (expect #{1 2} (conj #{} 1))
           expected: #{1 2}
                was: #{1}
           2 are in expected, but not in actual
failure in (core.clj:10) : sample.test.core
           (expect #{1 2} (conj #{} 1 3))
           expected: #{1 2}
                was: #{1 3}
           3 are in actual, but not in expected
           2 are in expected, but not in actual
Ran 2 tests containing 2 assertions in 15 msecs
2 failures, 0 errors.

expectations does this type of detailed failure reporting for maps as well, and this might be one if the biggest advantages expectations has over clojure.test - especially when dealing with nested maps.

(expect {:one 1 :many {:two 2}}
        (assoc {} :one 2 :many {:three 3}))

jfields$ lein expectations
failure in (core.clj:13) : sample.test.core
           (expect {:one 1, :many {:two 2}} (assoc {} :one 2 :many {:three 3}))
           expected: {:one 1, :many {:two 2}}
                was: {:many {:three 3}, :one 2}
           :many {:three with val 3 is in actual, but not in expected
           :many {:two with val 2 is in expected, but not in actual
           :one expected: 1
                     was: 2
Ran 1 tests containing 1 assertions in 19 msecs
1 failures, 0 errors.

expectations also provides a bit of additional help when comparing the equality of strings.

(expect "in 1400 and 92" "in 14OO and 92")

jfields$ lein expectations
failure in (core.clj:17) : sample.test.core
           (expect in 1400 and 92 in 14OO and 92)
           expected: "in 1400 and 92"
                was: "in 14OO and 92"
            matches: "in 14"
           diverges: "00 and 92"
                  &: "OO and 92"
Ran 1 tests containing 1 assertions in 8 msecs
1 failures, 0 errors.

That's basically all you'll need to know for using expectations to equality test. If you're still with me at this point, why not take a look at the examples of using expectations with regexs, expected exceptions and type checking?