expectations

adding signal, removing noise

Expecting More

expectations obviously has a bias towards one assertion per test; however, there are times that verifying several things at the same time does make sense. For example, if you want to verify a few different properties of the same Java object it probably makes sense to make multiple assertions on the same instance.

One of the biggest problems with multiple assertions per test is when your test follows this pattern:

create some state
verify a bit about the state
alter the state
verify more about the state

Part of the problem is that the assertions that occurred before the altering of the state may or may not be relevant after the alteration. Additionally, if any of the assertions fail you have to stop running the entire test - thus some of your assertions will not be run (and you'll be lacking some information).

more, more->, & more-of

expectations takes an alternate route - embracing the idea of multiple assertions by providing a specific syntax that allows multiple verifications and the least amount of duplication.

expectations has always given you the ability to test against an arbitrary fn, similar to the example below.

(expect nil? nil)

The ability to specify any fn is powerful, but it doesn't always give you the most descriptive failure messages. In expectations 2.0 we introduce the 'more, 'more->, & 'more-of macros, which are designed to allow you to expect more of your actual values.

Below is a simple example of using the more macro.

(expect (more vector? not-empty) [1 2 3])

As you can see from the above example, we're simply expecting that the actual value '[1 2 3] is both a 'vector? and 'not-empty. The 'more macro is great when you want to test a few 1 arg fns; however, I expect you'll more often find yourself reaching for 'more-> and 'more-of.

The 'more-> macro is used for threading the actual value and comparing the result to an expected value. Below is a simple example of using 'more to pull values out of a vector and test their equality.

(expect (more-> 1 first
                3 last)
  [1 2 3])

The 'more-> macro threads using -> (thread-first), so you're able to put any form you'd like in the actual transformation.

(expect (more-> 2 (-> first (+ 1))
                3 last)
  [1 2 3])

Finally, 'more-> can be very helpful for testing various kv pairs within a map, or various Java fields.

(expect (more-> 0 .size
                true .isEmpty)
   (java.util.ArrayList.))

(expect (more-> 2 :a
                4 :b)
   {:a 2 :b 4})

Threading is great work, if you can get it. For the times when you need to name your actual value, 'more-of should do the trick. The following example demonstrates how to name your actual value and then specify a few expectations.

(expect (more-of x
                 vector? x
                 1 (first x))
  [1 2 3])

If you've ever found yourself wishing you had destructuring in clojure.test/are or expectations/given, you're not alone. The good news is, 'more-of supports any destructuring you want to give it.

(expect (more-of [x :as all]
                 vector? all
                 1 x)
  [1 2 3])

from-each

It's common to expect something from a list of actual values. Traditionally 'given was used to generate many tests from one form. Unfortunately 'given suffered from many issues: no ability to destructure values, failure line numbers were almost completely useless, and little visibility into what the problem was when a failure did occur.

In expectations 2.0 'from-each was introduced to provide a more powerful syntax as well as more helpful failure messages.

Below you can see a very simple expectation that verifies each of the elements of a vector is a String.

(expect String
  (from-each [letter ["a" "b" "c"]]
    letter))

Hopefully the syntax of 'from-each feels very familiar, it's been written to handle the same options as 'for and 'doseq - :let and :when.

(expect odd? (from-each [num [1 2 3]
                         :when (not= num 2)]
               num))

(expect odd? (from-each [num [1 2 3]
                         :when (not= num 2)
                         :let [numinc1 (inc num)]]
               (inc numinc1)))

While 'from-each is helpful in creating concise tests, I actually find it's most value when a test fails. If you take the above test and remove the :when, you would have the test below.

(expect odd? (from-each [num [1 2 3]
                         :let [numinc1 (inc num)]]
               (inc numinc1)))

The above test would definitely fail, but it's not immediately obvious what the issue is. However, the failure message should quickly lead you to the underlying issue.

    failure in (success_examples.clj:206) : success.success-examples
    (expect
     odd?
     (from-each [num [1 2 3] :let [numinc1 (inc num)]] (inc numinc1)))

               the list: (3 4 5)

    (expect odd? (inc numinc1))

                 locals num: 2
                        numinc1: 3
               4 is not odd?

As you can see above, when 'from-each fails it will give you values of every var defined within the 'from-each bindings. As a result, it's fairly easy to find the combination of vars that led to a failing test.