Expectations

a minimalist's unit testing framework

This project is maintained by seancorfield and jaycfields

expectations

adding signal, removing noise

Stacktraces, Test Names, Focused Tests, READMEs, & JUnit Integration

The previous sections on expectations unit testing syntax cover all of the various ways that expectations can be used to write tests and what you can expect when your tests fail. However, there are a few other things worth knowing about expectations.

Stacktraces
expectations aggressively removes lines from the stacktraces. Just like many other aspects of expectations, the focus is on more signal and less noise. Any line in the stacktrace from clojure.core, clojure.lang, clojure.main, and java.lang will be removed. As a result any line appearing in your stacktrace should be relevant to your application or a third-party lib you're using. expectations also removes any duplicates that can occasionally appear when anonymous functions are part of the stacktrace. Again, it's all about improving signal by removing noise. Speaking of noise...

Test Names
You might have noticed that expectations does not require you to create a test name. This is a reflection of my personal opinion that test names are nothing more than comments and shouldn't be required. If you desire test names, feel free to drop a comment above each test. Truthfully, this is probably a better solution anyway, since you can use spaces (instead of dashes) to separate words in a comment. Comments are good when used properly, but they can become noise when they are required. The decision to simply use comments for test names is another example of improving signal by removing noise.

By contrast, if you use Expectations' clojure.test integration with defexpect, names are required because that's how clojure.test knows which functions (tests) to run, how to run only failed tests, and so on.

Running Focused Expectations
Sometimes you'll have a file full of expectations, but you only want to run a specific expectation - expectations solves this problem by giving you 'expect-focused'. If you use expect-focused only expectations that are defined using expect-focused will be run.

For example, if you have the following expectations in a file you should see the following results from 'lein expectations'.
(ns sample.test.core
  (:require [expectations :refer :all]))

(expect zero? 0)
(expect zero? 1)
(expect-focused nil? nil)

jfields$ lein expectations
Ran 1 tests containing 1 assertions in 2 msecs
IGNORED 2 EXPECTATIONS
0 failures, 0 errors.
As you can see, expectations only ran one test - the expect-focused on line 6. If the other tests had been run the test on line 5 would have created a failure. It can be easy to accidentally leave a few expect-focused calls in, so expectations prints the number of ignored expectations in capital letters as a reminder. Focused expectation running is yet another way to remove noise while working through a problem.
Note that focused expectations are only available with Expectations-specific tooling, using the original, unnamed test approach. If you use Expectations' clojure.test integration with defexpect, you can use regular metadata on tests and Leiningen's "test selectors" approach to choose which tests to run. With Boot, you can use boot-test's filters to achieve the same thing.

Test Running
If you always use 'lein expectations' to run your tests you'll never even care; however, if you ever want to run individual test files it's important to know that your tests run by default on JVM shutdown. When I'm working with Clojure and Java I usually end up using IntelliJ, and therefore have the ability to easily run individual files. When I switched from clojure.test to expectations I wanted to make test running as simple as possible - so I removed the need to specify (run-all-tests). Of course, if you don't want expectations to run for some reason you can disable this feature by calling (expectations/disable-run-on-shutdown).

READMEs
You project's README.md file is often chock full of examples that just scream out "Test me!".

As of version 2.2.0-beta3, Expectations adds the ability to generate a readme.clj test file from a specified README file and then run the examples as tests automatically.

Specify the system property expectations.readme or the environment variable EXPECTATIONS_README to identify the README file -- in the project.clj file for Expectations itself, you'll see examples.md identified via the system property.

You can optionally specify the output folder via the system property expectations.test.path or the environment variable EXPECTATIONS_TEST_PATH. It defaults to test -- in the project.clj file for Expectations itself, you'll see test/clojure identified via the system property.

If the README file is newer than readme.clj in the output folder -- or the latter does not exist -- then Expectations will generate readme.clj from that file, maintaining line numbers (hopefully!), and then require the new test file (so that it is run as part of the test suite). If readme.clj is newer, it will just be treated as a normal test file.

Example code in your README that is introduced with a triple back-tick and either the language identifier clojure or clj is copied into the readme.clj file. A line beginning with => is assumed to be the expected result of evaluating that code and becomes the "expected" value (and the lines of code prior to => are treated as the "actual" value). A code block that has no => is copied directly to readme.clj so your examples can contain separate blocks of setup code. For example:
```clojure
(* 1 2 3)
=> 6
```
will generate:

(expectations/expect 6
  (* 1 2 3))

A new expectations/readme function has been added with can be invoked by tooling with the file path of the input (README) file and the folder path where readme.clj should be written. If this function is invoked with no arguments -- which happens when the expectations namespace is loaded! -- it uses the system properties and environment variables (above) to determine the input file path and output file path.

JUnit Integration
Lack of JUnit integration was a deal breaker for my team in the early days, so expectations comes with an easy way to run all tests as part of JUnit. If you want all of your tests to run in JUnit all you need to do is implement ExpectationsTestRunner.TestSource. The following example is what I use to run all the tests in expectations with JUnit.
import expectations.junit.ExpectationsTestRunner;
import org.junit.runner.RunWith;

@RunWith(expectations.junit.ExpectationsTestRunner.class)
public class SuccessTest implements ExpectationsTestRunner.TestSource{

    public String testPath() {
        return "test/clojure/success";
    }
}
As you can see from the example above, all you need to do is tell the test runner where to find your Clojure files.