Unit Tests in Racket

CSC 261 - Artificial Intelligence - Weinman

Summary:: This guide provides information on how to do unit testing in Racket.

Introduction

Of course, unit tests are a great way to verify your program (especially pure functions, which comprise most of what we'll do here) has the desired behavior. We encourage you to use them wherever possible to both document and verify the expected results of your procedures on a variety of inputs, specifically inputs that ensures every path through your code

The complete guide to Racket Unit Testing may be found here:

Welsh, N. and Culpepper, R. RackUnit: Unit Testing. https://docs.racket-lang.org/rackunit

We give a very brief overview of how one might use this unit testing framework. At a high level, your testing program will consist of one or more test suites, each of which is comprised of one or more tests; each test is comprised of one or more checks. We proceed with describing these from the bottom-up : checks to tests to suites.

Writing

Format

To import the RackUnit library routines, you need to include the following at the top of your test file (the after #lang racket line)

: (require rackunit) ; includes the general framework
(require rackunit/text-ui) ; includes (optional) interactions commands

Checks

The basic unit test consists of checks that assert some property holds. The following are those you may most commonly use:

(check-equal? expression expected message)

Checks whether the value of expression, when evaluated, is equal? to that of expected. If not, the check fails and the error is reported with the given message string.

: (check-equal? (reverse (list 1 2 3)) (list 3 2 1) "three-element list")

(check-pred predicate? val message)

Checks whether predicate? is not #f when applied to val. If it is, the check fails and the error is reported with the given message string.

: (check-pred even? (* 2 437) "doubling a value is even")

(check-= expression expected tolerance message)

Checks whether the value of expression, when evaluated, is within tolerance of expected (all three values must be numbers). If not, the check fails and the error is reported with the given message string.

: (check-= (sqrt 16) 4 1e-12 "root of a perfect square")

(check-true expression message), (check-false expression message), (check-not-false expression message)

Checks whether the value of expression is #t, #f or not #f (aka true-ish), respectively. If not, the check fails and the error is reported with the given message string.

There are several more checks, and you can find them in the RackUnit documentation linked above.

When run on their own (i.e., outside a test suite), they evaluate immediately. Furthermore, a check produces no output when it succeeds. When it fails, a descriptive error message is printed. It is helpful to describe the motivating purpose of the check in your message string because the expected result is itself printed when the error arises.

Tests

Typically we want to group checks together into a test that corresponds to a general category of property we'd like to assess, where each check may be one example from that category. The test-case syntax groups a sequence of checks together, evaluates them in order, and stops evaluating the checks if one of them fails.

(test-case description check-1 ...)

Runs check-1 (and any subsequent expressions, generally checks, as described above) and prints the description of the first failed check in an error message (halting further execution of arguments), or else no output if all the checks succeed.

: (test-case "positive values"
           (check-= (abs 1) 1 0 "edge case")
           (check-= (abs 1000) 1000 0 "bigger number")
           (check-= (abs 1.234) 1.234 0 "inexact values"))

Sometimes we have only a single check for a test case, and Racket offers some convenient shortcuts corresponding to the checks given above:

(test-equal? description expression expected)
(test-pred description predicate? val)
(test-= description expression expected tolerance)
(test-true description expression), (test-false description expression), (check-not-false description expression)

When one of the test-____ procedures is run on its own, the checks therein are run; if any one of them fails, the check's error message is printed along with the high-level description given. No display output is produced when the checks all succeed.

Suites

In RackUnit, test suites are motivated by at least two considerations:

We often want to define our tests without actually running them right away.
It is useful to group test cases together, particularly so multiple program properties can be verified (each with its own test case).

We group test cases using the test-suite syntax:

(test-suite name test-1 ...)

Creates a test suite with the given name that will execute the test parameters in sequence when invoked

: (test-suite "absolute value (abs)"
  (test-case "positive values"
             (check-= (abs 1) 1 0 "edge case")
             (check-= (abs 1000) 1000 0 "bigger number")
             (check-= (abs 1.234) 1.234 0 "inexact values"))
  (test-= "zero value" (abs 0) 0 0)
  (test-case "negative values"
             (check-= (abs -1) 1 0 "edge case")
             (check-= (abs -1234) 1234 0 "bigger number")
             (check-= (abs -1.234) 1.234 0 "inexact values"))

As indicated above, the resulting test suites are not run right away, but must be invoked by some other means, which we describe next.

Interestingly, the arguments to test-suite may be other test suites (allowing suites of suites).

Running

There are two ways to run your tests: in the interactions or from the command-line. In either case, we typically ask you to put your tests into a separate file from the file containing the definitions of your procedures. This is for two reasons.

Your assignment may require a mix of testing methods (some involving unit tests, but others may be simply demonstration outputs)
It keeps the implementation files (some of which are provided as template or starter code) fairly uniform in content, and maximizes the coherence (test versus implementation) of each file

We briefly show how to use each method below. In both cases, the file containing the implementation should be required in the test file. For instance, if myfun is defined and provided in implementation.rkt, as in the following

: #lang racket
;; File: implementation.rkt
(provide myfun) ; explicitly export the name to other files

(define myfun
  (lambda (args)
     ;; calculate cool stuff here
  ))

then the tester.rkt (in the same directory) can then require the implementation and then create its tests:

: #lang racket
;; File: tester.rkt
(require rackunit)
(require rackunit/text-ui)

(require "implementation.rkt")

;; tests may go here ...

Interactions

To run the test suites explicitly (as when running the definitions or through the interactions pane), you will need to define the suites for reference with the run-tests procedure.

: (define abs-tests
(test-suite "absolute value (abs)"
;; see the rest above

After this definition, you can give the call to

: (run-tests abs-tests)

putting it either in your definitions (i.e., within tester.rkt)itself, or by typing the command in the interaction pane if, for some reason, you didn't want the tests to execute every time the definitions got run.

Note that run-tests returns the number of failed tests in the suite and prints a summary diagnostic (number of tests cases run, passed, and failed) as well as any error messages that arise from failed checks.

Command-Line

The Racket command line tool raco will execute the script given, producing all output (i.e., created by display, printf, etc.) and reporting additionally the total number of tests passed (e.g., "26 tests passed" or the fractional number of test failures among the gests run (e.g., "4/26 test failures") before exiting.

: $ raco test tester.rkt