CIT 591 Unit Testing (Python)
Fall 2010, David Matuszek

Tests and Testability

The most powerful new programming technique--TDD, or Test-Driven Development--results in better, more correct code, and practically guarantees a continuous rate of progress. However, TDD depends heavily on the ability to write and execute tests much more easily than in the past. Therefore, we first discuss the unit testing framework.

Unit Testing

The first unit testing framework, JUnit, was invented by Kent Beck and Erich Gamma in 1997, for testing Java programs. It was so successful that the framework has been reimplemented in every major programming language. Here we discuss the Python version, unittest.

Advantages of testing

First and foremost, of course, a program that has been thoroughly tested will have many fewer bugs. But there are other ways in which testing improves code. A method that has been written with testing in mind:

The unit testing approach

Unit testing is testing individual "units," or functions, of a program. It does not have a lot to say about system integration--whether the various parts of a program fit together. That's a separate issue.

The goals of a unit testing framework are:
For a first example, we will use the following code:

def add(x, y):
    return x * y
and save it in a file named

We write our tests as a set of funTests are usually put in a separate file, and we'll do that here. We'll call the file Since it is a different file, it will have to import arithmetic in order to access our add method. It will also have to import unittest, Python's unit test framework.

Finally, we need to put our test methods in a class, and tell unittest to run the methods in that class. Because they are in a class, we need to use the variable self in several places. We haven't discussed classes yet, so for the time being, just follow the pattern shown below. Here's the complete structure:

import arithmetic   # import your module (leave off the .py)
import unittest

class TestArithmetic(unittest.TestCase):  #  use a meaningful name

    ## Your test methods go here. Indent your
    ## methods, because they belong inside the class.
unittest.main()  # outside the class--this tells the framework to run
Here's an actual test:

def testAdd(self):
    self.assertEqual(4, arithmetic.add(2, 2))
and here are the results of the test:
Ran 1 test in 0.009s


(plus several other lines, which you can ignore)
You probably noticed that there is a mistake in the add method. I "accidentally" picked a pair of numbers that would cause the test to pass (there is one other pair). We can put additional tests within the same method--in fact, it's just a Python method, so we can put arbitrary Python code in it. For example,

def testAdd(self):
    self.assertEqual(4, arithmetic.add(2, 2))      # first test
    self.assertEqual(10, arithmetic.add(3, 7))     # second test
    self.assertEqual(90, arithmetic.add(-10, 100)) # third test
With this code, the "first test" will pass, but the "second test" will fail. The failure will halt execution of this test method (your other test methods will still run), so the "third test" will never be used. The result looks like:

FAIL: testAdd (__main__.TestArithmetic)
---------------------------------------------------------------------- Traceback (most recent call last): File "/Volumes/THUMBDRIVE/PythonPrograms/", line 15, in testAdd self.assertEqual(10, arithmetic.add(3, 7)) # second test AssertionError: 10 != 21 ---------------------------------------------------------------------- Ran 1 test in 0.027s FAILED (failures=1) (plus several other lines, which you can ignore)

Notice that the test shows us:
How many tests should you have? You would normally test one "typical" case (for example, 3+7==10 above), and try to test every "extreme" case you can think of. For example, if you were to write and test a function to sort a list, some things you might consider are: What if the list contains some equal numbers? Do the first and last elements get moved to the correct position? Can you sort a 1-element list without getting an error? How about an empty list?

If your test method starts getting too long, or you are testing different aspects of your application method, feel free to create additional test methods.

Here are the rules for writing test methods:
There are two ways you can call the application methods (the methods being tested):
Here are some of the built-in test methods you can call. Each has an optional message parameter, to be printed if the test fails. In most cases, the message is superfluous; don't use it unless you really have something useful to add.
assertEqual(expectedResult, actualResult)
assertEqual(expectedResult, actualResult, message)

Test that the two values are exactly equal.
assertNotEqual(firstValue secondValue)
assertNotEqual(firstValue secondValue, message)

Test that the two values are different, and fail if they are equal.
assertAlmostEqual(expectedResult, actualResult)
assertAlmostEqual(expectedResult, actualResult, places)
assertAlmostEqual(expectedResult, actualResult, places, message)
Test that the two numeric values are equal, after rounding to places decimal places (default is 7).
assertAlmostNotEqual(expectedResult, actualResult)
assertAlmostNotEqual(expectedResult, actualResult, places)
assertAlmostNotEqual(expectedResult, actualResult, places, message)
Test that the two numeric values are equal, after rounding to places decimal places (default is 7).
assertTrue(booleanCondition, message)
Test that the booleanCondition is true.
assertFalse(booleanCondition, message)
Test that the booleanCondition is false.
assertRaises(exception, functionName, parameter, ..., parameter)
Test that the named function, when called with the given (zero or more) parameters, raises the given exception. Note that, for this assertion method, there is no option for a message parameter.
Fail the test. This is useful if you want to create your own tests, more complex that the above methods support.

Dealing with global variables

Recall that the order in which your test methods run is not guaranteed. If one of your tests changes the values of global variables, those changes may or may not have been performed when some other test is run. In order to test successfully, you need to restore all global variables to their initial state before each test. This is where the setUp(self) method comes in.

If you write a method setUp(self), that method will be called before each and every test. Like this:

list = a list of all your test methods
for testMethod in list:
    call setUp()
    call testMethod

Test Driven Development

Test-driven development takes a simple but counterintuitive approach: Test the code before you write it. In pseudo-Python, we can describe the process as:

  while the method isn't complete:
      write a test for the desired feature
      run all tests and make sure the new one fails
      while any test fails:
          add/fix just enough code to try to pass the tests
      refactor the code to make it cleaner

Except for very simple methods, you can usually write the method in stages. Try to take small steps. Write a test for the simplest case you can think of, then run it. The test should fail--if it doesn't, then either the method already handles that case, or you did something wrong in the test. This is a quick way to make sure you aren't wasting your time adding code to the method.

Next, write just enough code to make the test pass. The temptation may be strong to write more code than necessary, but resist. Don't write any code that you don't already have a test for. Make sure the new code passes, or debug it until it does.

At this point your method may not do everything it needs to, but it will be correct in the part that it does. Now is a good time to think briefly about how to refactor (make cleaner) the code. Is there a variable that could have a better name? Would it be more readable if you rearranged the parts of an if statement? Could you pull out a chunk of the code and make it a function? Did you repeat code? Refactoring is all about making the code better without changing what it does.

Advantages of TDD

Writing the tests first has advantages over and above simply having a good set of tests. By writing the tests first: