CIT 590 Unit Testing and TDD (Python)
Spring 2013, David Matuszek

Tests and Testability

The most powerful new programming technique--TDD, or Test-Driven Development--results in better, more correct code, and practically guarantees a continuous rate of progress. However, TDD depends heavily on the ability to write and execute tests much more easily than in the past. Therefore, we first discuss the unit testing framework.

Unit Testing

The first unit testing framework, JUnit, was invented by Kent Beck and Erich Gamma in 1997, for testing Java programs. It was so successful that the framework has been reimplemented in every major programming language. Here we discuss the Python version, unittest.

Unit testing is testing individual "units," or functions, of a program. It does not have a lot to say about system integration--whether the various parts of a program fit together. That's a separate issue.

The goals of a unit testing framework are:

Let's be honest: If it isn't easy, nobody is going to do it. Programmers are only human. The genius of the JUnit framework (and similar frameworks for just about every language) is that it does make it--well--a lot easier, anyway. Easy enough so that the next time you are too lazy to write good tests, and consequently spend hours debugging, you'll promise yourself to use TDD the next time.

For a first example, we will use the following code:

def add(x, y):
    return x * y

and save it in a file named arithmetic.py.

We write our tests as a set of functions. Tests are usually put in a separate file, and we'll do that here. We'll call the file testArithmetic.py. Since it is a different file, it will have to import arithmetic in order to access our add method. It will also have to import unittest, Python's unit test framework.

Finally, we need to put our test methods in a class, and tell unittest to run the methods in that class. Because they are in a class, we need to use the variable self in several places. We haven't discussed classes yet, so for the time being, just follow the pattern shown below. Here's the complete structure:

from arithmetic import *   # import everything from your module
import unittest  # This loads the testing methods and a main program

class TestArithmetic(unittest.TestCase):  # use any meaningful name

    ## Your test methods go here. Indent your
    ## methods, because they belong inside the class.
    
unittest.main()  # outside the class--this tells the framework to run
Here's an actual test:
def testAdd(self):
    self.assertEqual(4, add(2, 2))
and here are the results of the test:
.
----------------------------------------------------------------------
Ran 1 test in 0.009s

OK

(plus several other lines, which you can ignore)
You probably noticed that there is a mistake in the add method. I "accidentally" picked a pair of numbers that would cause the test to pass (there is one other pair). We can put additional tests within the same method--in fact, it's just a Python method, so we can put arbitrary Python code in it. For example,
def testAdd(self):
    self.assertEqual(4, add(2, 2))      # first test
    self.assertEqual(10, add(3, 7))     # second test
    self.assertEqual(90, add(-10, 100)) # third test
With this code, the "first test" will pass, but the "second test" will fail. The failure will halt execution of this test method (your other test methods will still run), so the "third test" will never be used. The result looks like:
F
======================================================================
FAIL: testAdd (__main__.TestArithmetic)
---------------------------------------------------------------------- Traceback (most recent call last): File "/Volumes/THUMBDRIVE/PythonPrograms/arithmeticTest.py", line 15, in testAdd self.assertEqual(10, arithmetic.add(3, 7)) # second test AssertionError: 10 != 21 ---------------------------------------------------------------------- Ran 1 test in 0.027s FAILED (failures=1) (plus several other lines, which you can ignore)
Notice that the test shows us:

How many tests should you have?

While you can put as many tests as you like into one test method, you shouldn't. Test methods (like all methods) should be short and single-purpose. If you are testing different aspects of a function, they should be in separate tests.

Here are the rules for writing test methods:
There are two ways you can call the application methods (the methods being tested):
Here are some of the built-in test methods you can call. Each has an optional message parameter, to be printed if the test fails--the brackets, [ ], indicate an optional argument that can be omitted. In most cases, the message is superfluous; don't use it unless you really have something useful to add.
self.assertEqual(expectedResult, actualResult, [message])
Test that the two values are exactly equal.
self.assertNotEqual(firstValue secondValue,[message])
Test that the two values are different, and fail if they are equal.
self.assertAlmostEqual(expectedResult, actualResult, [places,[message]])
Test that the two numeric values are equal, after rounding to places decimal places (default is 7).
self.assertAlmostNotEqual(expectedResult, actualResult,[places,[message]])
Test that the two numeric values are equal, after rounding to places decimal places (default is 7).
self.assertTrue(booleanCondition,[message])
Test that the booleanCondition is true.
self.assertFalse(booleanCondition,[message])
Test that the booleanCondition is false.
self.assertRaises(exception, functionName, parameter, ..., parameter)
Test that the function functionName, when called with the given (zero or more) parameters, raises the given exception. Note that, for this assertion method, there is no option for a message parameter.
self.fail([message])
Fail the test. This is most useful if you want to create your own tests, more complex that the above methods support.

Dealing with global variables

If you write a method setUp(self), the unittest framework will call setUp before each and every one of your tests.

Remember that the order in which your test methods run is not guaranteed. If one of your tests changes the values of global variables, those changes may or may not have been performed when some other test is run. In order to test successfully, you need to restore all global variables to their initial state before each test. Do this initialization in the setUp(self) method.

Better yet, don't use global variables. There is almost always a better way to do things.

Test Driven Development (TDD)

Test-driven development takes a simple but counterintuitive approach: Test the code before you write it. In pseudo-Python, we can describe the process as:
  pick a method that doesn't depend on other, untested methods
  while the method isn't complete:
      write a test for the desired feature
      run all tests and make sure the new one fails
      while any test fails:
          add/fix just enough code to try to pass the tests
      refactor the code to make it cleaner

Except for very simple methods, you can usually write the method in stages. Try to take small steps. Write a test for the simplest case you can think of, then run it. The test should fail--if it doesn't, then either the method already handles that case, or you did something wrong in the test. This is a quick way to make sure you aren't wasting your time adding code to the method.

Next, write just enough code to make the test pass. The temptation may be strong to write more code than necessary, but resist. Don't write any code that you don't already have a test for. Make sure the new code passes, or debug it until it does.

At this point your method may not do everything it needs to, but it will be correct in the part that it does. Now is a good time to think briefly about how to refactor (make cleaner) the code. Is there a variable that could have a better name? Would it be more readable if you rearranged the parts of an if statement? Could you pull out a chunk of the code and make it a function? Did you repeat code? Refactoring is all about making the code better without changing what it does.

How TDD improves the quality of your code

A program that has been thoroughly tested:

How TDD improves your programming experience