MacDevCenter    
 Published on MacDevCenter (http://www.macdevcenter.com/)
 See this if you're having trouble printing code examples


Unit Testing with OCUnit

by Jim Menard
04/23/2004

Are you sure your code works -- all of it? If you make a change in one place, are you sure you haven't broken something else? When you fix a bug, how do you know that it stays fixed?

Testing frameworks helps you make sure. They provide a way to write and run unit tests consisting of test cases: groups of small tests that exercise a particular class or feature. For example, one test case may exercise a Checkbook class, making sure that it adds and deletes entries properly and returns the correct account balance value. Another test case may exercise a CheckbookEntry class, making sure that it accepts and returns the monetary amount you give it.

Much of the XUnit literature describes the ideal development process as follows.

  1. Write a test case for a single feature.
  2. Run the test and watch it fail.
  3. Write the code that implements the feature.
  4. Make it work.
  5. GOTO 1.

This is called Test-Driven Development or Test First Development. Practically speaking, I find myself writing the code first and then writing a test case for it soon after. My development process ends up being:

Related Reading

Learning Cocoa with Objective-C
By James Duncan Davidson, Apple Computer, Inc.

  1. Write a large chunk of the application.
  2. Write test suites that exercise everything written so far.
  3. Fix the bugs found by the tests.
  4. Write a few more tests; fix some more bugs.
  5. Add new features.
  6. Write tests for the new features.
  7. GOTO 5.

This process lets me experiment and make a few design mistakes during step 1, perhaps starting from scratch a few times. Since I wait before writing the initial set of test cases, I don't have to write and rewrite tests that are exercising code that I'm probably going to throw away.

When I developed the simple OS X checkbook register program CheckbooX, I didn't write any unit tests. I used the program myself, but had no plans to release it. To further excuse this unconscionable behavior, I was not aware of any XUnit testing frameworks for Objective-C at the time. In this article, I am going to describe testing frameworks and walk through the process of adding unit tests to CheckbooX. The complete unit tests for CheckbooX are included in the source.

Adding unit tests to an existing application is harder than starting out with unit tests, for two reasons. First, your classes are not designed with testing in mind. For example, they may be tightly coupled, making it difficult to write tests that exercise single classes or application features. Second, motivating yourself to write tests for code you already know works can be difficult.

Testing Frameworks

Writing an individual test case isn't hard. Running all of your test cases and interpreting the results can be, unless you use a testing framework that takes care of all of those details for you.

There is a family of related testing frameworks, collectively called XUnit, that includes JUnit for Java; SUnit for Smalltalk; RUnit, which now ships with Ruby; and more. Until recently, I wasn't aware of any XUnit testing frameworks for Objective-C. I didn't look hard enough. There are at least two: OCUnit by Sen:te and TestKit by Both are open source projects and are freely available. TestKit works with ProjectBuilder, and OCUnit works with Xcode or ProjectBuilder. They both include programs that let you run your tests from the command line. OCUnit also lets you run your tests from within Xcode or ProjectBuilder. TestKit comes with a separate GUI test runner application.

Most test frameworks let you run one or more suites of tests. Each suite is most often made up of all of the methods in a single class that have names that begin with "test." Languages such as Java, Ruby, and Objective-C allow the framework to discover such methods at runtime.

The output of a test run is usually a summary of the test results: X tests passed, Y tests failed. If there are any failures, the frameworks usually display the error messages you supplied or those supplied by your runtime environment. Note that "failure" is relative: if you want your code to fail, then you can write a test that makes sure that it fails properly. When your code fails, the test succeeds. Some testing frameworks include a GUI application that show you a "green light" if all tests pass or a "red light" if something went wrong.

A single test contains one or more assertions: statements that, when false, signal an error. OCUnit defines a series of macros such as STAssertEquals, STAssertEqualObjects, STAssertTrue, and STFail. Here is an example test:


- (void) testEntryAmount {
    // entry is created in setUp; see below
    [entry setAmountInPennies:123];
    STAssertEquals(123L, [entry amountInPennies],
                   @"bad amount; 123 != %ld",
                   [entry amountInPennies]);
    STAssertEquals((float)1.23, [entry amount],
                   @"bad amount; 1.23 != %f",
                   [entry amount]);
}

The custom bad amount error messages aren't really all that useful; OCUnit shows you the two values if they don't match.

The macro STAssertEquals takes at least three arguments. The first two are the values to be compared. The third is the message to print if the test fails. It is not optional, but it may be nil. This argument is just like an NSLog string; it is a format string and any number of additional arguments may be used.

Since a test case often needs an object to test, testing frameworks provide a way to set up common conditions before each test and to tear down after each test is run. These methods, cleverly called setUp and tearDown in OCUnit, are run before and after each test method is run. For example, the test class that contains the testEntryAmount method also contains the code:


- (void) setUp {
    // entry is declared in the @interface;
    // that is not shown here.
    entry = [[CheckbookEntry alloc] init];
}

- (void) tearDown {
    [entry release];
}

Writing test cases for non-GUI code is easy. Testing your GUI is harder. There are testing frameworks out there for web-based GUIs, Java GUIs, and more. This article completely glosses over this important aspect of application testing. Performing a web search for a phrase like "gui testing framework" will help you find plenty of relevant information.

Installing OCUnit

When you download OCUnit, you must choose which version you want: Xcode or ProjectBuilder. Additionally, you need to choose either a version with an installer that puts OCUnit in the root level of your hard drive, or a version that comes with a script that installs OCUnit in your home directory. Some of the versions are disk images, some are .zip files, and some are tarballs. The disk images are for the Mac-OS-X-only versions of OCUnit. The versions of OCUnit that work equally well with GNUSTEP, Rhapsody, and friends are packaged as tarballs.

I downloaded OCUnitRoot-v37.dmg, which is the version for Xcode that installs OCUnit at the root level. The installer launched automatically and showed me exactly what it would be installing and where it goes. The documentation gets installed at /Developer/Source/OCUnit/Documentation/index.html.

OCUnit adds a number of templates to those available when you create a new project. For example, the "Cocoa Application + Test" template creates a project that links against Cocoa and the SetTestingKit frameworks. You need to decide where you want the test cases to go: in a separate project, in the same project but a different target, or in the same project and target.

To add OCUnit tests to CheckbooX, I decided to create a new target in the same project and to configure the project so that the tests were run each time the test target is built. Following the instructions in the OCUnit documentation, here is what I did:

  1. Opened the CheckbooX project.
  2. Right-clicked on "Targets" and selected "Add/New Target."
  3. In the dialog that appeared, selected "Test Framework" from under "Cocoa."
  4. Named it "Testing" and clicked "Finish."
  5. Noticed that the new target already has a shell script build phase and the build setting TEST_AFTER_BUILD as described in the OCUnit documentation.
  6. Found SenTestingKit.framwork in /Library/Frameworks and dragged it into the Linked Frameworks folder inside of the External Frameworks folder. I unchecked the "CheckbookX" target and instead checked the "Testing" target.
  7. Just to make sure that everything was fine so far, I recompiled and ran CheckbooX.
  8. Built the new testing target and verified that nothing happens. That's because we haven't written any test code yet.

Writing Tests

The easiest way to implement test cases using OCUnit is to implement them as subclasses of SenTestCase. Here is a complete (though by no means thorough) test case for the CheckbooX CheckbookEntry class.


// TestCheckbookEntry.h
#import <SenTestingKit/SenTestingKit.h>

@class CheckbookEntry;

@interface TestCheckbookEntry : SenTestCase
{
    CheckbookEntry *entry;
}

// Note that you don't have to declare your test
// methods here.

@end

// TestCheckbookEntry.m
#import "TestCheckbookEntry.h"
#import "CheckbookEntry.h"

@implementation TestCheckbookEntry

- (void) setUp {
    entry = [[CheckbookEntry alloc] init];
}

- (void) tearDown {
    [entry release];
}

- (void) testEntryAmount {
    [entry setAmountInPennies:123];
    STAssertEquals(123L, [entry amountInPennies],
                   @"bad amount; 123 != %ld",
                   [entry amountInPennies]);
    STAssertEquals((float)1.23, [entry amount],
                   @"bad amount; 1.23 != %f",
                   [entry amount]);
}

@end

I added a new "Testing" group to the top level of the project. Next, I created a new Objective-C class named TestCheckbookEntry, remembering to uncheck the "CheckbooX" target and to check the "Testing" target instead.

I entered and saved the code above. Next, I tried running this simple test.

Objective-C Pocket Reference

Related Reading

Objective-C Pocket Reference
By Andrew M. Duncan

Running Tests

I made sure that "Testing" was the active target and clicked the "Build" button. After a few false starts (I started with C strings instead of NSStrings in the error message arguments), the code compiled. The linker then complained that it didn't know about CheckbookEntry. So I added both CheckbookEntry and Checkbook to the "Sources" section of the "Testing" target, clicked "Build," and voilà! I saw in the build window:


// ...build stuff...
-[TestCheckbookEntry testEntryAmount] :
Type mismatch -- bad amount; 123 != 123
-[TestCheckbookEntry testEntryAmount] :
Type mismatch -- bad amount; 1.23 != 1.230000
-[TestCheckbookEntry testEntryAmount] :
<3ff3ae14 7ae147ae >' should be equal to
'<3f9d70a4 >' bad amount; 1.23 != 1.230000

The code we saw in the previous section did not cause these errors; that code is correct. These errors were caused by my original code, which looked like this:


- (void) testEntryAmount {
    [entry setAmountInPennies:123];
    STAssertEquals(123, [entry amountInPennies],
                   @"bad amount; 123 != %ld",
                   [entry amountInPennies]);
    STAssertEquals(1.23, [entry amount],
                   @"bad amount; 1.23 != %f",
                   [entry amount]);
}

I simplified the test case by removing all but the first Assert and removing my custom error message. This is what the code looked like.


- (void) testEntryAmount {
    [entry setAmountInPennies:123];
    STAssertEquals(123, [entry amountInPennies],
                   nil);
    // commented-out old versions not shown
}

I clicked "Build" and saw the error message:


-[TestCheckbookEntry testEntryAmount] :
Type mismatch --

The clue I needed was in the phrase "Type mismatch." After a bit of thinking, I looked at the amountInPennies method. Aha! It returns a long, not an int. I changed the first argument of the test to 123L like this:


- (void) testEntryAmount {
    [entry setAmountInPennies:123];
    STAssertEquals(123L, [entry amountInPennies],
                   nil);
    // commented-out old versions not shown
}

I rebuilt the test target, and it worked! I then wrote all the other tests, which worked perfectly the first time. (If you believe that, I've got some dot-com stock options for you.)

Configurable Tests

In order to test much of the code, we need a checkbook object that contains things to find. We could either create the data manually and feed it to the checkbook object or read it from an XML file (CheckbooX stores its data as XML). Let's use the latter method. That way, we can change the test data without having to recompile the code.


- (void) setUp {
    checkbook = [[Checkbook alloc] init];
    [checkbook openFile:TEST_CHECKBOOK_FILE];
}

While we're at it, let's store the expected answers in an XML file, too. Then we can write test methods that don't know the expected answer; they just have to know the name of the answer in the dictionary and the type of the answer. Since we want more than one test class to read an answer file, let's abstract that behavior into a new superclass, TestWithAnswers. The method TestWithAnswers.readAnswersFrom: and the first few answerAs* methods look like this:


-(void)readAnswersFrom:(NSString *)filePath {
    NSArray *contents = [NSArray arrayWithContentsOfFile:filePath];
    NSEnumerator *e = [contents objectEnumerator];
    answers = [[e nextObject] retain];
}

-(NSString *)answerAsString:(NSString *)answerName {
    return (NSString *)[answers objectForKey:answerName];
}

-(int)answerAsInt:(NSString *)answerName {
    return [[answers objectForKey:answerName] intValue];
}

Now we can write test methods that don't have the answer hard-coded in the test. For example, here is the test method TestCheckbook.testBalance:


- (void) testBalance {
    STAssertEquals([self answerAsFloat:
                       @"balance"],
                   [checkbook balance],
                   nil);
    STAssertEquals([self answerAsFloat:
                       @"markedBalance"],
                   [checkbook markedBalance],
                   nil);
}

Final Thoughts

Now when I make a change to CheckbooX, I rebuild the testing target that runs all of the unit tests. Since writing the unit tests for CheckbooX, I feel more confident that any changes I make -- whether they're bug fixes or new features -- will not break the existing code. Though the initial set of tests is sparse, over time I will be adding more tests to CheckbooX that cover more of the code. Any new Mac OS X applications I write will be developed with unit tests from the start.

Testing is good. Just do it.

Resources

The Portland Pattern Repository and WikiWikiWeb top-level pages are deceptively sparse. Browse the main Category page or the Category Testing page. Search for programming terms and you'll come across an amazingly deep ocean of discussion. For example, searching for page titles containing "test" returns 529 pages.

JUnit.org is a good jumping-off point for learning about JUnit and other XUnit frameworks.

Jim Menard is an independent consultant who has developed many open source projects, including the Java GUI report writer DataVision, the first pure-Ruby XML parser NQXML, TwICE (an Information and Content Exchange (ICE) reference implementation), and, of course, CheckbooX.


Return to the Mac DevCenter

Copyright © 2009 O'Reilly Media, Inc.