JUnit Factory Part 1: Generating Tests

Save/Share Google Yahoo! Digg It Reddit del.icio.us
My Zimbio

JUnit Factory is a free Eclipse plug-in from Agitar Software that generates characterization tests for your Java code. For more background on what characterization tests are, and how you use them, you’ll want read my post “Characterization Tests: How To Deal With Legacy Java Code”.

This article describes how to generate tests for a simple Java class and how to read the tests. Not all of your real code will be this simple, and not all generated tests will be this simple, either. But, bear with me as we start small and work our way up.

This post is part of a series:

1. Characterization Tests: How To Deal With Legacy Java Code
2. JUnit Factory Part 1: Generating Tests
3. JUnit Factory Part 2: Finding Regressions
4. JUnit Factory Part 3: Improving Code Coverage

Let’s create a simple class

If you want to follow along, you can download the characterization tests sample code as an Eclipse project archive.

The sample code defines a simplistic CreditCard type. It’s admittedly inappropriate for real world use, but it’ll make for excellent illustration. The complete class definition can be found in the sample code download. For brevity, let’s start by looking just at the constructor:

public class CreditCard

{

private String accountNumber;

private double creditLimit;

private double balance;

/**

* Construct a new CreditCard

* @throws CreditCardException if any parameter is invalid

*/

public CreditCard( String accountNumber, double creditLimit,

double balanceTransfer ) throws CreditCardException

{

// validate the account number

if (!accountNumber.matches(“(\\d{4}[ ]){3}\\d{4}”))

{

throw new CreditCardException(“Invalid credit card number”);

}

this.accountNumber = accountNumber;

// validate the credit limit

if (creditLimit <= 0)

{

throw new CreditCardException(“Credit limit must be positive”);

}

if (creditLimit > 15000)

{

throw new CreditCardException(“Credit limit may not exceed 15,000”);

}

this.creditLimit = creditLimit;

// validate the balance

validateBalance( balanceTransfer );

this.balance = balanceTransfer;

}

public double getBalance()

{

return balance;

}

public double getCreditLimit()

{

return creditLimit;

}

//

// …other methods omitted for clarity

//

}

The constructor is pretty straightforward. It accepts an account number, a credit limit, and a starting balance (our credit card company allows new accounts to be created with a transferred opening balance).

The behavior of our constructor is to validate all three of these parameters. The account number must be of the form “nnnn nnnn nnnn nnnn”. The credit limit must be positive, and our cautious credit card company never grants a credit limit greater than $15,000. Finally, the opening balance is validated by a method called validateBalance() which we’ll look at later.

We’ll generate characterization tests for this class by selecting the class in the Eclipse package explorer and clicking the test generation button on the toolbar as shown in the screenshot below:
JUnit Factory Test Generate Button

This will send our project to JUnit Factory which will analyze and execute our code, make observations about its behavior, and generate a series of unit tests which will capture those observations. This will take several seconds.

The generated tests

If you’re already familiar with JUnit, you know JUnit test cases for a class under test will extend junit.framework.TestCase. JUnit Factory extends this type with AgitarTestCase. This is the base class of all JUnit Factory generated tests. So, our CreditCard test case is declared like this:

import com.agitar.lib.junit.AgitarTestCase;

public class CreditCardAgitarTest extends AgitarTestCase

{

//

// …generated tests

//

}

AgitarTestCase provides additional data gathering and utility methods used by the tests. You don’t need to worry about adding the Agitar jar files to your classpath because the Eclipse plug-in will automatically configure your project the first time you send a test generation request.

If you’re following along, the tests you get back might look slightly different than what we’re about to look at – particularly if you’re typing it in by hand. Tests will vary based on the presence (or absence) of property getters and setters, other methods which might produce execution coverage, and the latest test generation techniques developed by Agitar Labs.

You can expect to get back several tests for the constructor. One positive test case looks like this:

public void testConstructor1() throws Throwable {

CreditCard creditCard = new CreditCard(“2298 9812 4566 1184”,

13.872842788696289,

13.872842788696289);

assertEquals(“creditCard.accountNumber”,

“2298 9812 4566 1184”,

getPrivateField(creditCard, “accountNumber”));

assertEquals(“creditCard.getCreditLimit()”,

13.872842788696289,

creditCard.getCreditLimit(),

1.0E-6);

assertEquals(“creditCard.getBalance()”,

13.872842788696289,

creditCard.getBalance(),

1.0E-6);

}

This test starts by instantiating a CreditCard with legal values (we’ll cover negative test cases later). Notice JUnit Factory figured out how to create credit card numbers in the required format. How is this possible? When analyzing the bytecode, JUnit Factory found this line:

if (!accountNumber.matches(“(\\d{4}[ ]){3}\\d{4}”))

Because regex was used validate the account number string, JUnit Factory used this information to create a legal parameter.

Arbitrary values appear for the credit limit and starting balance. These values, of course, have far too many digits after the decimal point to be legitimate monetary values. This is because there’s nothing in the code to indicate there should be only two digits after the decimal point. JUnit Factory simply chose values that allowed the constructor to execute without throwing an exception.

Following the constructor are a series of assertions reflecting the state of the CreditCard object observed by JUnit Factory. First, JUnit Factory observed the private attribute accountNumber was set to the same value as the accountNumber parameter. Notice this assertion uses reflection because accountNumber is a private attribute. There is no public getter for accountNumber.

The assertions for the credit limit and balance are similar, but notice JUnitFactory discovered JavaBeans-style getters for these attributes and uses them instead of reflection. This creates a test that is more readable. This also creates a test that is sensitive to changes in both the class’s constructor and some of its method members.

Here’s another interesting test generated for the constructor:

public void testConstructor2() throws Throwable {

CreditCard creditCard = new CreditCard(“2298 9812 4566 1184”,

15000.0,

0.0010);

assertEquals(“creditCard.accountNumber”,

“2298 9812 4566 1184”,

getPrivateField(creditCard,

“accountNumber”));

assertEquals(“creditCard.getCreditLimit()”,

15000.0,

creditCard.getCreditLimit(),

1.0E-6);

assertEquals(“creditCard.getBalance()”,

0.0010, creditCard.getBalance(),

1.0E-6);

}

This test looks similar to the previous test, but note that 15000.0 is passed for the credit limit. This time, the credit limit value is not arbitrary. By analyzing the constructor bytecode, JUnit Factory noticed that 15000.0 is an interesting value – a branch of execution is triggered based off this corner condition. Thus, JUnit Factory generates a test for this scenario.

JUnit Factory also generates negative test cases – tests that capture the behavior of a method when things go wrong. Here’s one such test generated to ensure an exception is thrown when the credit limit parameter is less than zero:

public void testConstructorThrowsCreditCardException3() throws Throwable {

try

{

new CreditCard(“2298 9812 4566 1184”, -0.0010, 100.0);

fail(“Expected CreditCardException to be thrown”);

}

catch (CreditCardException ex)

{

assertEquals(“ex.getMessage()”,

“Credit limit must be positive”,

ex.getMessage());

assertThrownBy(CreditCard.class, ex);

}

}

How many tests does JUnit Factory generate? In this case, four tests for normal outcomes, and sevens tests for exceptional outcomes. JUnit Factory attempts to generate just enough tests for 100% code coverage. You might wonder why there aren’t tests using a richer data set with hundreds of possible input combinations. The answer is simple: those tests don’t add value. There are no other input parameters that would provide additional code coverage.

Unit testing private methods

JUnit Factory easily supports testing of private methods by invoking those methods through reflection.

The second to last line of the CreditCard constructor invokes validateBalance() to test whether the opening balance is legal. Here’s how validateBalance() is implemented:

private void validateBalance( double balance ) throws CreditCardException

{

if (balance < 0.00)

{

throw new CreditCardException(“Balance can’t go below minimum balance”);

}

if (balance > creditLimit)

{

throw new CreditCardException(“Balance can’t exceed credit limit”);

}

}

This method tests whether a proposed balance value falls within a range. The behavior has been extracted to a private method because it is shared by multiple methods. JUnit Factory is able to independently test this behavior using reflection. Four such tests, two positive and two negative, are generated for validateBalance(). Here’s one example:

public void testValidateBalance() throws Throwable {

CreditCard creditCard = new CreditCard(“2298 9812 4566 1184”,

100.0, 0.0);

callPrivateMethod(“example1.CreditCard”,

“validateBalance”,

new Class[] {double.class},

creditCard,

new Object[] {new Double(0.0)}

);

assertEquals(“creditCard.getCreditLimit()”,

100.0,

creditCard.getCreditLimit(),

1.0E-6);

}

This test creates a CreditCard object and invokes validateBalance() by passing 0.0. Afterward, the test asserts that the validation method, which utilizes the creditLimit attribute, leaves the credit limit value unchanged.

This test is harder to read, but it is this sort of complexity that makes unit testing private methods costly and practically nonexistent in tests created by hand. With JUnit Factory, the test is written with the click of a button.

This feature adds tremendous value. Independently testing private methods improves a developer’s ability to track down the source of behavioral changes whenever tests fail.

This post is continued:
JUnit Factory Part 1: Generating Tests (page 2) »

Save/Share Google Yahoo! Add to Technorati Favorites Digg It Reddit
del.icio.us My Zimbio

Leave a Reply