JUnit Factory Part 2: Finding Regressions

Save/Share Google Yahoo! Digg It Reddit del.icio.us
My Zimbio

Characterization tests provide a safety net for your legacy Java code by helping identify unintended changes in software behavior caused by code maintenance. JUnit Factory from Agitar Software may be used to automatically generate these tests for you. In this post, we’ll take a look at what happens to these characterization tests when a simple code change is made.

This post is part of a series:

1. Characterization Tests: How To Deal With Legacy Java Code
2. JUnit Factory Part 1: Generating Tests
3. JUnit Factory Part 2: Finding Regressions
4. JUnit Factory Part 3: Improving Code Coverage

A simple requirements change

Suppose we need to implement a simple requirements change to the CreditCard class we’ve already created. Presently, the validateBalance() method disallows negative balances. The business leadership of our company, however, has decided to allow customers to overpay their credit card balances.

Before writing any code, we must first verify that all of our current characterization tests still pass. The fastest way to do this is to select the source folder containing our characterization tests in the Eclipse Package Explorer. Right click on the folder and select “Run As / Agitar JUnit Test”.

If any tests fail, it means some other developer has already introduced a behavior change into the system without addressing how this affects tests. If all of the tests pass, we know we have a safety net. We may now move forward making changes confident any regressions will be detected before we commit our work to version control.

Credit card payments are recorded in the makePayment() method:

public void makePayment( double amount ) throws CreditCardException

{

if (amount <= 0)

{

throw new CreditCardException(“Payment amount must be positive”);

}

validateBalance( balance – amount );

balance = balance – amount;

}

After ensuring a postive payment amount has been made, the method invokes validateBalance() on the proposed new balance to check if it’s legal.

Presently, validateBalance() disallows negative values, but we’ll comment out the section of the method that performs this check:

private void validateBalance( double balance ) throws CreditCardException

{

//if (balance < 0.00)

//{

// throw new CreditCardException(“Balance can’t go below minimum balance”);

//}

if (balance > creditLimit)

{

throw new CreditCardException(“Balance can’t exceed credit limit”);

}

}

The makePurchase() method now allows payments that exceed the outstanding balance on the credit card.

Rerun the tests to find behavior changes

When we rerun our characterization tests, we discover that three of the tests are now failing. It shouldn’t come as any surprise, though, since changing the behavior of the class is what we intended to do.

So, we should just regenerate the CreditCard tests, right? Absolutely not! We must first analyze each of the test failures to determine if all of the failures reflect an expected change in behavior.

One failing test is for validateBalance():

public void testValidateBalanceThrowsCreditCardException() throws Throwable

{

CreditCard creditCard = new CreditCard(“2298 9812 4566 1184”, 100.0, 0.0);

try

{

callPrivateMethod(“example1.CreditCard”,

“validateBalance”,

new Class[] {double.class},

creditCard,

new Object[] {new Double(-1.0)}

);

fail(“Expected CreditCardException to be thrown”);

}

catch (CreditCardException ex)

{

assertEquals(“ex.getMessage()”, “Balance can’t go below minimum balance”, ex.getMessage());

assertThrownBy(CreditCard.class, ex);

}

}

This test passes -1.0 to validateBalance() and expects a CreditCardException to be thrown. This test passed when run against the old code, but it fails now because an exception is no longer thrown for negative balances. This is an expected test failure. The failing test reflects an intended change of behavior.

As you might expect, there’s also a failing test for makePayment():

public void testMakePaymentThrowsCreditCardException() throws Throwable

{

CreditCard creditCard = new CreditCard(“2298 9812 4566 1184”, 1000.0, 100.0);

creditCard.makePayment(0.10000000149011612);

creditCard.makePurchase(new Purchase(new Date(0L), 1.4178674221038818));

creditCard.makePurchase(new Purchase(new Date(100L), 100.0));

creditCard.makePurchase(new Purchase(new Date(1000L), 3.9999998989515E-5));

creditCard.makePurchase(new Purchase(new Date(1L), 1.0E-5));

creditCard.makePurchase(new Purchase(new Date(-1L), 3.6909053325653076));

creditCard.makePayment(100.0);

creditCard.makePayment(5.630099296569824);

creditCard.makePayment(1.8264453411102295);

creditCard.makePayment(97.55227811549801);

try

{

creditCard.makePayment(0.0010);

fail(“Expected CreditCardException to be thrown”);

}

catch (CreditCardException ex)

{

assertEquals(“ex.getMessage()”,

“Balance can’t go below minimum balance”,

ex.getMessage());

assertThrownBy(CreditCard.class, ex);

assertEquals(“creditCard.getBalance()”,

0.0,

creditCard.getBalance(),

1.0E-6);

}

}

This is somewhat predictable since makePayment() depends on validateBalance(). This tests performs a series of purchase and payment transactions and checks that a payment creating a negative balance throws an exception. We no longer want such transactions to throw an exception, so this, too, is an expected test failure.

The third test failure, however, is cause for concern. This test of the constructor fails:

public void testConstructorThrowsCreditCardException1() throws Throwable

{

try

{

new CreditCard(“2298 9812 4566 1184”, 15000.0, -1.0);

fail(“Expected CreditCardException to be thrown”);

}

catch (CreditCardException ex)

{

assertEquals(“ex.getMessage()”,

“Balance can’t go below minimum balance”,

ex.getMessage()

);

assertThrownBy(CreditCard.class, ex);

}

}

This test checks that an exception is thrown when creating a CreditCard with a negative opening balance. Our requirements change permits a balance to go negative when making a payment, but not when opening a new account. However, we inadvertently changed the behavior of the constructor because the constructor, just like the makePayment() method, is dependent on validateBalance()!

Real regressions are usually far more obscure

This is a rather simplistic example of where a change in one part our code creates a regression somewhere else. In our CreditCard class, it would be trivial for a developer to grasp the entire behavior of the class and foresee the impact to the behavior of the constructor.

In real world applications, however, large legacy code bases contain complex relationships spanning hundreds of classes. It’s simply not possible for a developer to approach unfamiliar code and understand all of the interdependencies. Characterization tests will highlight behavior changes introduced by the developer, and allow the developer to analyzes those changes for correctness.

So, what shall we do about the regression we introduced in CreditCard? One solution is to create a new method for validating the opening balance. Invoke that method in the constructor instead of validateBalance():

public CreditCard( String accountNumber, double creditLimit,

double balanceTransfer ) throws CreditCardException

{

// … validate and assign account number and credit limit

// validate the balance

//validateBalance( balanceTransfer );

validateOpeningBalance(balanceTransfer);

this.balance = balanceTransfer;

}

private void validateOpeningBalance( double balance )

throws CreditCardException

{

if (balance < 0.00)

{

throw new CreditCardException(“Opening balance can’t be negative”);

}

if (balance > creditLimit)

{

throw new CreditCardException(“Balance can’t exceed credit limit”);

}

}

When we run our characterization tests now, the constructor test passes and the only failing tests are the expected failures.

Regenerate tests and commit changes

Now that we’re convinced we haven’t introduced any behavior changes into the code except for the changes we meant to implement, we can commit the updated CreditCard class to version control.

At the same time, we also should regenerate our CreditCard characterization tests and commit the new tests to version control. The update characterization tests will reflect the new behavior, and they’ll be available the next time a requirements change is needed.

In my next post, we’ll start to look at code coverage issues. Sometimes, JUnit Factory is unable to figure out how to execute all paths of your code. This could be due to the need for objects to exist in a complex state, the need to interact with an external resource such as a database, or simply due to dead code. Look for this post in the next few weeks.

Save/Share Google Yahoo! Add to Technorati Favorites Digg It Reddit
del.icio.us My Zimbio

Leave a Reply