JUnit Factory Part 3: Improving Code Coverage

Save/Share Google Yahoo! Digg It Reddit del.icio.us
My Zimbio

JUnit Factory is rather clever how it analyzes and executes your code to generate characterization tests. However, legacy Java code was generally not written with testability in mind. This sometimes makes it difficult for JUnit Factory to attain complete coverage of your code due to the need for objects to exist in a complex state or the need to interact with an external resource such as a database.

JUnit Factory is often able to generate mock instances automatically for problematic classes. When automocking fails, the developer can improve coverage by either extracting behaviors into private methods or by providing hints to JUnit Factory in the form of test data helpers.

This post is part of a series:

1. Characterization Tests: How To Deal With Legacy Java Code
2. JUnit Factory Part 1: Generating Tests
3. JUnit Factory Part 2: Finding Regressions
4. JUnit Factory Part 3: Improving Code Coverage

Handling External Resources

A classic unit testing problem is testing business logic that depends on an external resource. The external resource is usually a database, but it could also be a Web service, the file system, or a user interface. This is challenging because a true unit test executes in the absence of external dependencies, but creating mocks is an expensive, laborious process for the developer.

JUnit Factory’s answer to this is the Mockingbird framework. When code to be tested must retrieve data from a database, JUnit Factory will repeatedly see some failure (like a SQLException) being thrown every time it tries to execute a database call. To move beyond the database call, JUnit Factory will create a mock implementation of the failing method.

Let’s look at a modified version of the makePurchase() method we discussed earlier:

public void makePurchase(Purchase purchase) throws CreditCardException

{

validateBalance(balance + purchase.getAmount());

CreditCardDAO dao = CreditCardDAO.getDAOImplementation();

dao.recordPurchase(accountNumber, purchase);

balance = balance + purchase.getAmount();

}

We’ve introduced a data access object called CreditCardDAO encapsulating our interface to an external system. In this case, it’s probably a database. There’s likely one or more subtypes of CreditCardDAO, such as CreditCardOracleDAO or CreditCardMySqlDAO, providing implementation-specific behaviors for the data storage system.

Notice there are no subtypes of CreditCardDAO in the example Eclipse project archive. They wouldn’t be usable, anyway, since a properly written unit test should execute in the absence of a database. A developer would need to create a mock instance of CreditCardDAO in order to write the test.

If we submit this class to JUnit Factory to autogenerate a characterization test, we can see how the call to CreditCardDAO.recordPurchase() gets mocked out using the Mockingbird framework:

public void testMakePurchaseWithAggressiveMocks() throws Throwable

{

CreditCard creditCard =

(CreditCard) Mockingbird.getProxyObject(CreditCard.class, true);

Purchase purchase =

(Purchase) Mockingbird.getProxyObject(Purchase.class);

CreditCardDAO creditCardDAO = new CreditCardDAO();

setPrivateField(creditCard, “creditLimit”, new Double(0.0));

setPrivateField(creditCard, “accountNumber”, “”);

setPrivateField(creditCard, “balance”, new Double(0.0));

setPrivateField(purchase, “amount”, new Double(0.0));

CreditCardDAO.setDAOImplementation(creditCardDAO);

Mockingbird.enterRecordingMode();

Mockingbird.setReturnValue(false,

creditCardDAO,

“recordPurchase”,

“(java.lang.String,example2.Purchase)void”,

null,

1

);

Mockingbird.enterTestMode(CreditCard.class);

creditCard.makePurchase(purchase);

assertEquals(“creditCard.getBalance()”,

0.0,

creditCard.getBalance(),

1.0E-6

);

}

First, notice the suffix “WithAggressiveMocks” added to the end of the test method name. This indicates JUnit Factory needed to create mocks for a few classes in order to create a test. Aggressive mocks aren’t a problem, although they can make tests harder to read. Later, we’ll discuss test data helpers which can sometimes be used to avoid aggressive mocking and create clearer test code.

The test method begins with Mockingbird creating a number of proxy instances to create a hook into the call to be mocked. The actual insertion of the mock behavior occurs on the line that begins “Mockingbird.setReturnValue“.

Instead of instantiating a subtype of CreditCardDAO, JUnit Factory simply uses the base type. However, the base implementation of recordPurchase() always throws a RuntimeException indicating it must be overridden by a subclass. The “setReturnValue()” call adjusts the behavior of recordPurchase() to return normally instead. This allows the rest of the makePurchase() method to execute and update the balance.

These are unit tests – not integration tests

You’ll see Mockingbird used in a number of scenarios, not just where calls to external resources would otherwise fail. Mockingbird is applied anywhere JUnit Factory needs to control the return value of dependent types in order to continue executing a method.

This may seem a little disconcerting at first. After all, how good are these tests if we’ve mocked the behavior of dependent classes? This is okay. Remember these are characterization tests. This isn’t about asserting the correct behavior of our code. We’re only trying to capture the behavior of legacy code so we can find regressions.

Method extraction and test data helpers

Next, let’s take a look at a method called fraudCheck(). This implements an admittedly simplistic rule that flags account activity as suspicious if there are several small purchases (less than $10) that add up to over $100.

public boolean fraudCheck()

{

CreditCardDAO dao = CreditCardDAO.getDAOImplementation();

List purchases = dao.getRecentPurchases(accountNumber);

boolean isSuspicious;

double purchaseTotal = 0;

for (Iterator iter = purchases.iterator(); iter.hasNext();)

{

Purchase purchase = (Purchase) iter.next();

if (purchase.getAmount() < 10.00)

{

purchaseTotal += purchase.getAmount();

}

}

if (purchaseTotal > 100.00)

{

isSuspicious = true;

}

else

{

isSuspicious = false;

}

return isSuspicious;

}

When we execute CreditCardAgitaTest, we can look at the coverage information provided by the JUnit Factory plug-in on the left side of the editor window:

Missing Coverage

Here, we see line 138 is never executed when we run the test. For a series of purchases to be flagged as suspicious, we’d need at least 11 transactions of $9.99. Although JUnit Factory can mock the return value of getRecentPurchases(), it was unable to discover the proper state of a purchase list that would result in the execution of this line. So, we’re not able to reach all branches of our code.

Controlling preconditions

We know how to create a list of purchases that satisfies the criteria for suspicious activity, but we need some way of inserting that list into a test.

A unit test involves creating a set of preconditions to a method (the parameters) and asserting a set of postconditions (the state manipulated by the method). Before we can control the list of purchases as a test precondition, we need to refactor our code to turn the list of purchases into a method parameter.

The safest way to do this is to use Eclipse’s refactoring tools. Select the code in fraudCheck() from the first line after the call to getRecentPurchases() and on to the end of the method. Right-click the highlighted code in the editor and select “Refactor -> Extract Method”. We’ll name the new method “hasSuspiciousActivity()“.

Extract Method

This is completely safe because extracting this code into its own method doesn’t actually change the behavior of the fraudCheck() method. All we’ve done is make the fraudCheck() method testable. The new code looks like this:

public boolean fraudCheck()

{

CreditCardDAO dao = CreditCardDAO.getDAOImplementation();

List purchases = dao.getRecentPurchases(accountNumber);

return hasSuspiciousActivity(purchases);

}

private boolean hasSuspiciousActivity(List purchases)

{

boolean isSuspicious;

double purchaseTotal = 0;

for (Iterator iter = purchases.iterator(); iter.hasNext();)

{

Purchase purchase = (Purchase) iter.next();

if (purchase.getAmount() < 10.00)

{

purchaseTotal += purchase.getAmount();

}

}

if (purchaseTotal > 100.00)

{

isSuspicious = true;

}

else

{

isSuspicious = false;

}

return isSuspicious;

}

What we’ve achieved is the creation of a method that takes a list of purchases as a parameter. Now, we can give JUnit Factory a hint about how to create this parameter in such a way that isSuspicious will sometimes be set to “true”.

Giving hints using test data helpers

A test data helper is simply a class that creates objects JUnit Factory can pass as parameters to methods being tested. Test data helper methods must 1) have a name that begins “create”, 2) take no parameters, and 3) return an instance of the target data type. You may have as many test data helper methods on a test helper class as you want.

In the sample Eclipse project archive, look for a source folder named “testhelpers”. In there you’ll find a class named PurchaseListTestHelper. This class implements com.agitar.lib.TestHelper, so JUnit Factory will recognize it as a global test helper (as opposed to the ScopedTestHelper interface which ties the helper methods to a particular type). The class contains a single test data helper method named createSuspiciousPurchaseList(). Here is the complete class definition:

package example2;

import java.util.*;

import com.agitar.lib.TestHelper;

public class PurchaseListTestHelper implements TestHelper

{

public static List createSuspiciousPurchaseList()

{

ArrayList purchaseList = new ArrayList();

try

{

for (int i = 0; i < 11; i++)

{

Date purchaseDate = new Date(1000);

double amount = 9.99;

Purchase purchase = new Purchase( purchaseDate, amount );

purchaseList.add(purchase);

}

}

catch( CreditCardException exc )

{

throw new RuntimeException(“Failed to create purchase list”);

}

return purchaseList;

}

}

This helper create a list of 11 purchases of $9.99 each. If this object is passed to our hasSuspiciousActivity() method, we can expect it to execute the line setting isSuspicious to true.

Can we be sure JUnit Factory will use our test helper? Not necessarily. JUnit Factory prefers to use developer provided test data helpers whenever possible – as long as they provide unique code coverage or an interesting outcome. If you have multiple test helpers, but they all provide the same test coverage, JUnit Factory will only use one of them.

If no test helper provides a desired execution flow, JUnit Factory will create an instance of an object on its own by reflecting constructors off of the Class object. It is only when this final strategy fails that JUnit Factory resorts to aggressive mocks and the Mockingbird framework.

Will JUnit Factory use the test data helper we just created? We can generate tests for CreditCard again and find out. JUnit Factory will search our entire Eclipse project for all available test data helpers. There’s nothing to configure, because all classes implementing TestHelper or ScopedTestHelper will be used as candidates.

After regenerating tests for CreditCard, we can see the answer is “yes,” JUnit Factory found our test data helper and used it. Some of the test cases for hasSuspiciousActivity() use PurchaseListTestHelper as shown in the generated test:

public void testHasSuspiciousActivity() throws Throwable

{

CreditCard creditCard = new CreditCard(“3317 3013 6259 0300”, 100.0, 0.0);

List suspiciousPurchaseList =

PurchaseListTestHelper.createSuspiciousPurchaseList();

boolean add =

suspiciousPurchaseList.add(new Purchase(new Date(100L), 100.0));

boolean result = ((Boolean) callPrivateMethod( “example2.CreditCard”,

“hasSuspiciousActivity”,

new Class[] {List.class},

creditCard,

new Object[] {suspiciousPurchaseList})

).booleanValue();

assertTrue(“result”, result);

}

When we execute CreditCardAgitarTest, the code coverage bars in the editor window show JUnit Factory is now covering all of our business logic in CreditCard. It has also created additional assertions in CreditCardAgitarTest to check the outcome when isSuspicious returns true.

Test coverage strategy

With so many tools at your your disposal, where do you begin?

First, don’t concern yourself with aggressive mocks. Aggressive mocks happen automatically, and they’re okay. They’re merely a sign your legacy code wasn’t written for testability. Aggressive mocks give you the code coverage you need to detect changes in the behavior of your code. If you have 100% code coverage already, the only reason you might look into one of the other two strategies is if you want to improve the readability of your tests.

If aggressive mocks don’t give you enough coverage, consider refactoring your code using the Eclipse tool to extract a method. This allows you to create method parameters out of local variables that JUnit Factory couldn’t get into an appropriate state. Often times, you won’t even need to create a test data helper after refactoring your code. If the required state for a parameter is straightforward enough, JUnit Factory can figure out on its own how to create it properly.

Reach for test data helpers last. They’re a powerful tool. But, they require writing additional code – and that’s just more code you’ll need to maintain. Use them only when refactoring doesn’t give you the coverage you need, or when team development policies preclude modifying existing code.

Lastly, always consider whether the missing coverage is worth solving. You might be dealing with truly dead code that can never be reached. You might also be dealing with one or two minor lines that simply aren’t worth the effort. Aiming for 100% code coverage is a costly and, generally, foolish objective. Consider the cost against the benefit.

After all, if you’ve brought your test coverage from 0% up to 80% with the single click of a button, aren’t you profoundly better off than before?

Save/Share Google Yahoo! Add to Technorati Favorites Digg It Reddit
del.icio.us My Zimbio

Leave a Reply