Characterization Tests: How To Deal With Legacy Java Code

Save/Share Google Yahoo! Digg It Reddit del.icio.us
My Zimbio

Companies have invested billions of dollars over the last decade building components and applications based on the Java framework. This work represents a wealth of expertise and collective knowledge that firms must protect and maintain. Unfortunately, in the dynamic field of software development where programmers change jobs, on average, every 18 months, the original developers on these past projects probably aren’t around anymore.

As a result, Java developers seldom have the luxury of working on true greenfield projects. Instead, they are faced with adding enhancements and fixing bugs on projects built upon a code base they didn’t write and don’t fully understand. How can developers safely make changes to legacy code without accidentally breaking something unrelated?

Characterization tests provide a safety net – a change detection engine – that identifies behavioral changes in legacy code in order to remedy regressions early in the development process. Fixing regressions early shortens development timelines, increases code quality, and allows a team to become more agile. You can automatically generate your team’s characterization tests using the free JUnit Factory for Java from Agitar Software.

This post is part of a series:

1. Characterization Tests: How To Deal With Legacy Java Code
2. JUnit Factory Part 1: Generating Tests
3. JUnit Factory Part 2: Finding Regressions
4. JUnit Factory Part 3: Improving Code Coverage

Use JUnit Tests for Regression Testing – If You’re Lucky Enough to Have Any

If you’re lucky, you inherit legacy Java code that already comes with a full JUnit test suite. Running unit tests written by the previous development team can detect unintended behavior changes introduced by new modifications. Running these tests on a regular basis can identify regressions long before they reach QA (where they’re more costly to fix).

The reality of existing unit tests, however, is they generally cover only a small subset of the behavior of the code – core business logic or especially complex algorithms. There is seldom substantial coverage of the entire code base, and there is even less focus on negative test cases (tests that assert a particular response when things go wrong).

What’s even worse is, more often than not, unit tests are nonexistent. Is this because developers are lazy? Maybe. The more likely explanation, however, is that maintaining unit tests is hard! It is both time consuming and expensive.

Unit testing is a combinatorial problem – every boolean decision statement requires at least two tests: one with an outcome of “true” and one with an outcome of “false”. Developers need to write 3 to 5 lines of test code for every line of production code to be tested. It is enough work to write tests in the first place, but refactoring the application code requires rewriting the corresponding unit tests, too.

Developers aren’t punished for bugs – they’re punished for missing deadlines. Consequently, when given a choice between delivering on time or delivering quality software, software quality – along with the unit tests intended to ensure it – gets ignored at crunch time.

Characterization Tests are not Functional Tests

If you’ve ever written a unit test, you probably wrote it to assert particular behaviors in the code you just wrote. If you’re in a Test-Driven Development environment, you wrote the tests first, and then you went on to write just enough code to get your tests to pass.

Characterization tests are different. The term was coined by Michael Feathers in his book “Working Effectively with Legacy Code”. Characterization tests are usually generated using an automated tool like the free JUnit Factory for Java from Agitar Software. Rather than asserting the correctness of a code unit, characterization tests simply capture the behavior of the code, as written, in order to detect behavioral changes later.

Obtaining meaningful code coverage from hand written JUnit tests is difficult, and aiming for any substantial coverage target is generally not cost effective. Remember, you’ll be writing 3 to 5 times more Java code if you’re trying to achieve 100% coverage. Automatically generating characterization tests using JUnit Factory, however, can give you 80% coverage or better with just the click of a button.

How to Use Characterization Tests

The JUnit tests generated by JUnit Factory simply capture how your legacy code behaves today. You’ll want to generate tests for your entire legacy code base before you begin new work. This will create a comprehensive test suite, providing at least 80% coverage, that will detect behavioral changes as you implement new functionality.

With this safety net in place, you can write new code with confidence. But, what does it mean when your tests fail?

A failed characterization test doesn’t always mean a bug. In fact, if you’re intentionally adding new functionality, you should expect some of your tests to fail. After all, altering the behavior of your code was the goal you set out to achieve.

However, be sure you can explain the failure of every test as an intended change. If a unit test fails that you wouldn’t expect to fail, it means you’ve introduced a regression. Continue working on your code until those tests pass.

Distinguishing between expected and unexpected characterization test failures is where the analytical skills of the developer come into play. Work is complete when the only failing tests are the ones expected to fail. At that point, you may safely commit your changes to version control with a high level of confidence you haven’t broken anything.

You’ll also want to use JUnit Factory to generate a new set of tests for the Java classes you modified. This will capture the behavior of your new code, identify future regressions, and eliminate the tests that failed due to deliberate changes. After all, today’s greenfield projects are tomorrow’s legacy code!

How Does JUnit Factory Do It?

JUnit Factory is far more sophisticated than a static analysis tool. In addition to analyzing, and attempting to reach, all of the boundary conditions and branches in your code, JUnit Factory will actually execute your code – instantiating complex parameter objects by reflecting constructors – and record outcome states and exceptions.

What about code that requires data from some external source like a database or a Web service? JUnit Factory is surprisingly effective at auto-mocking external resources using its Mockingbird framework. External dependencies are eliminated to create repeatable unit tests that are independent of their environment.

How To Get Started

Download the free Eclipse plugin for JUnit Factory and try it out on some of the classes in your own projects. Inspect the tests it generates, and you’ll get a feel for what’s going on. Unresolved dependencies get mocked out, private methods get called by reflection, and both positive and negative tests are created.

The Eclipse plugin will also report on the code coverage achieved by the generated tests. Sometimes, certain fragments of code require classes to enter a complex state in order to get executed. If JUnit Factory can’t figure out how to create this precondition, you’ll have some untested sections of code. However, JUnit Factory can, if necessary, be configured with test data helpers that provide hints on how to create this coverage and improve the quality of your tests.

Is this the end of functional testing? No. It’s still a good idea to create functional tests to assert the behavior of your newly written code. After all, that’s the only way to verify intended behavior. Use your hand coded tests as a complement to JUnit Factory change detection tests, and see them all reported together in JUnit Factory’s testing dashboard.

Legacy code is often ugly, but it isn’t going away – companies have invested too many resources to scrap it and start over. Characterization tests make ugly legacy code maintainable, and JUnit Factory produces characterization tests with low cost, high coverage, and a lack of drudgery for which all developers will be grateful.

Go to “JUnit Factory Part 1: Generating Tests” »

Save/Share Google Yahoo! Add to Technorati Favorites Digg It Reddit
del.icio.us My Zimbio

Leave a Reply