Business
Manage and integrate JUnit tests
Although testing individual classes is helpful, unit tests work best when they're written to cover multiple aspects of a program. However, developing comprehensive unit tests means creating many test cases over many files, which can get unwieldy before long.
In a
recent article, I covered the creation of JUnit tests to perform unit testing on
Java classes. Although testing individual classes is helpful, unit tests work
best when they're written to cover multiple aspects of a program. However,
developing comprehensive unit tests means creating many test cases over many
files, which can get unwieldy before long. This article will explain how to
manage those test cases and how to integrate them into your development
environment.
Creating test suites: Manual vs. automatic
In the previous article, I discussed the creation of test suites that grouped individual tests or every test of a class. In both cases, we created an application that called the JUnit tests. However, as you start to create new tests, you must decide if it is best to manually determine which tests to run or to use an automated tool to build the test suite. Manually maintaining the application test suite is certainly acceptable, although it requires recompiling the application to add new tests. The manual approach is necessary when you're selecting specific tests. Listing A shows a simple test runner that builds the test suite, first adding individual tests and then adding the entire test suite.
The other option is to build a dynamic list of tests. In this example, I created a test runner that will run all the tests in a directory, sending the results to the screen. Our dynamically created test suite assumes that all tests can be run and will be run in alphabetical order as read from the directory. Giving up the ability to select which tests are executed may have implications for your testing needs, in which case you’ll need to manually add tests to your suite.
In Listing B, I’ve created an application that reads a test directory from the command line, iterates through the directory looking for “.class” files, and adds them to a test suite.
Our test runner uses the Java class loader to import the classes, so it is important that the classes it tries to load are in the class path. The output of this test runner will be the test results, just as in the previous examples. Listing C shows the output run against a directory of tests.
JUnit executes the test in one large test unit. The output from Listing C represents tests run against four test classes, each with one test each. This is helpful since the output represents the status of all the tests in the directory.
Integrating with Ant
You can streamline your testing efforts even more by integrating unit tests into your build systems. Ant, the Java-based tool available from The Jakarta Project, makes it easy. Integrating Ant with JUnit will help you quickly test your changes across the system. Fortunately, the Jakarta Project has created Ant tasks to integrate JUnit with Ant. These tasks help automatically run tests and generate output and reports. To get started, you’ll need to install Ant and the optional Ant tasks, both of which are available at the Jakarta Web site. Once Ant is installed, you’ll need to install the optional.jar file in the $ANT_HOME/lib/ directory. This file includes the JUnit integration tasks. To use the tasks, you also need to install the Junit.jar package that shipped with JUnit. Install it into the $ANT_HOME/lib/ directory as well.
In Listing D, we’ve created a simple Ant build file that compiles the test cases from the src directory into the build directory. (This is the build file we used in our XMLTest example in the previous article.)
Once the test cases are built, it's time to run them. Ant supplies a task named junit to run the tests. This task replaces the need to create test suites and test runners, greatly simplifying the process. What was contained in the test runner and test suites is now configured from within the Ant build file. Listing E shows how we can run a single test.
The JUnit task can be configured to stop on an error or perform all tests and print a summary of errors at the end. Running the tests requires setting the class path, so we use the classpath directive to add directories and the jar files we need. Our tests are contained in a package named xmltest, so we need to explicitly declare the package name in the test. Output can be in either text or XML format. The advantage of receiving output in XML is that it allows you to generate aggregate reports in the end.
The Ant JUnit task provides two ways to execute tests. You can manually define each test, as in Listing E, or perform a batch test. The batchtest directive, shown in Listing F, can be used in place of individual tests or mixed and matched.
In Listing F, batchtest has been used to run all tests in the build directory and subdirectories. Since we do not want to test any files that are not Java code, we tell the fileset to explicitly add files ending in Test.class. Using the fileset, we could also explicitly exclude files. When we run the Ant task, the output will look like Listing G.
The output from the examples is contained in the directory test_output. Each test class is output into a separate file, broken down by test. The output from the SimpleTest appears in Listing H.
Generating HTML reports
One additional benefit of using Ant tasks is that they have built-in tasks for generating reports. The junitreport task uses the XSLT package Xalan, from The Apache XML Project. You will need to download the Xalan-j package and install the xalan.jar and bsf.jar files in the $ANT_HOME/lib directory, as well as a copy of the Xerces XML parser.
Configuring the reports is simple. As Listing I shows, you must tell junitreport which XML files to include in the test report and which directory to output the results to. The reporting software will do the rest, breaking down each result by time and success. The resulting report is a framed HTML page like the one shown in Figure A.Figure A
Sample JUnit report
Creating test suites: Manual vs. automatic
In the previous article, I discussed the creation of test suites that grouped individual tests or every test of a class. In both cases, we created an application that called the JUnit tests. However, as you start to create new tests, you must decide if it is best to manually determine which tests to run or to use an automated tool to build the test suite. Manually maintaining the application test suite is certainly acceptable, although it requires recompiling the application to add new tests. The manual approach is necessary when you're selecting specific tests. Listing A shows a simple test runner that builds the test suite, first adding individual tests and then adding the entire test suite.
The other option is to build a dynamic list of tests. In this example, I created a test runner that will run all the tests in a directory, sending the results to the screen. Our dynamically created test suite assumes that all tests can be run and will be run in alphabetical order as read from the directory. Giving up the ability to select which tests are executed may have implications for your testing needs, in which case you’ll need to manually add tests to your suite.
In Listing B, I’ve created an application that reads a test directory from the command line, iterates through the directory looking for “.class” files, and adds them to a test suite.
Our test runner uses the Java class loader to import the classes, so it is important that the classes it tries to load are in the class path. The output of this test runner will be the test results, just as in the previous examples. Listing C shows the output run against a directory of tests.
JUnit executes the test in one large test unit. The output from Listing C represents tests run against four test classes, each with one test each. This is helpful since the output represents the status of all the tests in the directory.
Integrating with Ant
You can streamline your testing efforts even more by integrating unit tests into your build systems. Ant, the Java-based tool available from The Jakarta Project, makes it easy. Integrating Ant with JUnit will help you quickly test your changes across the system. Fortunately, the Jakarta Project has created Ant tasks to integrate JUnit with Ant. These tasks help automatically run tests and generate output and reports. To get started, you’ll need to install Ant and the optional Ant tasks, both of which are available at the Jakarta Web site. Once Ant is installed, you’ll need to install the optional.jar file in the $ANT_HOME/lib/ directory. This file includes the JUnit integration tasks. To use the tasks, you also need to install the Junit.jar package that shipped with JUnit. Install it into the $ANT_HOME/lib/ directory as well.
In Listing D, we’ve created a simple Ant build file that compiles the test cases from the src directory into the build directory. (This is the build file we used in our XMLTest example in the previous article.)
Once the test cases are built, it's time to run them. Ant supplies a task named junit to run the tests. This task replaces the need to create test suites and test runners, greatly simplifying the process. What was contained in the test runner and test suites is now configured from within the Ant build file. Listing E shows how we can run a single test.
The JUnit task can be configured to stop on an error or perform all tests and print a summary of errors at the end. Running the tests requires setting the class path, so we use the classpath directive to add directories and the jar files we need. Our tests are contained in a package named xmltest, so we need to explicitly declare the package name in the test. Output can be in either text or XML format. The advantage of receiving output in XML is that it allows you to generate aggregate reports in the end.
The Ant JUnit task provides two ways to execute tests. You can manually define each test, as in Listing E, or perform a batch test. The batchtest directive, shown in Listing F, can be used in place of individual tests or mixed and matched.
In Listing F, batchtest has been used to run all tests in the build directory and subdirectories. Since we do not want to test any files that are not Java code, we tell the fileset to explicitly add files ending in Test.class. Using the fileset, we could also explicitly exclude files. When we run the Ant task, the output will look like Listing G.
The output from the examples is contained in the directory test_output. Each test class is output into a separate file, broken down by test. The output from the SimpleTest appears in Listing H.
Generating HTML reports
One additional benefit of using Ant tasks is that they have built-in tasks for generating reports. The junitreport task uses the XSLT package Xalan, from The Apache XML Project. You will need to download the Xalan-j package and install the xalan.jar and bsf.jar files in the $ANT_HOME/lib directory, as well as a copy of the Xerces XML parser.
Configuring the reports is simple. As Listing I shows, you must tell junitreport which XML files to include in the test report and which directory to output the results to. The reporting software will do the rest, breaking down each result by time and success. The resulting report is a framed HTML page like the one shown in Figure A.
Sample JUnit report
Easier testing, fewer errors
Whether you create
them manually or automatically, test suites make it easier to manage unit tests
that cover multiple aspects of a program. Integrating unit testing into build
systems can make your testing efforts even more efficient and help increase the
quality of the software engineering process. By integrating the Ant build system
with the JUnit unit tests, you can easily create a simple system to test your
code.