Try our Online Forums, or contact us directly.

Code Coverage Analysis is the process of discovering code within a program that is not being exercised by test cases. This information can then be used to improve the test suite, either by adding tests or modifying existing tests to increase coverage.

Code Coverage Analysis shines a light on the quality of your unit testing. It enables developers to quickly and easily improve the quality of their unit tests which ultimately leads to improved quality of the software under development.

A good introduction to the various types of Code Coverage Analysis can be found here.

Code Coverage is not a "silver bullet" of software quality, and 100% coverage is no guarantee of a bug free application. You can infer a certain level of quality in your tests based on their coverage, but you still need to be writing meaningful tests.

As with any metric, developers and project management should be careful not to over-emphasize coverage, because this can drive developers to write unit tests that just increase coverage, at the cost of actually testing the application meaningfully.

Clover was originally developed at Cenqua as an internal tool to support development of large J2EE applications. Existing tools were found to be too cumbersome to integrate with complex build systems and often required specialized development and/or runtime environments that were not compatible with target J2EE Containers. Another feature that we found lacking in other tools was simple, source-level coverage reporting - the kind that is most useful to developers.

Clover is actually a shortened version of the tool's original name, "Cover Lover", from the nick name that the tool's author gained while writing Clover ("Mr Cover Lover").

Clover has no dependence on JUnit. We mention it frequently in our documentation only because of JUnit's widespread use in the Java dev community. You can certainly instrument your code and run it however you like; Clover will still record coverage which can then be used to generate reports.

Clover is fully compatible with JUnit4 and TestNG.

Source code instrumentation is the most powerful, flexible and accurate way to provide code coverage analysis. The following table compares different methods of obtaining code coverage and their relative benefits:

Clover provides integrated plugins for IntelliJ IDEA 4.x and 5.x, NetBeans, and Eclipse, JBuilder and JDeveloper. Clover should also work happily with any IDE that provides integration with the Ant build tool.

There is a Clover Plugin for Maven and Maven2 - both are independent open source developments supported by Cenqua. See the Maven and Maven2 websites for details.

Clover makes use of the following excellent 3rd party libraries:

The "total" coverage percentage of a class (or file, package, project) is provided as a quick guide to how well the class is covered - and to allow ranking of classes. The Total Percentage Coverage (TPC) is calculated using the formula:

 TPC = (CT + CF + SC + MC)/(2*C + S + M)

 where

 CT - conditionals that evaluated to "true" at least once
 CF - conditionals that evaluated to "false" at least once
 SC - statements covered
 MC - methods entered

 C - total number of conditionals
 S - total number of statements
 M - total number of methods

Clover fully supports all JDK1.5 language features.

1. Does my code compile and run as expected without Clover?
   You need to ensure that your project compiles and runs as expected before attempting to use Clover.
2. Am I using the latest version of Clover?
   The latest version of Clover incorporates many bugfixes and improvements.

If the answers in this section don't fix the problem you are encountering, please don't hesitate to contact us.

You need to install Clover in Ant's classpath. Depending on what version of Ant you are using, there are several options to do this. See Installation Options

This probably indicates that you do not have clover.jar in your runtime classpath. See Classpath Issues

This is a limitation of the Java implementation on Unix. Prior to JDK 1.4, the java graphics toolkit (AWT) requires the presence of an XServer, even in the case where no "on-screen" graphics are rendered. With JDK1.4, you can set the System property java.awt.headless=true to avoid this problem. When running Ant, this is most easily achieved by using the ANT_OPTS environment variable:

export ANT_OPTS=-Djava.awt.headless=true

When running your code outside Ant, you may also need to set this system property.

With earlier JDKs, you need to use a virtual X Server. See http://java.sun.com/products/java-media/2D/forDevelopers/java2dfaq.html#xvfb.

This occurs because Clover hasn't had a chance to flush coverage data out to disk. By default Clover flushes coverage data only at JVM shutdown or when explicitly directed to (using a inline directive). The simplest thing to do is to use the fork="true" attribute when running your tests. The tests will be then run in their own JVM, and the coverage data will be flushed when the that JVM exits. Alternatively, you can use interval-based flushing by changing the Flush Policy.

Instrumenting with Clover increases the amount of memory that the compiler requires in order to compile. To solve this problem, you need to give the compiler more memory. Increasing the memory available to the compiler depends on how you are launching the compiler:
If you are using the "in-process" compiler (the <javac> task with the "fork" attribute set to false), you will need to give Ant itself more memory to play with. To do this, use the ANT_OPTS environment variable to set the heap size of the JVM used to run Ant:

export ANT_OPTS=-Xmx256m

If you are using an external compiler (the <javac> task with the "fork" attribute set to true), you can set the memoryInitialSize and memoryMaximumSize attributes of the javac task:

 <javac srcdir="${src}"
       destdir="${build}"
       fork="true"
       memoryInitialSize="128m"
       memoryMaximumSize="256m"/>

Clover will not measure coverage of a conditional expression if it contains an assignment operator. In practice we have found this only a minor limitation. To understand why Clover has this limitation, consider the following (very contrived) code fragment:

 1  public int foo(int i) {
 2    int j;
 3    if ((j = i) == 1) {
 4      return j;
 5    }
 6    return 0;
 7  }

 at (2) the variable "j" is declared but not initialised.
 at (3) "j" is assigned to inside the expression
 at (4) "j" is referenced.

During compilation, most compilers can inspect the logic of the conditional at (3) to determine that "j" will be initialised by the time it is referenced (4), since evaluating the expression (3) will always result in "j" being given a value. So the code will compile. But Clover has to rewrite the conditional at (3) so that it can measure coverage, and the rewritten version makes it harder for compilers to infer the state of "j" when it is referenced at (4). This means that the instrumented version may not compile. For this reason, Clover scans conditionals for assignment. If it one is detected, the conditional is not instrumented.

There are two possible causes.

1. Cascading build files:
   

   Clover uses Ant patternsets to manage the includes and excludes specified in the clover-setup task. By default Ant does not pass these patternsets to the sub-builds. If you are using a master-build/sub-build arrangement, with compilation occuring in the sub-builds and <clover-setup> done in the master-build, you will need to explicitly pass these patternsets as references:

    <ant ...>
       <reference refid="clover.files"/>
       <reference refid="clover.useclass.files"/>
    </ant>

2. Excluded files are still registered in the Clover database:
   Clover's database is built incrementally, and this can mean that files that are now excluded but were previously included are still reported on. The simple workaround is to delete the Clover database whenever you change the clover includes or excludes. This is fixed in Clover 1.2.

The Ant <mail> task depends on external libraries that are not included in the Ant distribution. You need to install the following jars in ANT_HOME/lib, both freely available from Sun:

1. mail.jar - from the JavaMail API (http://java.sun.com/products/javamail/)
2. activation.jar - from the JavaBeans Activation Framework (http://java.sun.com/products/javabeans/jaf/index.jsp)

You should also check the details of your local SMTP server with your SysAdmin. It may help to specify these details directly to the <mail> task:

 <mail mailhost="smtp.myisp.com" mailport="25" from="build@example.com"
     tolist="team@example.com"  subject="coverage criteria not met"
     message="${coverageFailed}" files="coverage_summary.pdf"/>