Bredex GmbH Logo
Skip navigation
  • Home
  • Tools
    • Tool overview
      • How we're different
      • Jubula and GUIdancer
      • Selection criteria
    • Feature overview
  • FAQs
  • Videos
    • Presentations
    • Webinars
    • Interviews
  • Examples
    • Module examples
  • Articles
    • Testing in general
    • Naming conventions
    • Test design
    • Feature highlights
    • Continuous Integration
    • Mobile testing
    • Tool reviews
    • German articles
  • Presentations
    • Agile testing
    • Automated testing
    • UI testing and user focus
    • Mobile testing
    • Continuous Integration
    • Training for testing
    • Tools
  • Resources
    • Datasheets
    • Quality stamp
  • Software and services
    • Overview
    • Training and consulting
    • Support
    • Sponsored development
    • Software downloads
    • Migration information
    • License Agreement
  • Blog
  • Contact
    • Contact form
    • Contribute
    • Newsletter
 
Skip navigation
  • Login
  • Register
 
You are here: Home » Articles » Feature highlights » Code coverage

Using the JaCoCo integration

The code coverage feature in GUIdancer and Jubula uses JaCoCo (a Java Code Coverage library available under the Eclipse Public License). Code coverage monitoring can be added to an Application Under Test (AUT) in the AUT Configuration dialog for Swing and RCP AUTs, as described in the user manual.

Viewing the report

In the Reporting Perspective, you can see the code coverage report for each test run:

Code coverage report

The philosophical part…

A recurring issue with code coverage, though, is what to use it for. One of the first things that comes to mind about code coverage was this blog entry. Experience shows that it is a good way of explaining that the numbers gathered by code coverage are not the aim of the exercise. The absolute values delivered by a monitoring report can’t be used as a guide to quality, nor should they be formulated as an aim for the team.

It’s not about the absolute numbers

What code coverage numbers or percentages tell us is only what lines, methods, classes or decisions have been executed. It tells us nothing about whether they were executed correctly, or whether the test that caused these things to be executed was a good one. It can be far too easy to attach too much importance to a simple number removed from its context – and this can drive the wrong kind of behavior in our processes.

So what is it good for?

It’s pleasing to have good, high percentages for code coverage, but it’s not the absolute values of the individual percentages that we should be looking at. Instead, there are two other things we should be looking at.

The trend of the lines

The first thing of interest is to move away from individual numbers and look at the trend, most especially the development over time of amount of tests : code coverage : amount of code. The graph generated by GUIdancer and Jubula shows the first two numbers as lines over time. The report below shows how code coverage can increase with the addition of tests.

BIRT Report containing code coverage

Using graphs such as this, we can see whether a new test added increases the coverage or not. If so, then if we assume no large additions or subtractions from the amount of code, we can reason that the test has executed other lines than executed by previous tests.

If the code coverage value does not increase, it could mean that new code has been added, thus reducing the overall percent of the code coverage. Looking at the detailed report would tell us whether new code was executed or not (more on this in the next section). Even if the amount of code had remained the same, and therefore new lines of code weren't actually executed, that doesn't necessarily mean that the new test isn’t performing an important use case or workflow through the application, just that the lines executed have remained the same.

What is being missed

Another use of code coverage over and above isolated values is the ability to look at what is not being tested. Don’t look at the green; look at the red. The red areas tell us what parts of the software we’re missing. We can look at e.g. class or method names and decide whether we can design tests that should cover these areas.

Detailed code coverage

When a new test is added we can check whether it covers the areas we thought it would. Instead of just looking at a simple number, we should look at the details.

New questions

As it turns out, the questions about effectiveness and risk can’t really be answered by code coverage. What it can answer though, is "where are we not performing any testing?", "where are we performing too little testing?" and "what effects have my last changes (more/less code, more/less tests) had on the trend?". Based on the answers to these questions, we can design new tests. Code coverage can certainly help us to get better quality software, but perhaps not in the way that is often imagined.

Articles

Skip navigation
  • Testing in general
  • Naming conventions
  • Test design
  • Feature highlights
  • Continuous Integration
  • Mobile testing
  • Tool reviews
  • German articles
 

External Links

BREDEX GmbH
Jubula Project Page
Jubula Bugzilla
GUIdancer Bugzilla
Portal Bugzilla
Jubula developers mail list
Jubula Forum

News

Jubula 8.6.0 has been released

2018-03-01 10:41

The Jubula team has released the latest official version of Jubula. 

Version 8.6.0 was released on 1st March 2018. This version contains a variety of useful UI and Jubula API improvements.

Read more … Jubula 8.6.0 has been released

Site notice | Data protection | Sitemap
© 2019 BREDEX GmbH