Test coverage: four kinds and why 80 percent tells you little

Four kinds of coverage

Coverage always means the fraction of some population that the tests touched, and the population is what changes. Requirements coverage counts requirements with at least one linked test. Risk coverage counts items from the risk analysis with a test that would detect them. Code coverage counts structural elements of the code executed during the test run, at one of several granularities. Model coverage counts states, transitions or paths of a behavioral model visited by tests. The table sets them side by side, including what 80 percent actually means in each case.

Code coverage levels, briefly

Statement coverage asks whether each line ran. Branch coverage asks whether each decision went both ways, so an if with no else needs a test where the condition is false. Condition coverage asks whether each boolean sub-condition inside a decision took both values. Modified condition/decision coverage (MC/DC) goes further and requires each sub-condition to be shown to independently change the decision outcome, which for a decision with n conditions takes roughly n plus 1 tests instead of 2 to the power of n. Statement is the weakest and most often quoted; branch is the level a working team should report; MC/DC is mandated for the highest assurance levels in avionics and is rarely worth its cost elsewhere. Model-based testing supplies the model coverage measures.

Why 80 percent line coverage tells you little

Three reasons. First, coverage measures execution, not checking. A test that calls a function and asserts nothing covers every line the function runs. Suites written to hit a coverage target are full of these. Second, statement coverage is blind to the branches that matter most. A function that is 100 percent statement covered may have never taken its error path, because the error path is a single return on a line that also executed for the happy case. Branch coverage of the same function is typically 8 to 15 points lower, and that gap is where the production incidents live. Third, the 20 percent that is uncovered is not a random sample. It is the code nobody could figure out how to test: concurrency, retries, failure handling, the code that runs at 3 a.m. when the network is flapping. Eighty percent coverage with the hard 20 percent missing is worse than 60 percent that includes it.

Coverage as a gap-finder, not a target

The right use of a coverage report is to read the red lines. A coverage tool's output is a map of what the tests never reached, and that map is worth more than the percentage at the top. Open the report for the module with the most production defects, look at which branches are uncovered, and ask whether each one is dead code (delete it), untestable (refactor it) or simply missed (write the test). A team that spends 30 minutes per sprint reading the red lines will improve faster than one that mandates a number. Setting a coverage target, by contrast, produces exactly the behavior described under Goodhart's law in software testing metrics: assertion-free tests, excluded files and a rising number that measures nothing. If you must gate on coverage, gate on the delta (new code must not reduce branch coverage) rather than the absolute.

A worked example

A payments release with 47 requirements, 12 risk items, and a service of 14,200 statements and 3,900 branches. The four numbers from the hero figure came from a real-shaped release, and reading them together is the point. Requirements coverage is 94 percent: 44 of 47 requirements have a linked passing test, and the three without are two reporting requirements and one about audit log retention. Risk coverage is 100 percent: all 12 risk items, including duplicate charge and refund to a closed card, have a test that would fail if the risk materialized. Statement coverage is 83 percent, branch coverage 71 percent, and the 12-point gap sits almost entirely in the retry and timeout handling for the payment provider call. The team's conclusion: ship-readiness on requirements and risk is good, the three uncovered requirements are low risk and scheduled, and the branch gap is the one real finding, because a payment retry that double-charges is exactly the risk item the suite claims to cover. The risk test exercises the happy retry; the uncovered branches are the ones where the retry itself fails. That is a gap in the risk analysis revealed by the code coverage, which is the two measures doing their job together.

How to report coverage to non-testers

Managers hear a percentage and assume it is a grade. Report coverage in a way that resists that reading. State the kind every time (branch coverage, not coverage). Show the trend over releases, not the absolute. Translate the gap into risk in plain words: the retry path for payment provider timeouts has no automated test, which means a regression there would reach production. Pair code coverage with requirements and risk coverage on the same slide so that no single number stands alone. And when asked what the target should be, answer that the target is zero uncovered high-risk paths, not a percentage, and show the list. A short list of named gaps is more persuasive and more honest than a number, and it turns the conversation from how high to what next. Terms are in the glossary.

This guide is part of the software testing techniques hub. It is best read alongside black box vs white box testing and software testing metrics, which cover the neighbouring questions.