Risk Based Testing: 5x5 Scoring and Effort Allocation

Why score risk at all

Test effort is finite and defects are not evenly distributed. Some features are complex, new, or built on a component with a bad history; some features would cost a fortune if they failed in production. Scoring makes that judgment explicit so it can be reviewed, rather than leaving it to whichever tester picked up the ticket. It also gives the test plan a rationale that survives a difficult conversation at release time: we tested the refund flow deepest because we all agreed it was a 15.

Scoring likelihood and impact

Likelihood is the chance the feature contains a defect that testing could find. Score it 1 to 5 from complexity, novelty, size of change, defect history of the component, and the experience of the people who built it. Impact is the cost if a defect reaches production. Score it 1 to 5 from financial loss, number of users affected, regulatory or legal exposure, reputational damage, and how visible the failure would be. Multiply the two for a product risk score from 1 to 25.

Score in a one-hour session with a tester, a developer and a product owner present. Disagreement is useful: a developer scoring likelihood at 2 and a tester at 4 usually means one of them knows something the other does not. Record the reason next to the number, not just the number.

Allocating effort by risk class

Depth follows class. The shares are a starting point for a typical release; the worked example below shows how they move with the actual feature mix.
Risk class	Score	Depth of testing	Techniques	Share of effort
High	15 to 25	Every requirement covered by at least two techniques, exploratory sessions, non-functional checks, full regression each build	Boundary value analysis, state transition testing, decision tables, security and performance checks, two or more 90-minute exploratory sessions	About 55 percent
Medium	8 to 12	Every requirement covered once, one exploratory session, regression on change	Equivalence partitioning, use-case tests, one exploratory session	About 30 percent
Low	1 to 6	Smoke check, sampled regression	One positive path test, one negative path test, automated smoke	About 15 percent
Accepted	Any, by decision	Not tested this release, recorded in the plan	None; the acceptance is signed by the product owner	0 percent

Worked example: six features in a release

A release of a subscription billing product contains six changes. The scoring session produces the following: card payment capture, likelihood 4 and impact 5, score 20, high. Refund processing, likelihood 3 and impact 5, score 15, high. Promotion codes with stacking rules, likelihood 4 and impact 3, score 12, medium. Invoice PDF layout, likelihood 3 and impact 3, score 9, medium. Audit log export, likelihood 2 and impact 4, score 8, medium. Address book edits, likelihood 2 and impact 3, score 6, low.

With 200 hours of test effort available, the two high-risk features get about 110 hours between them: full technique coverage, state models for the authorization and refund lifecycles, security checks on stored card tokens, and four exploratory sessions. The three medium features share about 60 hours: partitioned functional tests, one exploratory session each, and a regression run when their code changes. The low-risk address book gets 30 hours at most, most of it automated smoke and a single exploratory pass. The split, roughly 55, 30 and 15 percent, is shown below.

Figure 2Test effort by risk class for the six-feature release (200 hours)

Two features consume more than half the effort. That is the point: effort follows the cost of being wrong.

Re-scoring every release

Risk scores are not permanent. Likelihood falls as a feature stabilizes and rises when it is changed or when a new team takes it over. Impact rises when usage grows or a regulation lands. Re-score at the start of each release in the same one-hour session, starting from last time's numbers and changing only what has changed. A feature that has shipped three releases with no production defects and no code change can drop from 20 to 10; a feature that had a severity 1 escape goes up regardless of what the developers say.

Track the escapes. If production defects cluster in features you scored low, your likelihood scoring is wrong and the session needs different inputs, usually defect history and change size rather than gut feel. The defect leakage metric by risk class is the single best check on whether the scoring works.

Where teams go wrong

Scoring everything 4 or 5 so nothing is low. If more than a third of features land in the high class, the scale has lost its meaning. Force-rank if needed: at most 30 percent high, at least 20 percent low.

Fitting it into the strategy

The test strategy fixes the scale, the thresholds and the class-to-depth rule once. Each release then only has to score its features and apply the rule, which takes an hour rather than a debate. The scoring table goes into the plan, the effort split goes into the estimate, and the re-scoring notes go into the completion report so the next release starts from evidence.

Common questions

What is the difference between product risk and project risk?

Product risk is the risk that the software fails in use; it drives what to test and how deeply. Project risk is the risk that the testing itself is derailed, by late builds or missing environments; it drives contingency in the plan. Score product risk; list and mitigate project risk.

Is a 3 by 3 matrix enough?

It works for small teams but collapses too many features into the same cell. A 5 by 5 gives 25 scores and enough spread to rank; anything larger produces false precision.

Who should score the risks?

A tester, a developer and a product owner together. The tester knows defect history, the developer knows complexity and change size, the product owner knows business impact. Any one of them alone gets half the picture.

How does risk based testing work in agile sprints?

Score at backlog refinement as part of story preparation. High-risk stories get more test tasks and an exploratory session in the sprint; low-risk stories get automated checks and a review. Re-score at each refinement.

Can you automate risk scoring?

Partially. Change size, defect history and code complexity can be pulled from version control and the defect tracker to propose a likelihood. Impact still needs a human who understands the business.

What do you do with a feature the product owner accepts as untested?

Record it in the features-not-tested section of the plan with the owner's name and the date. That is a legitimate risk decision, as long as it is written down and visible at release.

Risk based testing: scoring, effort allocation, worked example