Model based testing: models, generation strategies, coverage

What a model is

A model in this sense is a description of expected behavior precise enough for a program to walk. Three forms cover most testing needs. A state machine lists the states a system can be in and the events that move it between them; it suits anything with a lifecycle (sessions, orders, documents, devices). A decision table lists conditions and the action each combination produces; it suits business rules with several inputs (pricing, eligibility, validation). A flow model describes the paths through a process, with branches and joins; it suits user journeys and batch pipelines. Each is a different view of the same idea: write down what should happen once, in a structured form, and let a generator enumerate the cases.

The model is not the code and should not be derived from it. It comes from requirements, from conversations with the product owner, and from the tester's understanding of how the system ought to behave. That independence is the point: when generated tests fail, either the code is wrong or the model is wrong, and both discoveries are valuable. Partitioning and boundary analysis still choose the concrete values for each transition; MBT decides which transitions are exercised and in what order.

Generation strategies and what each covers

Given a model, a generator walks it according to a coverage criterion. Four criteria account for nearly all practical use. The table shows what each guarantees, its cost in tests for the login model above, and the defect classes it tends to find.

Choose by defect history. If escaped defects are mostly of the form 'this action does nothing when I get here', all transitions catches them. If they are 'this works unless I did that first', you need all pairs. A nightly random walk of 10,000 steps on top of either finds the leaks and the states nobody drew.

A worked example: the login model

The model in the figure has four states: Logged out, Entering (credentials being submitted), Locked and Logged in. Six transitions: start (Logged out to Entering), valid credentials (Entering to Logged in), 3 failures (Entering to Locked), wait 15 minutes (Locked to Entering), logout (Logged in to Logged out), and the unlabeled return from Locked to Logged in (an administrator unlock, in this model).

1. All states: 3 tests. One path visits Logged out, Entering and Logged in (start, valid credentials). A second reaches Locked (start, 3 failures). A third is not strictly needed, but a generator adds one to return to Logged out and close the cycle.
2. All transitions: 6 tests. Each of the six edges appears in at least one generated sequence. A minimal set: start then valid credentials then logout (covers 3 edges); start then 3 failures then wait 15 minutes then valid credentials (covers 3 more, reusing start); start then 3 failures then admin unlock then logout (covers the last edge). Three sequences, six transitions, but generators usually emit one test per transition for clearer failure reports, hence 6.
3. All transition pairs: 11 tests. Count consecutive edge pairs that are possible in the model: from start you can go to valid credentials or 3 failures (2 pairs); from valid credentials only logout (1); from 3 failures, wait or unlock (2); from wait, valid credentials or 3 failures (2); from unlock, logout (1); from logout, start (1); and the loop pairs where a second start follows a logout that followed each entry route (2 more distinct pairs). Eleven pairs, eleven tests.
4. Random walk: as many as you like. Start at Logged out, pick an available transition at random, repeat 500 times, check the invariants after every step (a Locked account cannot reach Logged in without wait or unlock; logout always lands on Logged out). Run it nightly with a fresh seed and log the seed.

Concrete data is attached to each transition after generation. Valid credentials gets a known user; 3 failures gets three wrong passwords chosen at the boundary (empty, one character off, correct password for a different user). The generator produces 6 or 11 skeletons; the tester supplies the values once per transition, not once per test.

Where MBT pays and where it does not

• Pays: protocols and session handling, device and embedded state, workflow engines, order and payment lifecycles, anything with more than about 8 states or where sequence defects have escaped. Also pays when the specification changes often, because regenerating from an updated model is minutes and rewriting 60 scripts is days.
• Pays modestly: validation rules with many combinations, where a decision table model plus pairwise generation cuts hundreds of hand-written cases to a few dozen.
• Does not pay: simple forms and pages with 2 or 3 states, content sites, one-off migrations, and anywhere the team cannot describe expected behavior precisely enough to draw. If the product owner cannot agree what the states are, the model becomes the argument, which is useful but is a requirements activity rather than testing.

Introducing MBT in one sprint

Pick one feature with a lifecycle and a defect history: session handling, an approval workflow, a shopping cart. Day 1 and 2, draw the state machine on a whiteboard with the developer and product owner, then transcribe it into whatever notation your generator reads. Expect the drawing to expose two or three undefined behaviors; write them down as questions, not defects. Day 3 and 4, wire each transition to an existing automation step (most teams have the login and logout steps already) and generate all-transitions tests. Day 5 to 8, run them on every pipeline, fix the model where it was wrong and raise defects where the code was. Day 9 and 10, add all-pairs and a nightly random walk of 1,000 steps, then present the transition coverage figure alongside the team's existing coverage numbers. If the sprint finds one sequence defect the old suite missed, the case for a second feature makes itself; if it finds none, that is also an answer and the model still serves as living documentation. Terms used here are defined in the glossary.

This guide is part of the test automation hub. It is best read alongside test automation strategy and flaky tests, which cover the neighbouring questions.