Software Testing Glossary

More than a hundred terms a working tester meets, including the vocabulary of AI detection and content authenticity, each defined in one or two sentences. Where a guide on this site explains the idea in depth, the term links to it. Definitions follow ISTQB usage where the term is defined there.

A

Acceptance criteria

The conditions a specific story or feature must satisfy to be accepted by the product owner or customer. Per-item, unlike the definition of done. Guide: Definition of done

Acceptance test driven development (ATDD)

A collaborative practice in which the team writes concrete acceptance examples before development and turns them into automated acceptance tests. Guide: Acceptance test driven development

Acceptance testing

Testing against business needs and user requirements, usually by or for the customer, to decide whether a system is fit to release. Guide: Software test process

Agile inspection

A sampling-based review of a specification in which checkers count major defects on a few pages against clear rules and extrapolate the defect density. Guide: Agile inspection

Agile testing

Testing practised inside an agile team: continuous, collaborative, example-driven, with no separate test phase. Hub: Agile Testing

All-pairs testing

A combinatorial technique that covers every pair of parameter values at least once, cutting case counts sharply against full combination. Guide: Equivalence partitioning and boundary values

Automation pyramid

A model recommending many fast unit tests, fewer service or API tests and few end-to-end UI tests. Guide: Test automation pyramid

B

Base rate

The proportion of a population that actually belongs to the positive class, for example the share of submitted essays that are machine-written. It sets what a positive verdict is worth: at a low base rate, most flags from a detector with a modest false positive rate are wrong. Guide: False positives, base rates and the confusion matrix

Black-box testing

Testing based on the specified behaviour of a component or system without knowledge of its internal structure. Guide: Black box vs white box testing

Boundary value analysis

A design technique that tests at the edges of equivalence partitions, where defects concentrate. Guide: Equivalence partitioning and boundary values

Branch coverage

The percentage of decision outcomes in the code exercised by a test suite. Guide: Test coverage

Bug bash

A time-boxed session in which many people, often from outside the test team, test a product at once to find defects quickly. Guide: Exploratory testing

C

C2PA manifest

The data structure the C2PA specification defines to carry provenance: a set of assertions (ingredients, actions, creator), a claim that hashes them, and a signature over the claim. Manifests nest, so an edited file carries its parent manifest as an ingredient. Guide: Testing a content provenance pipeline

Charter

A one- to three-line mission for an exploratory testing session: what to explore, with what resources, to find what kind of information. Guide: Exploratory testing

Code coverage

A measure of how much of the source code a test suite executes, expressed as statement, branch, condition or path coverage. Guide: Test coverage

Condition coverage

The percentage of boolean sub-conditions evaluated to both true and false by a test suite. Guide: Test coverage

Content Credentials

The public name for provenance attached to a file under the C2PA specification: a signed manifest recording who made the content, with what tool, and which edits followed. Valid credentials show a signer made a claim and the file is unchanged since, and nothing about whether the claim is true. Guide: How to verify content credentials

Continuous testing

Running automated tests at every stage of a delivery pipeline so that feedback arrives within minutes of a change. Guide: Continuous testing

Contract testing

Verifying that a service and its consumers agree on the shape and behaviour of their interface, without running both together. Guide: Open source testing tools

Cost of quality

The total cost of preventing, appraising and failing on quality: prevention plus appraisal plus internal and external failure costs. Guide: ROI of testing

D

Decision table

A tabular technique listing combinations of conditions and the actions each combination should produce. Guide: Equivalence partitioning and boundary values

Defect

A flaw in a component or system that can cause it to fail to perform its required function. Also called a bug or fault.

Defect density

The number of defects per unit of size, typically per thousand lines of code, per function point or per requirement page. Guide: Software testing metrics

Defect leakage

The proportion of defects found after a phase or release that should have been found before it. Guide: Software testing metrics

Defect removal efficiency

Defects found before release divided by all defects found before and after release, as a percentage. Guide: Software testing metrics

Definition of done (DoD)

The team-wide checklist an increment must satisfy to be considered complete, usually layered by story, sprint and release. Guide: Definition of done

Disclosure label

A visible or machine-readable statement attached to content that it was generated or altered by AI, whether declared by the uploader, derived from signed provenance, or inferred by a detector. Tested as a feature: it must display on every surface and survive share, embed and re-upload. Guide: Testing AI disclosure labels

Dynamic analysis

Examining a running program for defects such as memory leaks, race conditions or security weaknesses. Guide: Security testing

E

Entry criteria

The conditions that must hold before a test activity may start, such as a stable build and an available environment. Guide: How to write a test plan

Equivalence partitioning

Dividing inputs into classes expected to behave the same so that one value tests the whole class. Guide: Equivalence partitioning and boundary values

Error guessing

Designing tests from experience of where defects typically hide. Hub: Testing Techniques

Escaped defect

A defect found in production that testing did not catch. Guide: Software testing metrics

Evidence-grade record

A test record complete enough to be relied on in a dispute or an audit: who ran the test, inputs identified by hash, the exact version of the system under test, criteria fixed before the run, raw outputs, timestamps and environment, stored write-once and reproducible by a second tester. Guide: Evidence-grade test records for authenticity systems

Exit criteria

The conditions that must hold for a test activity to be considered complete, such as coverage reached and no open critical defects. Guide: Software test process

Exploratory testing

Simultaneous learning, test design and execution, structured by charters and time-boxed sessions. Guide: Exploratory testing

F

False positive rate

False positives divided by all true negatives: the share of genuinely human-written documents a detector flags as machine-written. Measured on a labelled human slice with a confidence interval, and the rate whose ceiling should be fixed before a threshold is chosen. Guide: ROC curves and thresholds for AI text detectors

Flaky test

An automated test that passes and fails on the same code without any change, because of timing, state, environment or order. Guide: Flaky tests

Functional testing

Testing what the system does against its specified functions.

G

Grey-box testing

Testing with partial knowledge of internal structure, typically using design documents or logs to shape black-box tests. Guide: Black box vs white box testing

H

Hard binding

A content binding that ties a C2PA manifest to the exact bytes of an asset by cryptographic hash. Any change to the covered bytes, including a re-encode, breaks the hash and the manifest fails validation, which is the signal a pipeline test looks for. Guide: Testing a content provenance pipeline

I

Integration testing

Testing the interfaces and interactions between integrated components or systems. Guide: Test strategy

ISO/IEC/IEEE 29119

The international series of standards for software testing: concepts, processes, documentation, techniques and keyword-driven testing. Guide: Software test process

ISTQB

The International Software Testing Qualifications Board, whose syllabi and glossary define widely shared testing terminology. Archive: Recommended reading for testers

L

Load testing

Performance testing at the expected production load to check response times and throughput. Guide: Performance testing

M

MC/DC

Modified condition/decision coverage: every condition is shown to independently affect the decision outcome. Required in safety-critical standards. Guide: Test coverage

Metadata (EXIF, XMP, IPTC)

The descriptive layers embedded in a media file: EXIF written by the camera (device, exposure, time), XMP written by editing software (edit history, rights), and IPTC fields for captions and credits. Each layer is stripped, rewritten or preserved differently by each upload channel. Guide: Metadata survival testing

Metric

A measurement of a testing attribute, useful only when paired with the question it answers and the behaviour it might distort. Guide: Software testing metrics

Mobile app testing

Testing applications on phones and tablets across devices, platform versions, networks, interruptions and store rules. Guide: Mobile app testing

Model-based testing (MBT)

Generating test cases from a model of the system's behaviour, such as a state machine. Guide: Model based testing

Mutation testing

Introducing small faults into code to measure whether the test suite detects them. Guide: Test coverage

N

Non-functional testing

Testing how well the system works: performance, security, usability, reliability, maintainability, portability. Hub: Testing Techniques

O

OSSTMM

The Open Source Security Testing Methodology Manual, a methodology for measuring operational security by testing. Guide: Security testing

OWASP Top Ten

A regularly updated list of the most critical web application security risks, used as a shared vocabulary. Guide: Security testing

P

Pairwise testing

See all-pairs testing. Guide: Equivalence partitioning and boundary values

Penetration testing

Simulated attack on a system by skilled testers to find exploitable weaknesses. Guide: Security testing

Percentile (p95, p99)

The response time below which 95 or 99 percent of requests complete. More informative than the mean for performance targets. Guide: Performance testing

Performance testing

Testing to determine how a system behaves under load, including response time, throughput and resource use. Guide: Performance testing

Product risk

A risk that the product will fail to satisfy stakeholders, scored by likelihood and impact to steer test depth. Guide: Risk based testing

Provenance

The recorded origin and history of a piece of content: who created it, with what device or software, and which edits followed. Provenance systems record claims and sign them; a tester checks that the record survives every pipeline stage and validates at the end. Guide: Testing a content provenance pipeline

Q

Quarantine

Removing a flaky test from the gating suite for a fixed period while it is fixed, without deleting it. Guide: Flaky tests

R

Regression testing

Re-testing after a change to confirm that previously working behaviour still works. Guide: Test automation strategy

Requirements coverage

The percentage of requirements traced to at least one executed test. Guide: Test coverage

Risk-based testing

Prioritizing and sizing test effort by the assessed product risk of each area. Guide: Risk based testing

S

Security testing

Testing to find weaknesses that could allow unauthorized access, data loss or misuse. Guide: Security testing

Session-based test management (SBTM)

A way of structuring exploratory testing into charters, time-boxed sessions, notes and debriefs so that it can be measured and reported. Guide: Exploratory testing

Shift left

Moving testing activities earlier in the lifecycle, into requirements, design and development. Guide: Continuous testing

Smoke test

A shallow, fast set of tests that confirms a build is stable enough for further testing. Guide: Continuous testing

Soak test

A performance test at normal load over a long period to find leaks and degradation. Guide: Performance testing

Soft binding

A binding that links a manifest to content by a perceptual fingerprint or an invisible watermark rather than an exact hash, so the manifest can be found again after metadata is stripped. It tolerates re-encoding and resizing and gives weaker evidence than a hard binding. Guide: Metadata survival testing

Specification by example

Using concrete examples, agreed by the whole team, as the specification and the acceptance tests. Guide: Acceptance test driven development

Spike test

A performance test with sudden large increases in load to see how the system copes and recovers. Guide: Performance testing

State transition testing

Designing tests from the states of a system and the events that move it between them. Guide: Model based testing

Statement coverage

The percentage of executable statements exercised by a test suite. Guide: Test coverage

Static analysis

Examining code or documents without executing them, by tool or by review. Guide: Security testing

Stress testing

Performance testing beyond expected load to find the breaking point and the failure mode. Guide: Performance testing

System testing

Testing the integrated system as a whole against its requirements. Guide: Test strategy

T

Test automation strategy

The decisions about what to automate, in which layer, owned by whom, in which environments, and how results are reported. Guide: Test automation strategy

Test basis

The documents and knowledge from which test conditions are derived: requirements, designs, risks, user stories. Guide: Software test process

Test case

A set of preconditions, inputs, actions and expected results developed for a specific objective.

Test condition

An aspect of the test basis that could be verified by one or more tests. Guide: Software test process

Test coverage

The degree to which a specified coverage item (requirement, risk, code element, model element) has been exercised by tests. Guide: Test coverage

Test data

The data needed to execute tests: inputs, existing state and expected results. Guide: Test automation strategy

Test environment

The hardware, software, network and data configuration in which tests run. Guide: How to write a test plan

Test estimation

Predicting the effort, duration and cost of testing using techniques such as work breakdown, three-point or test point analysis. Guide: Test estimation

Test level

A group of test activities organized together, such as unit, integration, system and acceptance. Guide: Test strategy

Test management

Planning, estimating, monitoring and controlling test activities and reporting their results. Hub: Test Management

Test oracle

A source of the expected result for a test: a specification, a comparable system, a formula or a person.

Test plan

A document describing the scope, approach, resources and schedule of intended test activities. Guide: How to write a test plan

Test point analysis

An estimation technique that sizes testing from function points adjusted for quality requirements and productivity. Guide: Test estimation

Test process

The set of activities from planning through completion by which testing is carried out. Guide: Software test process

Test process improvement

Systematic improvement of how an organisation tests, often assessed against a maturity model such as TMMi. Guide: TMMi

Test strategy

A high-level description of the test levels, types and approach for an organisation or programme, from which plans are derived. Guide: Test strategy

Test type

A group of test activities aimed at a particular quality characteristic: functional, performance, security, usability, regression. Guide: Test strategy

Three amigos

A short conversation between a business, a development and a testing perspective to agree examples before work starts. Guide: Acceptance test driven development

Three-point estimation

An estimate combining optimistic, most likely and pessimistic values, commonly weighted (O + 4M + P) / 6. Guide: Test estimation

TMMi

The Test Maturity Model integration: five levels of test process maturity from Initial to Optimization. Guide: TMMi

Traceability

The ability to relate tests to the requirements and risks they cover, and defects to the tests that found them. Guide: Test coverage

Trust list

A published list of the certificate authorities or signers whose certificates a C2PA validator accepts. A manifest whose signature verifies but whose certificate chains to no trusted list validates as signed by an unknown party, which is one of the four verification outcomes. Guide: How to verify content credentials

U

Unit testing

Testing individual components in isolation, usually by the developer who wrote them. Guide: Test automation pyramid

Usability testing

Testing how easily users can learn and use a system to achieve their goals. Hub: Testing Techniques

V

Volume testing

Testing with large amounts of data to check capacity and behaviour at scale. Guide: Performance testing

W

Watermark (statistical text)

A hidden pattern a cooperating text generator embeds by nudging word choices towards a secret subset of the vocabulary. A detector holding the key counts how often the subset appears and runs a significance test. Evidence weakens with editing and short length, and absence proves nothing. Guide: Text watermarking for language models

White-box testing

Testing based on knowledge of the internal structure of the code. Guide: Black box vs white box testing

Workload model

A description of the mix, rate and pattern of transactions a performance test should reproduce, derived from production data. Guide: Performance testing