Testing AI Systems and Authenticity Pipelines

Testing applied to the systems that claim to prove who made a piece of content, how it was made and whether it was altered or generated. Twenty-seven guides: detector benchmarks, base rates, thresholds and intervals on one side; Content Credentials pipelines, metadata survival, disclosure labels and evidence-grade test records on the other.

Testers measure whether a system can be trusted, and an authenticity system is a system like any other. It arrives in one of three shapes. A classifier (an AI text detector, an image detector) returns a verdict that is wrong some of the time, so its oracle is a labelled evaluation set and its pass criterion is a rate fixed before the run: a false positive ceiling on human-written text, with a confidence interval. A signed pipeline (Content Credentials under the C2PA specification, the EXIF and XMP that a camera or an editor writes) carries a claim about who made a file and how, so its oracle is a validator and its pass criterion is a chain that verifies after every stage, or an honest absence. A label feature (an AI disclosure label on a platform) is a claim that has to appear on every surface and survive every transformation, so its oracle is one expected result per partition of origin, medium and edit state. Each has an oracle and a pass criterion, and each can be benchmarked, regression tested and recorded to an evidence standard. The twenty-seven guides in this hub cover the systems that claim to prove who made a piece of content, how it was made, and whether it was altered or generated, and they are arranged below by the question that brings a reader here.

If the question is about a detector, read how to test AI systems first for the three-layer approach: ordinary tests on the software around the model, rates on an evaluation set for the model's behaviour, and properties that must hold whatever the output. Then false positives, base rates and the confusion matrix, which shows why a detector that is right 98 percent of the time can be wrong about most of the documents it flags. Readers who have been handed a score, or who hand them out, continue with what a detector score means for a writer, teacher or editor, the three ways detectors work and why two detectors disagree about the same text. Readers who have to run the measurement continue with the detector benchmark protocol (300 human and 300 machine documents per slice, threshold fixed before the run, full confusion matrix reported), choosing a threshold from the ROC curve and Wilson intervals for every reported rate. The data side is evaluation set design by slices, size and leakage and agreement statistics for the labels; the slices that most often fail are covered in the fairness test on non-native writers and the adversarial slices: paraphrase, translation round trips and mixed documents. Statistical text watermarking closes the detector set: a watermark test can give strong evidence on long unedited text from a cooperating generator, and no evidence about anything else.

All 27 guides in this hub

In this hub: 27 guides, reviewed September 2026. Each opens with a figure, carries a quick answer and key figures, and names its sources.

#	Guide	What it settles	Level	Reading
1	How to test AI systems: oracles, evaluation sets and risk	A conventional test compares an output with an expected value.	Practitioner	9 min
2	How to benchmark an AI text detector: a test protocol	An AI text detector is a binary classifier with an unusually expensive false positive: a person accused of not writing their own work.	Lead	10 min
3	False positives, base rates and the confusion matrix	A classifier that is right 98 percent of the time sounds dependable.	Foundation	8 min
4	Evaluation test set design for AI models: slices, size, leakage	An AI evaluation is only as good as the cases it runs on. A test set that is too small cannot tell two models apart, one that leaks into training reports a skill the model does not have, and one that ignores the minority of inputs where the model fails reports a comforting average.	Lead	9 min
5	Test design for LLM features: assertions, rubrics and regression	A feature built on a large language model is a piece of software whose central function is written in natural language and executed by a component nobody on the team can read.	Practitioner	10 min
6	QA of AI-generated test cases and code: a review method	Assistants now write a large share of new unit tests and a growing share of production code.	Practitioner	9 min
7	ROC curves and thresholds for AI text detectors	A detector does not really say yes or no. It produces a score, and someone picks the number above which a document is flagged.	Practitioner	7 min
8	Confidence intervals for classifier metrics: a worked method	A false positive rate of 1.2 percent measured on 500 documents and one measured on 50,000 documents are not the same claim, even though the number is identical.	Practitioner	7 min
9	Fairness testing for AI detectors: non-native English writers	An AI text detector that works well on average can still fail badly for one group of writers.	Lead	7 min
10	Adversarial testing of AI text detectors: paraphrase and mixing	A detector measured only on untouched machine output is measured on the easiest case it will ever meet.	Lead	7 min
11	Metamorphic testing for language models: relations that hold	The hardest part of testing a language model feature is knowing what the right answer is.	Practitioner	7 min
12	Inter-rater agreement for AI evaluation labels	Every accuracy figure for an AI system rests on labels that people assigned.	Practitioner	7 min
13	Prompt regression testing: baselines, diffs and release gates	A one-word change to a prompt can fix the case in front of you and quietly break twenty others.	Practitioner	7 min
14	How AI text detectors work: statistics, classifiers, watermarks	An AI text detector looks at a piece of writing and returns a score that is meant to say how likely it is that a machine produced it.	Foundation	7 min
15	Why AI detectors disagree about the same text	Paste the same paragraph into three AI detectors and it is common to get three different answers: one says almost certainly machine-written, one says mostly human, one says it cannot tell.	Foundation	7 min
16	What an AI detector score means for a writer, teacher or editor	A detector reports 82 percent. The writer insists they wrote every word.	Foundation	7 min
17	Text watermarking for language models: what it can show	Detectors guess from the outside. A watermark is an attempt to work from the inside: the system that generates the text deliberately leaves a statistical trace that a matching test can find later.	Practitioner	7 min
18	Proving you wrote it: authorship evidence that holds up	Being told a machine wrote your work is disorienting, especially when you wrote every word.	Foundation	7 min
19	Testing a content provenance pipeline: does C2PA survive your CMS?	Testing a content provenance pipeline means feeding signed assets through every transformation your CMS applies and checking what comes out the other end.	Practitioner	8 min
20	How to verify content credentials: a tester's session	Knowing how to verify content credentials sounds like knowing where the button is, and the button exists.	Foundation	7 min
21	Evidence-grade test records for authenticity systems	A detector benchmark, a provenance pipeline check or a verification session produces a result that someone may later contest: the student who was flagged, the vendor whose tool failed, the counterparty who says the image was altered.	Practitioner	8 min
22	Metadata survival testing: what uploads and edits strip	Metadata survival testing answers a question every provenance claim depends on: after the file has been uploaded, resized, converted, shared and screenshotted, what is still attached to it?	Practitioner	8 min
23	Testing AI disclosure labels: cases, oracles and what breaks	Testing AI disclosure labels is a feature test with an unusual failure mode: when the label breaks, the user sees nothing wrong.	Practitioner	8 min
24	Testing deepfake detection: protocol, metrics and provenance	Testing deepfake detection follows the same discipline as testing an AI text detector: a labelled set the vendor could not have trained on, pass criteria written before the run, one fixed threshold, and a confusion matrix reported with its denominators.	Practitioner	8 min
25	Chain of custody for test evidence: logs to signed reports	A chain of custody is an unbroken, documented record of who held an item of evidence, when, and what they did to it.	Practitioner	8 min
26	Data lineage for evaluation sets: provenance of test data	Data lineage is the recorded path of each data item from its origin, through every transformation, to the place it is used.	Practitioner	8 min
27	Reproducibility: test results with provenance you can rerun	Reproducibility is the property that lets a second team take a test result, rerun the same procedure on the same setup and land inside the stated interval.	Practitioner	8 min

Where to start, by the question you have

Four guides cover generative features and generated code rather than detectors, because the same teams usually own both. Test design for language-model features layers deterministic, property and rubric assertions and passes on rates over repeated runs. Prompt regression testing turns that suite into a release gate with a baseline and an agreed margin. Metamorphic relations for language models supply an oracle when no expected output exists: invariance, direction and consistency across generated input pairs. Reviewing machine-generated tests and code belongs with them, because a generated test that kills no mutants is a false positive of a different kind.

If the question is about provenance, start with a verification session for content credentials: run the exact bytes through a validator, read the signature, the certificate chain, the content hashes and the assertions, and land on one of four outcomes, each of which proves less than people assume. Testing a content provenance pipeline is the practitioner protocol: a fixture of signed images pushed through every transformation stage of a CMS, the manifest validated after each one, and a worked example of 40 images through 9 stages. Metadata survival testing widens the fixture to every EXIF, XMP and C2PA field, hashes each layer separately and records preserved, rewritten or stripped per channel, with 12 probe images through 10 channels as the worked case. Testing AI disclosure labels treats the label as a feature and a claim at once: inputs partitioned by origin (declared, signed, inferred), medium and edit state, one expected result per partition, and 60 items across 5 surfaces in the example.

If the question is about evidence, two guides apply. Evidence-grade test records sets out what a record needs before it can be relied on in a dispute or an audit: the tester, inputs by hash, the exact version under test, criteria fixed before the run, raw outputs, timestamps and environment, stored write-once and reproducible by a second tester. Authorship evidence that holds up is the reader's side of the same problem: version history, dated drafts and the ability to explain the work carry weight, and a detector score on the finished text carries very little.

The guides lean on the rest of the site and on named sources. Risk-based testing decides how deep an evaluation goes, and its impact column is where a false positive on a person gets its weight. Testing metrics supplies the rule that no rate is reported without its denominator and its interval, and equivalence partitioning is how evaluation slices and label test cases are chosen. Sources are standards and published research rather than vendor material: the C2PA technical specification for manifests, bindings, validation and trust lists; the IPTC photo metadata standard and the ISO standard for XMP; the IETF specifications for signing structures, certificates and timestamps; the NIST overview of technical approaches to synthetic content and its guidance on forensic records; the ISO/IEC 29119 guidance on AI-based systems and the ISTQB AI testing syllabus; and peer-reviewed detector studies, in particular those that measured false positive rates on non-native writers and detection rates after paraphrasing. Every guide lists its sources by name and links them. None of the guides claims that a detector, a watermark or a provenance system is reliable in itself; each shows how to measure how far it can be trusted.

Four guides added in the bridge pass of September 2026 sit on the evidence side of the lane. Testing deepfake detection brings the detector protocol to images, audio and video, with recall per manipulation family and a provenance check on the same media. Chain of custody for test evidence says how a screenshot, a log or a detector report is hashed, held and handed over so it survives a dispute. Data lineage for evaluation sets records where every item in an evaluation set came from and how its label was established, which is the provenance behind every rate this hub reports. Reproducibility lists what to pin so that a second team reruns an AI evaluation to the same rates within the stated interval.