Testing AI disclosure labels: cases, oracles and what breaks

A disclosure label is a feature, and a claim

A disclosure label is a statement about content: this was generated or materially altered by an automated system. As a feature it carries three obligations. It must be displayed wherever the content is displayed, in a form a reader can perceive. It must be persisted with the content through every copy, transform and export. And it must be honoured downstream: an embed, an API response or a share card that carries the content must carry the label.

Because it is also a claim about a person's work, it has a second audience: the person the label is about. A missing label misleads the reader; a wrong label accuses the author. Both are defects, of different severities, and the test design has to find both.

Where labels come from: declared, signed, inferred

Labels enter a system by three routes, and each needs its own tests. A declared label is the user's statement at upload or publish time: a checkbox or a required question. A signed label is an assertion inside a provenance manifest: the C2PA actions assertion carries a digital source type that can record that content was created or edited by a generative model, and a validator can read it without trusting the uploader. An inferred label comes from detection: a classifier scored the content and the platform applied a label on its own authority.

The three differ in what they prove. A declaration proves the user said so. A signed assertion proves the named signer said so over these exact bytes, and how to read that result is the subject of how to verify content credentials. An inferred label proves a score crossed a threshold, a probabilistic statement with an error rate. Your test cases must not let the system fold the three into one boolean.

Partitioning the inputs

Three dimensions give the partitions: origin (declared, signed, inferred, none), medium (text, image, audio) and edit state (original, edited, excerpted), 36 classes in full combination. Equivalence partitioning collapses these to the classes where the expected result differs; the boundaries sit wherever a transform may or may not remove the label. The table lists the cases that most often find defects.

Testing AI disclosure labels: the oracles

Four oracles cover most of the requirement. Shown wherever shown: enumerate every surface that renders the content (item page, feed, embed, share card, API, notifications, search) and assert the label on each. Survives transport: share, embed and re-upload the item and assert the label at the destination. Not silently removable: apply a crop, a re-encode, a trim or an excerpt and assert that the label persists or that the system logs a deliberate removal. Signed implies labelled: an item with a generated-by assertion and no label is a defect, whatever the uploader declared.

Two supporting oracles catch the accusation side. The label's basis (declaration, assertion, or inference with score and model version) must be recorded and shown on request, an inferred label must be contestable with a logged outcome, and a control set of human-made items must come out unlabelled.

Worked example: 60 items across 5 surfaces

A team testing a publishing platform built 60 items: 20 declared, 20 signed, 10 inferred and 10 human controls, across text, image and audio, a third of each group edited or excerpted, and checked each on five surfaces against criteria fixed before the run. The counts are invented but realistic.

The item page failed once, on an excerpted text item whose label was lost when the quotation was rendered. The feed failed 6: 4 audio items, because the player component had no label slot, and 2 false positives. The API returned the label on 51 of 60; the 9 misses were the inferred items, whose label lived in a moderation table the API did not join. The embed failed 14: all 10 inferred items, plus 4 signed images whose crop had re-encoded the file and dropped the manifest with no platform record behind it. The share card failed 31: it was generated from the raw media and carried no label on any medium.

The 2 false positives were human controls, both photographs of screens, labelled by the inference pipeline with no declaration or assertion. Against the criteria, the platform failed on the share card, the embed and the false positive rate, and passed the item page and the signed-implies-labelled oracle on 19 of 20 signed items.

Two kinds of wrong: false negatives and false positives

A false negative, a generated item shown without its label, harms the reader and, where disclosure is required, exposes the operator. A false positive, a human-made item shown with a label, harms the author: it is an accusation made by the system, in public, with the visual weight of a verified fact. That is the same harm as a detector false positive, and the arithmetic in false positives and base rates applies unchanged: at low prevalence, a large share of inferred labels are wrong even with a good classifier.

So the two classes get different oracles and severities. False negatives are found by the display and persistence oracles; false positives by the control set and the contest path: a wrongly labelled author must be able to challenge, a person must review, and the label must be removed with a record. A platform that labels on inference alone, with no contest path, has shipped a detector as a verdict.

Regulation and accessibility, in general terms

Several jurisdictions now require disclosure of synthetic media in some circumstances. The EU Artificial Intelligence Act, Regulation (EU) 2024/1689, sets transparency obligations in Article 50, including that providers of systems generating synthetic audio, image, video or text ensure the output is marked in a machine-readable format and detectable as artificially generated or manipulated, and that deployers of deepfake content disclose it as such. None of this is legal advice: which obligations apply is for counsel to state; the tester turns the stated obligation into oracles and an evidence-grade record.

Accessibility is part of the requirement. A label conveyed by colour alone, or by an icon with no accessible name, does not exist for a screen reader user or a colour-blind reader. Test with a screen reader on each surface, check that the label has a text alternative and sits in the reading order, and check its contrast against WCAG 2.2. Mobile needs its own pass: a label that fits a desktop card is often truncated on a phone.

Pitfalls

• Testing the database, not the screen. The label record is correct and the surface omits it. Assert on rendered output, per surface.
• One boolean for three origins. Declared, signed and inferred collapse into a single flag, and the contest path, the basis display and the severity model break with it.
• Trusting the manifest to persist. Crops and re-encodes drop manifests, so the platform must keep its own label record; which channels strip which layer is measured in metadata survival testing.
• No control set. A run with only generated items cannot measure false positives.
• Label copy that overclaims. A label reading generated with AI on a photograph whose only automated action was a crop misleads. The text must match the action in the assertion.

Label testing sits between provenance and detection in the testing AI systems hub.

This guide is part of the testing AI systems hub. It is best read alongside metadata survival testing and how to verify content credentials, which cover the neighbouring questions.