Why clean-output benchmarks overstate detection
A detector learns, or is tuned on, the statistical fingerprint of text as a model produced it. Every change a person or a second tool makes blurs that fingerprint. Krishna and colleagues showed in 2023 that running machine text through a paraphrasing model cut the detection rate of several detectors to a small fraction of its original value at a fixed low false positive rate. Sadasivan and colleagues argued that as language models improve, the best possible detector approaches a coin toss for text that has been paraphrased repeatedly.
None of this makes testing pointless. It makes the clean benchmark one slice among several, and it changes what a responsible report says. A detector benchmark that omits adversarial slices describes a situation that users rarely face.
The transformations to test
| Slice | How to build it | What it represents |
|---|---|---|
| Light human edit | Writers revise machine drafts for 10 minutes: reorder, cut, add a sentence | Ordinary use of a drafting assistant |
| Automated paraphrase | Pass machine text through a paraphrasing tool at default settings | Deliberate evasion, and many writing aids |
| Translation round trip | Translate into another language and back with a machine translator | Multilingual writers and evasion |
| Mixed authorship | Documents with alternating human and machine paragraphs in stated ratios | Collaboration, and partial assistance |
| Human control | Human-written text passed through the same tools | Whether the transformation itself raises false positives |
The human control slice is the one teams forget. Grammar and paraphrasing tools can push human writing towards the statistical profile of machine text, which raises false positives. If a detector flags human text more often after a grammar pass, that is a fairness problem as much as a robustness one, because some writers depend on those tools.
Pass criteria and reporting
Write the criteria before the run. A reasonable pair: the false positive rate on every human control slice stays below the ceiling, and the report states the detection rate on every adversarial slice with its interval. There is rarely a sensible minimum detection rate for adversarial slices; the honest output is the size of the drop, so that decision makers know how little a negative result means once text has been reworked.
- Hold the threshold fixed across slices; do not tune it per transformation.
- Version the transformation tools and settings, because a paraphraser update changes the slice.
- Keep the adversarial slices out of any tuning, or the detector will be fitted to one paraphraser and fail on the next.
A detector that passes a clean benchmark can still miss most text that has been paraphrased. A low score is weak evidence that a text is human-written.
Worked example: one source set through every transformation
The figure at the top of this guide gives rates; the counts behind them are the useful part. The numbers below are invented but realistic: 500 machine-written source documents and 500 human-written controls, all between 400 and 800 words, scored at the production threshold of 0.95, with the same transformations applied to both sets. Detection is counted on the machine set and false positives on the human set.
| Slice | Machine documents flagged | Detection rate | Human documents flagged | False positive rate |
|---|---|---|---|---|
| Clean, no transformation | 390 of 500 | 78% | 5 of 500 | 1.0% |
| Light human edit, 10 minutes | 260 of 500 | 52% | 6 of 500 | 1.2% |
| Grammar tool pass | 305 of 500 | 61% | 14 of 500 | 2.8% |
| Translation round trip | 185 of 500 | 37% | 21 of 500 | 4.2% |
| Automated paraphrase | 95 of 500 | 19% | 9 of 500 | 1.8% |
| Half human, half machine | 155 of 500 | 31% | n/a | n/a |
Read down the last column first. The transformations that are supposed to test evasion also change how honest writing scores: a translation round trip quadruples the false positive rate, and a grammar pass nearly triples it. A writer who drafts in one language and polishes in another has done nothing wrong and is now four times as likely to be flagged. Read the detection column second: paraphrasing leaves fewer than one machine document in five flagged, so a pass at 0.95 says very little about paraphrased text, and the mixed slice shows that a document with half its paragraphs generated is flagged less than a third of the time. Neither column changes the threshold; both change what a report must say about it.
Common mistakes in adversarial testing
- Building each slice from different source documents, so the drop measures topic and length rather than the transformation. One source set, every transformation applied to it.
- Leaving out the human control, and so missing the rise in false positives that grammar tools and round trips cause.
- Using one paraphraser at one setting and calling the result general. Detection after paraphrase varies widely between tools and settings; test at least two and version them.
- Letting the mixed-authorship slice use a single ratio. Detection on a document that is 20 percent machine text differs from one that is 80 percent; report three or four ratios.
- Feeding adversarial slices back into tuning, which fits the detector to last month's paraphraser.
- Quoting the clean detection rate in the product description and the adversarial rates in an appendix. Users meet the appendix case far more often than the headline one.
How to report an adversarial result
Report every slice with its counts, rate and interval at the fixed threshold, the transformation tool and version, the settings used, and the drop from the clean baseline in percentage points. Show the human control rates next to the detection rates, because the two together describe what the detector does to real writers. Keep the transformed documents, since a later reader may want to know exactly what a 10-minute edit looked like. A negative result on transformed text is weak evidence, and a positive result may be a false positive caused by the transformation itself, so if any verdict from an adversarially tested detector could be used against a writer, keep the run as an evidence-grade test record that shows both facts.
- Source set
- identifier and version of the machine and human source documents
- Transformation
- tool, version, settings, and the editing instructions given to writers
- Threshold
- the production value, fixed across every slice
- Per slice
- documents flagged, rate, 95 percent interval, drop from clean in points
- Kept
- the transformed documents and per-document scores
Adversarial testing as exploratory work
Beyond the fixed slices, give testers time-boxed sessions to find other transformations that move the score: changing punctuation habits, inserting typos, adding a personal anecdote, switching register. Run them as exploratory testing sessions with charters, record what moved the score and by how much, and promote the most effective findings to fixed slices in the next release.
Common questions
What is adversarial testing of an AI detector?
Measuring how well a detector works on text that has been changed after generation, by editing, paraphrasing, translation or mixing with human writing, rather than on clean machine output.
Does paraphrasing defeat AI text detectors?
Published studies found that automated paraphrasing sharply reduced the detection rate of several detectors at low false positive thresholds. Results vary by detector, so test the one you use.
Why include human text in adversarial tests?
Because the same tools can make human writing look machine-like and raise false positives. The control slice shows whether the transformation itself causes wrongful flags.
Should the threshold change for edited text?
No. Hold the production threshold fixed and report the detection rate per slice; tuning per transformation hides the real behaviour.
How often should adversarial slices be refreshed?
Whenever common writing and paraphrasing tools change, and at least every few months, since the population of transformations moves quickly.
Which transformation reduces detection the most?
In published studies and in most in-house runs, automated paraphrasing causes the largest drop, followed by translation round trips. Light human editing reduces detection less but still substantially, and it is the most common case in real use.
Sources
This guide is part of the testing AI systems hub. It is best read alongside fairness testing for ai detectors and text watermarking for language models, which cover the neighbouring questions.