Adversarial testing of AI text detectors: paraphrase and mixing
A detector measured only on untouched machine output is measured on the easiest case it will ever meet. In real use people edit, paraphrase, translate and mix their own sentences with generated ones, sometimes to hide assistance and often simply because that is how writing works. Adversarial testing measures what is left of the detection rate when text has been through those hands.
Lead guide7 min readUpdated
Run this test on your own sample
The protocol guide sets out the sample set, the threshold and the report in the order you need them.
Open the protocolWhy clean-output benchmarks overstate detection
A detector learns, or is tuned on, the statistical fingerprint of text as a model produced it. Every change a person or a second tool makes blurs that fingerprint. Krishna and colleagues showed in 2023 that running machine text through a paraphrasing model cut the detection rate of several detectors to a small fraction of its original value at a fixed low false positive rate. Sadasivan and colleagues argued that as language models improve, the best possible detector approaches a coin toss for text that has been paraphrased repeatedly.
None of this makes testing pointless. It makes the clean benchmark one slice among several, and it changes what a responsible report says. A detector benchmark that omits adversarial slices describes a situation that users rarely face.
The transformations to test
The human control slice is the one teams forget. Grammar and paraphrasing tools can push human writing towards the statistical profile of machine text, which raises false positives. If a detector flags human text more often after a grammar pass, that is a fairness problem as much as a robustness one, because some writers depend on those tools.
Pass criteria and reporting
Write the criteria before the run. A reasonable pair: the false positive rate on every human control slice stays below the ceiling, and the report states the detection rate on every adversarial slice with its interval. There is rarely a sensible minimum detection rate for adversarial slices; the honest output is the size of the drop, so that decision makers know how little a negative result means once text has been reworked.
- Hold the threshold fixed across slices; do not tune it per transformation.
- Version the transformation tools and settings, because a paraphraser update changes the slice.
- Keep the adversarial slices out of any tuning, or the detector will be fitted to one paraphraser and fail on the next.
Worked example: one source set through every transformation
The figure at the top of this guide gives rates; the counts behind them are the useful part. The numbers below are invented but realistic: 500 machine-written source documents and 500 human-written controls, all between 400 and 800 words, scored at the production threshold of 0.95, with the same transformations applied to both sets. Detection is counted on the machine set and false positives on the human set.
Read down the last column first. The transformations that are supposed to test evasion also change how honest writing scores: a translation round trip quadruples the false positive rate, and a grammar pass nearly triples it. A writer who drafts in one language and polishes in another has done nothing wrong and is now four times as likely to be flagged. Read the detection column second: paraphrasing leaves fewer than one machine document in five flagged, so a pass at 0.95 says very little about paraphrased text, and the mixed slice shows that a document with half its paragraphs generated is flagged less than a third of the time. Neither column changes the threshold; both change what a report must say about it.
Common mistakes in adversarial testing
- Building each slice from different source documents, so the drop measures topic and length rather than the transformation. One source set, every transformation applied to it.
- Leaving out the human control, and so missing the rise in false positives that grammar tools and round trips cause.
- Using one paraphraser at one setting and calling the result general. Detection after paraphrase varies widely between tools and settings; test at least two and version them.
- Letting the mixed-authorship slice use a single ratio. Detection on a document that is 20 percent machine text differs from one that is 80 percent; report three or four ratios.
- Feeding adversarial slices back into tuning, which fits the detector to last month's paraphraser.
- Quoting the clean detection rate in the product description and the adversarial rates in an appendix. Users meet the appendix case far more often than the headline one.
How to report an adversarial result
Report every slice with its counts, rate and interval at the fixed threshold, the transformation tool and version, the settings used, and the drop from the clean baseline in percentage points. Show the human control rates next to the detection rates, because the two together describe what the detector does to real writers. Keep the transformed documents, since a later reader may want to know exactly what a 10-minute edit looked like. A negative result on transformed text is weak evidence, and a positive result may be a false positive caused by the transformation itself, so if any verdict from an adversarially tested detector could be used against a writer, keep the run as an evidence-grade test record that shows both facts.
Adversarial testing as exploratory work
Beyond the fixed slices, give testers time-boxed sessions to find other transformations that move the score: changing punctuation habits, inserting typos, adding a personal anecdote, switching register. Run them as exploratory testing sessions with charters, record what moved the score and by how much, and promote the most effective findings to fixed slices in the next release.
This guide is part of the testing AI systems hub. It is best read alongside fairness testing for ai detectors and text watermarking for language models, which cover the neighbouring questions.
Work it out
Questions
What is adversarial testing of an AI detector?
Measuring how well a detector works on text that has been changed after generation, by editing, paraphrasing, translation or mixing with human writing, rather than on clean machine output.
Does paraphrasing defeat AI text detectors?
Published studies found that automated paraphrasing sharply reduced the detection rate of several detectors at low false positive thresholds. Results vary by detector, so test the one you use.
Why include human text in adversarial tests?
Because the same tools can make human writing look machine-like and raise false positives. The control slice shows whether the transformation itself causes wrongful flags.
Should the threshold change for edited text?
No. Hold the production threshold fixed and report the detection rate per slice; tuning per transformation hides the real behaviour.
How often should adversarial slices be refreshed?
Whenever common writing and paraphrasing tools change, and at least every few months, since the population of transformations moves quickly.
Which transformation reduces detection the most?
In published studies and in most in-house runs, automated paraphrasing causes the largest drop, followed by translation round trips. Light human editing reduces detection less but still substantially, and it is the most common case in real use.
