What an ROC curve shows
A receiver operating characteristic curve plots two rates for every possible threshold. On one axis is the false positive rate, the share of human-written documents the detector flags. On the other is the detection rate, the share of machine-written documents it catches. A detector that guesses at random traces the diagonal; a perfect one hugs the top left corner. Real detectors sit in between, and the whole point of the curve is that you cannot move along it without trading one error for the other.
The curve is built from a labelled evaluation set, the same kind of set described in the guide to evaluation test set design. Score every document, sort by score, and at each distinct score compute the two rates. The result is a picture of the detector on that set, not a property of the detector in the world. Change the writers, the topics or the model that produced the machine text and the curve moves.
Choose the threshold from the harm, not the curve
Vendors and papers often summarise the curve as the area under it. That number answers a question nobody asks in practice: how well the detector ranks a random machine document above a random human one. A decision uses one threshold. The useful question is what happens at the threshold you will actually use, and for detectors used on people the harm is asymmetric. Accusing a student or a freelance writer is costly and hard to undo; missing some machine text is usually recoverable.
So start from a false positive ceiling written before you look at the results, just as a test plan writes its pass criteria first. A common ceiling is 1 percent on human-written text, with the upper end of the confidence interval below 2 percent. Find the highest threshold that meets it on the evaluation set, then report the detection rate at that point.
| Threshold | False positive rate (human text) | Detection rate (machine text) | Meets 1% ceiling? |
|---|---|---|---|
| 0.95 | 0.4% | 61% | Yes, with margin |
| 0.90 | 1.1% | 74% | No, just over |
| 0.80 | 2.6% | 86% | No |
| 0.70 | 5.3% | 92% | No |
| 0.50 | 11.8% | 97% | No |
Report the threshold, the false positive rate and the detection rate at that threshold, each with a confidence interval and the size of the set it came from. The area under the curve can accompany those numbers; it cannot replace them.
When the target is rare, add a precision view
An ROC curve does not change with the share of machine text in the population, which is both its strength and its trap. When machine text is rare, a small false positive rate still produces many wrong flags, as the guide to false positives and base rates works through. Saito and Rehmsmeier showed that precision-recall plots are more informative than ROC plots on imbalanced data for exactly this reason. Show both when the detector will be used on a population where most writing is human.
Thresholds drift, so test them like configuration
A threshold chosen in March is a configuration value tied to one model version and one corpus. When the detector is updated, when a new writing assistant becomes popular, or when the population changes, the same threshold can sit at a very different point on the curve. Treat the threshold as a tested setting: store it with the evaluation set version, re-run the curve on every change, and fail the release if the ceiling is no longer met. The same discipline appears in confidence intervals for classifier metrics, which decides whether a movement is real.
- Fix the threshold before the final evaluation run, never after.
- Report per slice (language, length, genre), because a threshold that meets the ceiling overall can fail it for one group of writers.
- Keep a small hidden set that is never used to tune anything, and use it only for the release decision.
Worked example: the same threshold checked per slice
The table above chose 0.95 because it met the 1 percent ceiling on the whole human set. Whether it meets the ceiling for every kind of writer is a separate question, and the answer often differs. The counts below are invented but realistic: the same 1,000 human documents split into four slices, scored at the 0.95 threshold, with the upper end of the 95 percent Wilson interval shown because that is the number the ceiling applies to.
| Slice | Flagged at 0.95 | False positive rate | Upper bound | Meets 1% ceiling? |
|---|---|---|---|---|
| Essays, over 600 words | 1 of 400 | 0.3% | 1.4% | Point yes, bound no |
| Essays, under 300 words | 2 of 200 | 1.0% | 3.6% | No |
| Technical reports | 1 of 250 | 0.4% | 2.2% | No |
| Second-language writers | 0 of 150 | 0.0% | 2.5% | No: too few cases |
A per-slice pass needs per-slice sample sizes, typically several hundred documents in each slice before the upper bound can fall under 1 percent. A slice with zero flags is no clean bill of health either: zero in 150 supports a bound near 2 percent and nothing tighter, so the fairness question for second-language writers stays open until more documents are scored.
Common mistakes when reading an ROC curve
| Mistake | Why it misleads | What to do instead |
|---|---|---|
| Choosing the threshold after seeing the results | The chosen point is fitted to one set and will not hold on the next | Write the ceiling into the plan; choose on a tuning set; confirm on a hidden set |
| Comparing detectors by area alone | Two curves with equal area can differ by a factor of three in false positives at the working point | Compare false positive and detection rates at the ceiling you will use |
| Reading the 0.5 default as neutral | Scores are rarely calibrated probabilities; 0.5 often flags 10 percent or more of human text | Treat the default as untested until the curve says otherwise |
| Reusing the curve after a model update | The score scale moves with every retraining, so the old threshold sits at a new point | Re-run the curve on every change and store it with the version |
How to report an operating point
A threshold decision is read by people who were not in the room, sometimes by people arguing about one flagged document. The minimum record is the detector name and version, the evaluation set identifier, the threshold, the false positive rate with its count, denominator and interval, the detection rate in the same form, the per-slice table, the date and who ran it. State the ceiling and where it came from, and name the slices that were too small to pass on their own. If a flag at this threshold may ever be used against a writer, keep the run in the form described in evidence-grade test records, where inputs, outputs and settings are preserved together and cannot be quietly regenerated later.
Common questions
What is an ROC curve for an AI detector?
A plot of the detector's detection rate on machine-written text against its false positive rate on human-written text, computed at every possible score threshold on a labelled evaluation set.
Is a higher area under the curve always better?
It means better ranking on average, but a decision uses one threshold. Two detectors with the same area can have very different false positive rates at the threshold you would actually use.
How should I pick a detector threshold?
Set a false positive ceiling on human-written text first, for example 1 percent, then choose the threshold that meets it with margin on a held-out set, and report the detection rate at that point.
Why does the default threshold of 0.5 cause problems?
Scores are not probabilities unless the detector is calibrated. At 0.5 many detectors flag a large share of human writing, as the worked example shows.
How often should a threshold be re-checked?
On every detector update, every change to the evaluation corpus, and on a regular schedule, because the population of writing and writing tools changes over time.
What is the difference between an ROC curve and a precision-recall curve?
An ROC curve plots detection rate against false positive rate and ignores how common machine text is. A precision-recall curve plots the share of flags that are correct against detection rate, so it changes with prevalence and is more informative when machine text is rare.
Sources
- Saito and Rehmsmeier, The Precision-Recall Plot Is More Informative than the ROC Plot on Imbalanced Datasets, PLOS ONE (2015)
- NIST/SEMATECH e-Handbook of Statistical Methods, 7.2.4.1 Confidence intervals for a proportion (Wilson method)
- Weber-Wulff et al., Testing of Detection Tools for AI-Generated Text (2023)
Further reading named in the text
- Tom Fawcett, An Introduction to ROC Analysis (Pattern Recognition Letters 27, 2006)
This guide is part of the testing AI systems hub. It is best read alongside false positives, base rates and the confusion matrix and confidence intervals for classifier metrics, which cover the neighbouring questions.