False positives, base rates and the confusion matrix
A classifier that is right 98 percent of the time sounds dependable. Whether it is depends on something the classifier does not control: how common the thing it looks for really is. When the target is rare, even a small false positive rate produces more wrong flags than right ones. Testers who report on detectors, fraud rules or any screening model need this arithmetic more than any other.
Foundation guide8 min readUpdated
Run this test on your own sample
The protocol guide sets out the sample set, the threshold and the report in the order you need them.
Open the protocolThe four cells and the rates built from them
Every binary classifier result, whether a spam filter, a fraud rule or an AI text detector, falls into one of four cells. A true positive is a flagged document that really is machine-written; a false negative is a machine-written document that passed; a false positive is a human-written document that was flagged; a true negative is a human-written document that passed. All the familiar rates are ratios of these four counts, and each one answers a different question.
The first three rates are properties of the classifier on each class and do not depend on how many documents of each class are in the set. Precision and accuracy do. That is the root of most misreadings: a benchmark with equal numbers of human and machine documents reports a precision that will not survive contact with a population where machine text is rare, or where it is the majority.
Base rates change what a flag means
Take the same detector, with a 94 percent detection rate and a 2 percent false positive rate, and apply it to 10,000 documents of which 5 percent are really machine-written. It catches 470 of the 500 machine documents and wrongly flags 190 of the 9,500 human ones. Of 660 flags, 190 are wrong: precision has fallen from 97.9 percent to 71.2 percent without anything about the detector changing. At a 1 percent base rate, precision is 32.2 percent: two of every three flags point at a person who wrote their own text.
This is the same arithmetic that makes screening tests for rare conditions produce many false alarms, and it is not a flaw that better engineering removes. Lowering the false positive rate helps most, which is why the figure above shows three detectors that differ only in specificity. Testers should report the expected precision at the population's plausible base rate, or a small table across a range of base rates when nobody knows it, next to the rates measured in the detector benchmark.
Thresholds trade one error for the other
Most classifiers output a score, and the flag depends on a threshold. Raising the threshold lowers the false positive rate and the detection rate together. An ROC curve plots detection rate against false positive rate across every threshold; Fawcett's introduction remains the standard reference. When the target is rare, Saito and Rehmsmeier showed that the precision-recall plot is more informative, because ROC curves can look excellent while precision is poor. For a decision that falls on a person, pick the threshold from the false positive ceiling first and accept the detection rate that results.
Uncertainty belongs in the number
A rate measured on a sample is an estimate. Report it with a confidence interval: the Wilson score interval behaves well for rates near zero, where the simple normal approximation fails. Twenty false positives in 1,000 human documents gives 2.0 percent with an interval of 1.3 to 3.1 percent. Two differences are only meaningful if the intervals say so; a detector at 1.8 percent is not better than one at 2.2 percent on samples of this size. The guide to evaluation test set design shows how to size the sample for the precision you need, and the testing metrics guide covers the general rule that a metric reported alone misleads.
For the published numbers behind these warnings, including a 2023 test in which no detector reached 80 percent accuracy, see do AI detectors work.
This guide is part of the testing AI systems hub. It is best read alongside how to benchmark an ai text detector and evaluation test set design for ai models, which cover the neighbouring questions.
Work it out
Questions
What is a confusion matrix?
A two-by-two table of a classifier's results: true positives, false negatives, false positives and true negatives. Every standard rate (detection rate, false positive rate, precision, accuracy) is a ratio of its cells.
What is the difference between false positive rate and precision?
The false positive rate is the share of real negatives that are wrongly flagged. Precision is the share of flags that are right. The first is a property of the classifier; the second also depends on how common positives are in the population.
Why does a low false positive rate still produce many wrong flags?
Because when most items are negative, a small rate applied to a large number produces many false positives. At a 1 percent base rate, a 2 percent false positive rate yields about two wrong flags for every right one, even with a 94 percent detection rate.
Is accuracy useful for AI detectors?
Only with care. Accuracy blends both error types and depends on the class mix in the test set. A detector that flags nothing is 99 percent accurate on a population where 1 percent of documents are machine-written.
Which curve should a detector report, ROC or precision-recall?
Both have uses, but when the target class is rare the precision-recall curve shows the practical cost of false positives more honestly. Report the operating point actually used, with its rates and intervals, whatever curve accompanies it.
Sources
- Saito and Rehmsmeier, The Precision-Recall Plot Is More Informative than the ROC Plot on Imbalanced Datasets, PLOS ONE (2015)
- Liang et al. (2023), detectors of machine-written text are biased against non-native English writers
- NIST/SEMATECH e-Handbook of Statistical Methods, 7.2.4.1 Confidence intervals for a proportion (Wilson method)
- Tom Fawcett, An Introduction to ROC Analysis (Pattern Recognition Letters 27, 2006)
- Edwin B. Wilson, Probable Inference, the Law of Succession, and Statistical Inference (Journal of the American Statistical Association 22, 1927)
- Alan Agresti and Brent A. Coull, Approximate Is Better than Exact for Interval Estimation of Binomial Proportions (The American Statistician 52(2), 1998)
- Gerd Gigerenzer, Calculated Risks: How to Know When Numbers Deceive You (Simon and Schuster, 2002)
