How AI text detectors work: statistics, classifiers, watermarks

Approach 1: predictability statistics

A language model assigns a probability to every possible next word. Text that a model generated tends to use words the model found likely, while people make more surprising choices. Statistical detectors run the text through a model and measure how predictable it is: the average probability of each word, how often words fall among the model's top choices, and how much that varies across sentences. Gehrmann, Strobelt and Rush built GLTR on this idea and found that showing people these rank statistics helped them spot generated text more often than they could unaided.

The weakness is that predictability is not authorship. Formulaic human writing, such as lab reports, legal boilerplate or essays by people writing in a second language, is predictable too. That is why these detectors produce the uneven false positive rates that fairness testing looks for.

Approach 2: trained classifiers

A classifier detector is itself a model, trained on many examples labelled human or machine. It learns whatever features separate the two in its training data, which can include vocabulary, punctuation habits, sentence rhythm and formatting. Classifiers can be accurate on text like their training data and much weaker on anything else: a new generator, a new genre, or text that has been edited. Their behaviour is only as well understood as their evaluation sets, which is why independent benchmarks matter.

Approach 3: watermarks

A watermark is a pattern the generator adds on purpose. In the scheme described by Kirchenbauer and colleagues, the generator nudges its word choices towards a pseudo-random list that depends on the preceding words; a detector that knows the key counts how often the text lands on that list and computes a statistical test. Watermarks can give strong evidence on long, unedited text from a cooperating system, but they say nothing about text from generators that do not embed them. The guide to text watermarking covers what they can and cannot show.

Worked example: one essay, three detectors

A 600-word essay on a set topic, written by a student without assistance, is run through one detector of each kind. The numbers are invented but realistic and show how three sound methods reach three different conclusions about the same honest text.

Each output is correct on its own terms. The essay is predictable, it does resemble the classifier's human examples, and it carries no watermark. What none of the three measured is how the essay was written, which is why the flag from the first tool is a statement about style. The student's drafts and version history settle the question in a way no fourth detector can.

Common misreadings of detector output

• Reading a score of 0.86 as an 86 percent chance the writer used a machine. Scores are resemblance measures on the tool's own scale and are rarely calibrated probabilities.
• Treating a low score as proof of human authorship. Edited and paraphrased machine text scores low on every approach.
• Averaging scores from tools that use different approaches. A statistical score and a classifier score measure different things and share no scale.
• Running a 120-word paragraph. Below a few hundred words every approach is unstable, and sentence-level highlights are less reliable still.
• Assuming a watermark test covers all machine text. It covers text from one generator with one key and nothing else, so a negative result is silent about every other tool.

Questions to ask before using a detector

• Which approach does it use, and if it is a classifier, what kinds of text and which generators was it trained on?
• What is its measured false positive rate on human writing like ours, at the threshold we will use, with the count and interval, and when was that measured?
• How does it behave on texts under 300 words, on second-language writers and on text that has been through a grammar tool?
• What is the score scale, and has it been calibrated so that 0.8 means anything in particular?
• Can we get the per-document score rather than a label, so that we can set our own threshold and test it?

A vendor who can answer these with numbers has done the work described in benchmarking AI text detectors. One who answers with an overall accuracy figure has not, and the accuracy figure will have been measured on the easy case: long, clean machine output against long, edited human prose.

Detection is one part of authenticity

All three approaches work backwards from a finished text. The other direction is provenance: a record made at the time of creation that says what tool produced the content and what was done to it since, signed so that changes can be found. Content credentials do this for images and video today and are being extended to other media, and they are checked by verification rather than by inference, as the guide to verifying content credentials describes. Where such a record exists, a detector is redundant; where it does not, a detector is an estimate, and the guide to what a detector score means sets out how far that estimate can be trusted.

Why every approach weakens with editing

All three approaches read a statistical fingerprint, and editing smudges it. Paraphrasing, translation, and the ordinary revision people do all move text away from what the generator produced. Sadasivan and colleagues argued that as generators improve, the best achievable detection of paraphrased text approaches chance. In practice this means detection is strongest on long, untouched output and weakest in exactly the mixed cases that real writing produces, which the guide to why detectors disagree explores.

This guide is part of the testing AI systems hub. It is best read alongside why ai detectors disagree about the same text and text watermarking for language models, which cover the neighbouring questions.