How AI detectors work
There is no database of generated text and nothing to match against. A detector is a classifier making a guess from statistical properties, and that explains almost everything about what it can and cannot do.
The most common misconception about AI detectors is that they look something up. They do not. There is no corpus of ChatGPT output sitting behind them, nothing to match your paragraph against, no source to click through to.
A detector takes text, pulls out numbers, and estimates how closely those numbers resemble the profile of machine-generated writing. Everything odd about how these tools behave follows from that.
The three signals
Nearly every detector in use combines some version of three things.
Predictability
Models are trained to pick likely continuations. Given "the results of the", a model puts high probability on "study" and very little on "kettle". Text produced by sampling from those probabilities is, averaged out, made of more predictable words than text a person wrote.
Two derived measures come up constantly. Perplexity is roughly the average surprise per word, so lower means more predictable. Burstiness is how much that surprise swings from sentence to sentence, and human writing spikes and dips where generated writing runs flat.
Neither settles anything alone. A formal methods section written by a person also has low perplexity, which is precisely why false positives happen.
Rhythm and structure
Properties with nothing to do with what the text is about:
- sentence length, and how much it varies between consecutive sentences
- punctuation rates, comma and semicolon density
- clause structure, how often subordination shows up
- function word frequencies, how often the, of, and, to appear
The first carries the most weight. Across our own corpus, the standard deviation of sentence length within a document has a median of 8.7 words for pre-2020 arXiv writing and 6.4 for generated text.
These are the stylometric signals, and they have two useful properties. They are topic independent, so a paper on protein folding and one on Byzantine history look similar if similar people wrote them. And they are durable, because they survive rewriting that changes every single word.
Vocabulary distribution
Which words and short phrases turn up, and how often. Not a banned-word list, a learned distribution. Certain terms get over-selected by models relative to comparable human writing, and it is the density that signals rather than the presence of any one word.
How a score gets made
Most production detectors combine several models rather than betting on one. Ours runs three: a small fine-tuned transformer reading meaning, a lexical model over word and phrase choice, and a stylometric model over sixty hand-built features covering the rhythm signals above.
Those three produce numbers on incompatible scales, so each gets calibrated separately before a small combiner weighs them into one figure. Full method in how the unslop AI detector works.
The threshold is a choice somebody made
A classifier outputs a continuous score. Turning that into flagged or not flagged needs a threshold, and where it goes is a decision with consequences rather than a fact about the world.
Low threshold: you catch more generated text and flag more human writing. High threshold: the reverse. Nothing avoids both, because the two distributions genuinely overlap.
Most tools pick whatever maximises accuracy on their own test set, which optimises the average and quietly accepts whatever false positive rate comes with it. We go the other way: choose a tolerable false positive rate first, accept the resulting detection rate. Ours is 0.8% at the public setting, 0.4% stricter.
Neither approach is wrong in the abstract. They just encode different opinions about which error costs more.
Detection gets harder every year, by construction
This is the uncomfortable part.
Detection works because generated text is more predictable and more uniform than human text, and each model generation shifts where that line sits. Newer models vary sentence length more and hedge less mechanically, so a detector has to be retrained on current output to keep up.
This is the difference between detectors that hold up and detectors that quietly decay. One built before a model family existed has never seen it. Ours is trained across 14,892 generated documents covering seven model families and five years of GPT versions, including current-generation models, which is what keeps the profile broad.
Four things no detector does
Identify which model wrote something. Some families have distinctive profiles, but this is not reliable enough to lean on.
Prove authorship. A score describes text. Text carries no record of its own production, so a model reproducing a human passage verbatim is indistinguishable from the human original.
Work reliably on short text. A few hundred words carries far less signal, which is why sentence-level error rates run several times document-level ones across the whole field.
Work outside its language. Ours is English only and returns an explicit unsupported-language flag rather than a confident wrong answer.
Run text through ours, free and unlimited, scored per passage rather than as one number for the document.