Perplexity and burstiness, explained
The two terms every AI detection explainer uses and most of them define badly. Both are simple, and knowing what they measure tells you exactly when they fail.
If you have read anything about AI detection you have met these two words. They are usually introduced and then never explained, which is a shame, because between them they account for most of how these tools work and all of why they misfire.
Perplexity: how surprised a model is by your next word
Take a language model and feed it your text one word at a time. At every position it has an opinion about what comes next. Given "the results of the", it puts high probability on "study" and almost none on "kettle".
Perplexity is roughly the average surprise across the whole passage. Low perplexity means the words were largely what the model expected. High perplexity means they were not.
Generated text tends to have low perplexity, and the reason is close to circular: models are trained to pick likely continuations, so text produced by sampling from a model is, on average, made of words that models find likely.
What it misses on its own. Plenty of human writing is also predictable. A methods section, a legal clause, a formal report. Any detector betting on perplexity alone will flag those, which is the single most common reason people meet a wrong flag.
Burstiness: how much that surprise swings around
Perplexity averaged over a whole document throws away something useful, which is how much it varies from sentence to sentence.
Human writing is bursty. A person writes a dense, unexpected sentence, then a plain one, then a long qualified one. Surprise spikes and dips. Generated writing tends to run flat, holding a fairly constant level of predictability from start to finish.
The everyday version of burstiness is sentence length variation, which you can measure without any model at all. Across our corpus, the standard deviation of sentence length within a document has a median of 8.7 words for pre-2020 arXiv writing and 6.4 for generated text.
What it misses on its own. Burstiness is a property of a passage, not a sentence, so it needs length to mean anything. It also drops when writing is edited hard, because editing smooths. A detector reading burstiness alone flags heavily revised human papers, which is why ours weighs it against two other models rather than trusting it.
Why neither is enough alone
Both measure the same underlying thing from different angles: how regular the text is. Both therefore fail on the same writing, which is regular human prose.
That is why serious detectors combine signals rather than betting on one. Ours runs three models, a small transformer reading meaning, a lexical model over word choice, and a stylometric model over sixty features including the sentence rhythm ones above. Each gets calibrated separately before a small combiner weighs them. See how AI detectors work.
Why single-signal detectors age badly
Perplexity and burstiness work because generated text is more predictable and more uniform than human text, and each model generation shifts where that line sits. Newer models vary sentence length more and hedge less mechanically.
A detector that reads one of these numbers and stops has to be retrained from scratch every time that happens, and most are not. Ours combines three models over 14,892 generated documents from seven model families including current-generation ones, and holds 89.9% on OpenAI GPT output across five years of releases at 99.6% specificity.
The practical version
If you want to know why a passage of yours scored high, check the two things these terms describe:
- Is it predictable? Conventional phrasing, formal register, field-standard constructions
- Is it flat? Count the words in each sentence and look at the spread. A narrow range is
the signal
Both are checkable by eye, and both are fixable without changing what you are saying. See what makes text read as machine written.
Ours is free and unlimited and scores each paragraph separately, so you can see which passages are flat rather than guessing.