How Turnitin's AI detection works

Turnitin added AI writing detection in April 2023, and put it in the same report as the similarity score institutions had been using for twenty years. That decision is responsible for most of the confusion around it, because the two numbers sit inches apart and measure completely different things.

Only one of them is comparing your work to anything

The similarity score is a matching problem. Turnitin holds a corpus of student papers, web pages and publications, compares your text against it, and reports the overlap. When it flags a passage there is a source behind it. You can click through and read the thing it matched.

The AI writing indicator is a prediction. There is no source, no corpus, nothing matched. A model looks at the text and estimates how likely it is that a language model produced it. When it flags a passage there is nothing to click through to, because nothing was found.

That difference decides what each number can carry. A similarity match is checkable: the source either says that or it does not. An AI score is a classifier's output, and classifiers are wrong at a rate you can measure.

What it is actually looking at

Turnitin splits a document into passages of a few hundred words, scores each, and aggregates them into the percentage on the report. The underlying signal is the one nearly every detector uses: how predictable each word is given the words before it.

Models are trained to pick likely continuations, so averaged over a long passage their output is more predictable than prose a person wrote. Human writing wanders. It swings sentence length around, reaches for an odd word, and repeats structures less evenly. More on the mechanics in how AI detectors work.

The published error rates

Turnitin publishes two numbers: under 1% of documents, and around 4% of sentences. They also state that documents scoring below 20% have a higher incidence of false positives, which is why their own interface treats low scores more carefully than high ones.

There is one more finding from their analysis worth knowing, because it explains a pattern people notice and misread. Of the human sentences that get falsely flagged, 54% sit directly next to a sentence the model scored as AI written, and another 26% sit two sentences away.

Four in five wrong flags land in the immediate neighbourhood of a real detection. They cluster rather than scatter. Which means a mixed document, part drafted and part written, is the hardest thing you can hand it, and mixed documents are quickly becoming the normal case as drafting tools get built into ordinary writing software.

Full numbers in Turnitin's AI false positive rate.

Why another tool gives you a different number

Detectors are trained on different data, calibrated to different thresholds, and chop documents up differently. Two of them can disagree hard on the same paragraph without either being broken. The reasons are in why AI detectors disagree.

Practical version: a score means something relative to the tool that produced it and that tool's own error rate. Putting a 30% from one detector next to a 60% from another and concluding anything is a mistake.

Where the number gets least reliable

Four situations, and they degrade any detector, not just this one.

Short text. A few hundred words carries far less signal than a paper. Same reason the sentence-level error rate is four times the document one.

Mixed documents. Wrong flags cluster around real ones, as their own sentence analysis shows.

Heavily edited generated text, or heavily formalised human text. Both drift toward the middle of the distribution, which is exactly where a classifier is least sure.

Non-native English writing. More regular construction, narrower vocabulary range, both pushing toward the predictable end. Research keeps finding elevated false positive rates here. See why non-native English writing gets flagged.

The short version

The AI indicator is a probability from a classifier with a published, non-zero error rate, measured over documents rather than sentences, and less reliable below 20%. Turnitin's own documentation says it should not be the sole basis for action.

If you are staring at a number right now, what a Turnitin AI score means covers how to read it and what to do when your writing is flagged covers the rest.

How we do it differently

We build a detector too, so this is disclosure rather than a neutral review.

We calibrate backwards. Instead of picking the threshold with the best headline accuracy, we fix a false positive rate we can live with and accept whatever detection rate comes out. At our public setting that is 0.8% of genuine pre-LLM academic writing flagged, and 0.4% at a stricter one.

You can run the same text through ours free and with no account, and see whether it agrees. The full method, including where ours is weakest, is in how the unslop AI detector works.