Why AI detectors disagree
Run one paragraph through four tools and you can get 12%, 40%, 61% and 80%. None of them are broken. There are five specific reasons.
Try it sometime. Take one paragraph, run it through four detectors, and watch the numbers come back nowhere near each other. People treat this as proof that detection is nonsense. It is not, though it does tell you something important about how much weight one score deserves.
Here is where the spread comes from.
1. They drew the line in different places
Every detector turns a continuous score into a verdict using a threshold, and where that goes is a business decision, not a mathematical one.
A tool optimising for headline accuracy puts it where the average looks best. A tool optimising against false positives puts it much higher and flags less of everything. Two detectors could compute an identical internal score for your paragraph and report different verdicts purely because of where each drew its line.
We pick ours by fixing the false positive rate first: the threshold is the score only 1% of known human documents exceed. That makes us flag less than a tool tuned for average accuracy, which shows up as disagreement on anything borderline.
2. They learned from different examples
A detector learns what generated text looks like from whatever it was shown. What it was shown decides what it recognises.
One trained mostly on 2023 ChatGPT output has learned that model's habits. Hand it text from a newer family, or a different lab, and it has much less to go on. Detection rates move by tens of points as a result, and almost no tool will tell you what it was trained on.
Ours is trained across 14,892 generated documents from seven model families and five years of GPT versions, which is why it holds 89.9% on OpenAI GPT output at 99.6% specificity.
3. They cut the document up differently
Some score the whole thing at once. Some split into passages and aggregate. Some report per sentence.
That alone produces different numbers from identical text. A document with one generated section and four human ones might score low as a whole, high on that section, and land somewhere in between if the tool aggregates. None of those is wrong. They are answers to different questions.
It also explains why sentence-level figures look so much worse than document-level ones. Less text, less signal. Every detector is less reliable per sentence, and Turnitin's published numbers put their own gap at roughly four times.
4. They weigh the signals differently
Detection rests on predictability, rhythm and vocabulary, described in how AI detectors work. Tools weight those very differently.
A perplexity-heavy detector reacts strongly to word choice, so paraphrasing moves it. A stylometry-heavy detector reacts to sentence rhythm, so paraphrasing barely touches it while restructuring does. Hand both the same paraphrased paragraph and they will disagree, in a direction you could predict if you knew what was under the hood.
5. Sometimes the text really is borderline
Some writing sits near the boundary for real. Formal, regular, template-shaped prose is statistically similar to generated text whoever produced it, and near a boundary, small differences in method produce large differences in output.
Which makes disagreement informative rather than embarrassing. Wide disagreement usually means the text is genuinely ambiguous, not that one tool is defective.
What to do with four different numbers
Do not average them. They are not measurements of the same quantity on the same scale. Averaging them produces a number that means nothing at all.
Ask what each tool's false positive rate is, and whether it publishes one. A score from a tool that will not say how often it is wrong about human writing is hard to weigh against anything.
Prefer per-passage output. A document percentage hides where the signal is. Knowing one paragraph carries it beats knowing the document averaged 30%.
Treat the disagreement as the finding. If four tools cannot agree, the honest conclusion is that the text is ambiguous, and that is worth knowing before anyone acts on any single one of those numbers.
Ours is free and unlimited, publishes its false positive rate, and scores each passage separately, which makes it useful as a second opinion whatever produced the first.