Over 30% of papers submitted on arXiv read as AI-written

Motivation

I had a talk with a friend of mine who was reviewing papers for a journal, and he told me that he was worried about the future of academia. He told me that most of the papers they are receiving are clearly AI-written, and some are actually just slop all the way down, with no real experiment behind them. I'm well aware that AI slop has penetrated every text-based medium; however, I was still surprised and worried about its prevalence in academia.

I did a small experiment: I opened ZeroGPT and pasted in a few recent papers. Most of them got flagged all the way through as AI-written. Then, however, I pasted in some really old papers, which also got flagged, and realised that you can't really use ZeroGPT to make any solid assertions about the current state of AI-written papers.

Unlike what a reasonable person with a realistic amount of free time would do, I decided to build my own detector, which allowed me to know and control the false-positive rate specifically for academic writing. I would never claim it's perfect, but it's much better than ZeroGPT for this specific task. This post is not about the detector, though, it's about the actual experiment that I ran. If you want to learn more about the detector, you can read up on it here.

What I measured

I sampled 25 papers per field per month across ten field groups from January 2023 to July 2026, plus eight control months across 2021 and 2022, for a total of 12,750 papers. For each one, I pulled the version 1 PDF specifically so that a paper that was revised in 2026 would not leak modern text back into its 2023 datapoint.

I then used my Unslop-MoE-v7 detector to label all the papers. The results are likely an understatement of the actual share of AI-written papers because the detector misses roughly 20% of them. This is because I specifically optimised it for a low false-positive rate. The trade-off is that I miss some AI-written papers. In this case, that does not matter much, though, because I can be reasonably sure that the positives are actual positives, so the statements I make are more reliable than they would be with a normal AI-text detector.

Results

The release of ChatGPT marks a clear inflection point in the share of AI-written papers. Starting from the end of 2022/beginning of 2023, the share steadily climbs into the 30–40% range, peaking near 39% in early 2026. There is a large spread across fields, as shown in the table below.

Computer science leads the way with about 65%, while mathematics is lowest, at around 0.7%. Again, this does not mean mathematics papers are not written with LLMs. I suspect that because maths papers are more dense than computer science papers, it is much more difficult to reliably identify them as machine-written. The control column is each field's average flag rate from 2021 to 2022; the fields that rise most are not the ones with the highest pre-LLM control level, so an elevated starting point does not explain the rise.

Limitations

Control sample size

Each field has 200 papers in the pre-ChatGPT control. At a false-positive rate of 0.4%, only eight papers are flagged across the entire 2,000-paper control. Once those eight papers are spread across ten fields, the individual control rates become quite noisy.

The overall 0.4% rate is well estimated and is what the experiment is anchored to. The control rate for each individual field should only be treated as an approximation, though. Fixing this would require thousands of pre-ChatGPT papers per field, which some arXiv fields simply do not have.

A low score does not necessarily mean low AI use

Mathematics is the clearest example. Maths papers contain much more notation and theorem-proof structure than most computer science papers. Once the equations and references are removed, there is often not much normal prose left for the detector to work with.

This means a maths paper could have been written with heavy LLM assistance and still receive a low score. The low number for mathematics could therefore mean that mathematicians use LLMs less, that the detector is worse at identifying AI writing in maths papers, or some combination of both. This experiment cannot reliably separate those explanations.

The detector works best on prose-heavy academic writing, and those are also the fields with the largest increases. I therefore do not think this explains the overall trend, but the results for the low-scoring fields should be interpreted more carefully.

The detector does not catch every model

The detector is better at identifying text from some models than from others. I also cannot test it against the exact mixture of models, prompts and editing methods that real authors use.

Because the detector misses some AI-written papers, the results are likely an understatement of the actual share. I cannot say exactly how large that understatement is. The detector write-up contains the results for each generator I tested.

A flag is not proof of authorship

The detector measures whether a paper reads as machine-written. It cannot tell whether the entire paper was generated, whether an LLM was only used for editing, or whether only a few sections were heavily rewritten.

A single score should therefore never be used to accuse a specific author of anything. This experiment is about the overall prevalence of machine-like writing on arXiv, not about identifying individual papers or people.

The data

For transparency's sake, here are all the paper IDs and their labels: download the full list (CSV). A paper being flagged does not constitute an accusation. Any individual paper may or may not be AI-written.

Each row is one paper: its arXiv ID, field, first-version month, the detector's score, and whether it was flagged.

Try the detector

You can run the detector completely for free, as many times as you want. You can also scan arXiv papers by link or paper ID here.