Why Is AI Writing So Hard to Read? A New Paper Measured It

TL;DR

A paper posted to arXiv on October 21, 2026, by Thomas Hikaru Clark (NYU), Matias B. Martin and Kyle Mahowald (UT Austin), measures why AI writing is hard to read. The authors counted six sentence features that psycholinguists have tied to slower human reading, in text from 14 models across chat, coding and essay settings. All nine frontier and near-frontier models used more low-frequency word pairs and more inanimate subjects with animate verbs (“this asymmetry bites”) than human writing, and most also used longer syntactic dependencies, more stacked nouns and more object relatives without “that” in conversation. In a small RL experiment, the authors penalized each extra token and watched a 1.5B model start dropping optional words almost every time. The habit spread to constructions it was never trained on. Claude Opus 5.5 scored better than Opus 5 on 19 of 21 comparisons, so a lab can improve this between releases.

The sentence that opens the paper

The introduction of Why is AI-generated language so hard to understand? starts with a planted specimen: “Your instinct is right; the convoluted sentence you are currently reading is quietly eating its own reader-patience budget — footnote it.”

Most people who read model output all day will recognize the voice. The authors take it apart. It has an object relative clause, a noun used as a verb (“footnote it”), an inanimate subject doing something only animate things do (a sentence “eating”), and a three-noun stack (“reader-patience budget”) that produces a word sequence you have probably never seen before. Each of those is a construction that reading-time and eye-tracking studies have linked to processing cost.

That turns a vague complaint into something countable. The usual explanations for bad AI prose (“it has no lived experience”, “it overuses delve”) are about word choice or taste. This paper is about grammar, and it makes two testable claims:

• H1: frontier LLM output overuses linguistic features associated with human comprehension difficulty.
• H2: pressure for information density during post-training is what pushes the output away from human-like language.

It got 256 points and 205 comments on Hacker News within a few days, and that’s no surprise: almost everyone who uses these models has an opinion about why the text reads the way it does, and very few of those opinions have had data behind them.

Six features linked to slow reading

The paper picks features with two properties: psycholinguists have tied them to reading difficulty for decades, and each one is usually the shorter of two ways to say the same thing. That second property is what connects H1 to H2. All of them are detected from spaCy dependency parses (en_core_web_trf), with an extra classifier pass for the two semantic ones.

Every row in the right-hand column costs words. “Bites” replaces “causes an issue”. Dropping “that” saves a token. A noun stack squeezes a prepositional phrase into one compound word. The paper’s own wording is that rephrasing an inanimate-subject sentence “would potentially require lengthening the sentence”, and the speculation is that models “converged to using such constructions as efficient, albeit non-standard, ways of expressing meaning which would otherwise take up more tokens.”

How the corpus was built

The data collection is more careful than I expected for a paper that cost about $250 to run. The authors ran 14 models through one open-source agent harness (Pi, pinned to version 0.84.4) so every model saw the same scaffold and prompt text instead of each vendor’s own app:

• Claude: Sonnet 5, Opus 5, Opus 5.5 and Fable 5
• OpenAI: GPT-5.6 Luna, Terra and Sol
• Open-weight near-frontier: GLM-5 and Kimi K2.5
• Small open models as controls: Llama 3.1 8B (base and Instruct) and Olmo 3 7B (base, Instruct and Think)

The nine large models ran on Amazon Bedrock at provider defaults with medium reasoning; the small ones ran locally on an Apple M4 Max with mlx-lm.

There were three settings. In chat, 12 of the models (the nine frontier ones plus Olmo Think, Olmo Instruct and Llama Instruct) each held ten conversations of up to 12 turns (learning guitar at 40, renting vs buying, finding a briefcase of cash) against a simulated user played by Claude Sonnet 4.6, for 1,428 replies in total. In code, the nine frontier models each did a bug sweep on five small MIT-licensed Python repos inside a sandbox, with the same simulated user acting as the repo owner, for 45 sessions. Only the text the model wrote back to the user counts in the coding setting; the code and tool calls are excluded. In essays, each model wrote ten essays for each of seven PERSUADE 2.0 assignments (980 essays), compared against 2,934 student essays that scored 5 or 6 on the corpus’s 1-6 scale.

Human baselines are genre-matched: OpenSubtitles for chat, StackExchange for coding, student essays for essays, and Wikipedia as an extra reference. The n-gram measures use a KenLM trigram model built on each reference corpus.

Round-trip translation as a control

Comparing a chatbot to movie subtitles has an obvious problem: the content differs as well as the grammar. The paper’s fix is round-trip translation. Every model sentence goes English → Japanese → English through Meta’s NLLB-200 (3.3B), which keeps the meaning roughly fixed while replacing the wording with more frequent forms. If a feature becomes rarer after the round trip, the model picked it over an available plainer way of saying the same thing. Japanese was chosen because pivots unrelated to English produce more varied paraphrases, and the authors repeated the check with a Russian pivot and with Google’s TranslateGemma.

What the 14 models did

The paper reports its results almost entirely as plots (Figures 2-6), so this section gives the direction of each effect and leaves exact values to the figures.

Frontier models overuse the difficult constructions relative to their own paraphrases. Across chat, code and essays, and for almost every frontier and near-frontier model, the text had fewer unattested bigrams and lower n-gram perplexity after the round trip than before it. Denominal verbs, inanimate subjects and object relatives without “that” followed the same pattern. Models under-used object relatives with “that”, which the authors read as a consistent preference for skipping the optional word. Stacked nouns and long dependencies followed suit in chat and coding.

Against human writing, the abstract’s summary is the cleanest statement: all nine frontier and near-frontier models use more inanimate subjects with animate verbs and more low-frequency n-grams than humans, and most also use longer dependencies, more stacked nouns and more reduced object relatives in conversation.

Essays show the effect least. Several measures moved less under round-trip translation for PERSUADE essays than for chat and coding. The authors’ guess is that essays are one prompt and one reply, while the patterns build up over long back-and-forth sessions. Anyone who has read the 40th turn of a coding-agent session will find that plausible.

Post-training is where it appears. The small controls are the most informative part of the paper. Llama 3.1 and the Olmo base model barely change under round-trip translation and sit closest to the student essays. Within the Olmo family the effect grows from base to Instruct to Think, and Olmo Think “shows effects as large as those of the frontier models”. The Olmo variants share their pre-training and differ in post-training, and their grammar diverges along the same line. That matches Reinhart et al. in PNAS (2025), which found that instruction-tuned Llama 3 models differ from human writing more than their base models do.

The effect also varies by model. Claude Opus 5.5 scored lower than its predecessor Opus 5 on nearly every difficulty-linked measure in every setting: 19 of 21 comparisons, with non-overlapping 95% intervals in eight. Opus 5.5 was collected on September 24, two weeks after the main run, under the same protocol. Whatever Anthropic changed between those releases, it moved the grammar in a measurable direction. If you formed opinions about Fable 5’s or Sonnet 5’s prose from daily use, this dataset puts the Claude family side by side on features linked to readability.

A caveat the authors state themselves: the paper measures features that correlate with processing difficulty. No human reading-time or comprehension study was run on this corpus. “Harder to read” is inferred from earlier psycholinguistics; this corpus has no reading measurements.

The length-penalty experiment

H1 says the features are there. H2 needs a mechanism, and the authors can’t post-train a frontier model, so they built a small version of the pressure they suspect.

The setup: Qwen2.5-1.5B-Instruct learns to turn JSON event descriptions into one English sentence, e.g. {"recipient": "the kids", "event": "mail", "agent": "he", "theme": "a book"}. Training uses GRPO (via TRL’s GRPOTrainer, LoRA rank 16) with a reward of correctness times format minus λ per token. Every training item can be phrased two ways that differ by a known number of tokens: dative (“gave his wife a gift” vs “gave a gift to his wife”), object relative with or without “that”, complement clause with or without “that”, and a particle-order control where both orders are the same length. The grid is λ ∈ {0, 0.01, 0.03, 0.1}, KL coefficient β ∈ {0, 0.04}, three seeds each: 24 runs, 31 GPU-hours on L40S cards.

Results, from Section 4.2 and Appendix C:

• The short variant’s share rises with λ, with and without the KL penalty. KL slows the drift but doesn’t stop it.
• Without KL, seeds at λ = 0 landed anywhere from 0% to 100% short, so there was no stable preference. At any λ > 0, every seed converged to a categorical preference for the short form: over 98% short for both trained “that” alternations.
• The preference transferred to “near” constructions never seen in training (“sure (that)”, “so (that)”).
• “Far” transfer was weak: of four unrelated alternations, only the genitive (“the old house’s roof” vs “the roof of the old house”) shifted, and only without KL. Passives and purpose clauses didn’t move.

The model was never instructed to compress; shorter correct answers scored slightly higher, and it found the syntactic alternations on its own. Human speakers pick between “the book the professor wrote” and “the book that the professor wrote” depending on context, and include “that” more often when the next words are less predictable. The trained model dropped the optional word regardless of context.

Running the math myself

The paper’s Appendix C gives the closed form that motivates the λ and β grid. If two answers tie on task reward and differ by Δ tokens, the KL-regularized optimum satisfies:

log π*(short)/π*(long) = log π_ref(short)/π_ref(long) + λΔ/β

In plain terms, the length penalty multiplies the base model’s odds of the short form by exp(λΔ/β). Here’s that formula in a few lines of stdlib Python, using the paper’s β = 0.04 and its λ grid:

import math

# Eq. (2) of Clark, Martin & Mahowald (arXiv:2610.29162):
#   log pi*(s)/pi*(l) = log pi_ref(s)/pi_ref(l) + lambda * delta / beta
# s = short variant, l = long variant, delta = token difference.

def short_share(p_ref_short: float, lam: float, beta: float, delta: int = 1) -> float:
    """Share of the short variant at the KL-regularized optimum."""
    log_odds = math.log(p_ref_short / (1 - p_ref_short)) + lam * delta / beta
    return 1 / (1 + math.exp(-log_odds))

BETA = 0.04                      # the paper's KL coefficient
LAMBDAS = [0, 0.01, 0.03, 0.1]   # the paper's length-penalty grid

print(f"beta = {BETA}, delta = 1 token (e.g. dropping an optional 'that')")
print(f"{'base share':>10} | " + " | ".join(f"lam={l:<5}" for l in LAMBDAS))
for p in (0.2, 0.4, 0.6, 0.8):
    row = " | ".join(f"{short_share(p, l, BETA):>9.1%}" for l in LAMBDAS)
    print(f"{p:>10.0%} | {row}")

print("\nodds multiplier exp(lam*delta/beta):",
      ", ".join(f"lam={l}: x{math.exp(l / BETA):.2f}" for l in LAMBDAS))
print("passive -> active (delta = 2), base 20%, lam=0.1:",
      f"{short_share(0.2, 0.1, BETA, delta=2):.1%}")

Running it prints:

beta = 0.04, delta = 1 token (e.g. dropping an optional 'that')
base share | lam=0     | lam=0.01  | lam=0.03  | lam=0.1  
       20% |     20.0% |     24.3% |     34.6% |     75.3%
       40% |     40.0% |     46.1% |     58.5% |     89.0%
       60% |     60.0% |     65.8% |     76.1% |     94.8%
       80% |     80.0% |     83.7% |     89.4% |     98.0%

odds multiplier exp(lam*delta/beta): lam=0: x1.00, lam=0.01: x1.28, lam=0.03: x2.12, lam=0.1: x12.18
passive -> active (delta = 2), base 20%, lam=0.1: 97.4%

The base shares in the left column are illustrative inputs I picked; the paper doesn’t report Qwen’s base rates in this form. The multipliers carry the result: at λ = 0.1 the odds of the shorter phrasing go up 12x per token saved, so a construction the base model used one time in five becomes the default. With β = 0 the formula has no finite answer at all, because without a KL term the policy has no tie to the base model, and the paper notes the shift is then “limited only by the length of training”, which is what the >98% result looks like.

A penalty of 0.1 per token is large against a reward that’s 0 or 1: one saved token is worth a tenth of a correct answer.

What I measured: rare word pairs, in stdlib Python

The paper’s simplest measure is the unattested-pair rate: the share of adjacent word pairs in a text that never occur in a reference corpus, per 1,000 pairs. The authors score essays against Wikipedia, so I rebuilt a rough version of that with no dependencies. A short fetcher pulled the plain text of 120 Wikipedia featured articles through the MediaWiki API (503,331 words). The first 100 became the reference and the last 20 a held-out human sample. samples.py holds the model sentences the paper quotes in its Table 2, copied verbatim, plus the ten opening user messages from Table 5a, which the authors wrote in a casual human register.

"""Unattested word-pair rate, after the n-gram measure in arXiv:2610.29162 (B.2).

A pair is "unattested" if two adjacent words never appear side by side
anywhere in the reference corpus. Reported per 1,000 pairs.
"""
import json, re, sys

sys.path.insert(0, "tmp/aiprose")
from samples import AI_CHAT, AI_ESSAY, HUMAN_CHAT

WORD = re.compile(r"[a-z0-9]+(?:'[a-z]+)?")
SPLIT = re.compile(r"(?<=[.!?;:])\s+|\n+")


def pairs(text):
    for chunk in SPLIT.split(text.lower().replace("’", "'")):
        words = WORD.findall(chunk)
        yield from zip(words, words[1:])


def rate(texts, seen):
    total = miss = 0
    for t in texts:
        for p in pairs(t):
            total += 1
            miss += p not in seen
    return miss, total, 1000 * miss / total


articles = json.load(open(sys.argv[1], encoding="utf-8"))
ref, held_out = articles[:100], articles[100:]
seen = {p for a in ref for p in pairs(a["text"])}
ref_words = sum(len(a["text"].split()) for a in ref)
print(f"reference: {len(ref)} Wikipedia featured articles, {ref_words:,} words, "
      f"{len(seen):,} distinct pairs\n")

rows = [
    ("Wikipedia, held-out articles", [a["text"] for a in held_out]),
    ("Model essay sentences (Table 2)", AI_ESSAY),
    ("Model chat sentences (Table 2)", AI_CHAT),
    ("Human chat openers (Table 5a)", HUMAN_CHAT),
]
print(f"{'sample':<34}{'pairs':>8}{'unattested':>12}{'per 1k':>9}")
for name, texts in rows:
    miss, total, r = rate(texts, seen)
    print(f"{name:<34}{total:>8,}{miss:>12,}{r:>9.0f}")

print("\nheld-out Wikipedia rate vs reference size:")
for n in (25, 50, 100):
    small = {p for a in ref[:n] for p in pairs(a["text"])}
    words = sum(len(a["text"].split()) for a in ref[:n])
    print(f"  {n:>3} articles ({words:>7,} words): "
          f"{rate([a['text'] for a in held_out], small)[2]:.0f} per 1k")

Output from the run:

reference: 100 Wikipedia featured articles, 427,330 words, 175,446 distinct pairs

sample                               pairs  unattested   per 1k
Wikipedia, held-out articles        72,727      30,725      422
Model essay sentences (Table 2)        205         139      678
Model chat sentences (Table 2)         183         133      727
Human chat openers (Table 5a)          310         193      623

held-out Wikipedia rate vs reference size:
   25 articles (110,758 words): 653 per 1k
   50 articles (229,440 words): 587 per 1k
  100 articles (427,330 words): 422 per 1k

Treat the middle rows as an illustration. The model sentences were picked by the authors to illustrate difficult features, so they’re a worst case, and 200 word pairs is a tiny sample. What the run does show is why the paper’s design looks the way it does.

First, the absolute number means little on its own. With a 427,000-word reference, 42% of word pairs in human-written Wikipedia text were never seen, and shrinking the reference to 25 articles pushes that to 65%. Rare word pairs are everywhere in real English; the measure only works as a comparison against a fixed reference.

Second, genre dominates the result. The casual chat openers (“ok random question for you. if you found a briefcase…”) score 623, much closer to the model sentences than to held-out Wikipedia, because casual chat is far from encyclopedia prose. That’s why the authors score chat against OpenSubtitles and code replies against StackExchange, and why the round-trip comparison carries most of the paper’s weight. It compares each model’s sentences against a paraphrase of the same sentences, so genre and topic cancel out.

What this means if you build with LLMs

Three practical consequences, in rough order of likely impact.

Your “be concise” prompt is a small length penalty. This is my extrapolation, not a claim the paper tests: the experiment applies the pressure during training, not through a prompt. But the mechanism carries over by analogy. A system prompt that says “be brief” or a hard token budget rewards the same token-saving alternations: drop the “that”, stack the nouns, let the docstring “promise” things. Plenty of advice about cutting costs (including the efficient-reasoning papers covered here before) is about shrinking the hidden reasoning trace, which nobody needs to read. Shrinking the answer the human reads is a different trade, and this paper is a reason to keep the two budgets separate.

Agent summaries are where the effect is strongest. The chat and coding settings showed bigger shifts than essays, and the coding examples in Table 2 read like every pull-request summary you’ve skimmed lately: “Prevented Git dry runs from installing hooks or staging changes”, “Fixed absolute dotfile backup path handling while preserving the : marker used during reinstall.” Those sentences are fine for a log and slow for a reviewer. If your team approves pull-request descriptions that agents wrote, the reviewer is the bottleneck, and context engineering for agents should cover the shape of the text that they hand back.

You can prompt against specific constructions. The six features are concrete enough to write rules for. Here’s a style block that targets each one; it’s my translation of the paper’s features into instructions, untested against the paper’s metrics:

When writing prose for a human reader (not code, not logs):
- Keep "that" in relative and complement clauses: "the bug that the test caught".
- Give verbs a human or system actor that can actually do the action.
  Write "the function returns False", not "is_file() swallows that".
- Never stack three or more nouns. Break them up with "of", "for" or a clause.
- Don't turn nouns into verbs unless the verb is standard English (email, test).
- Put the subject close to its verb. Split sentences over 25 words.
- Prefer the common phrasing over the vivid one.

Rules like these cost tokens, which is exactly the trade the paper describes. Whether a model holds to them across a 12-turn session is an open question; the paper suggests drift accumulates over long conversations, and nothing here tests a prompt-level fix.

For teams that post-train models, the paper points at two levers. KL regularization measurably slowed the drift in Experiment 2. And the paper’s related-work section points to UID-based preference optimization (Li and Xu, at the third Conference on Language Modeling), which uses uniform information density as a training signal for more human-aligned text. The Opus 5 → 5.5 change shows a lab can move these numbers between releases, whatever method it used.

Where the paper is thin

I think H1 is well supported and H2 is a plausible proof of concept that hasn’t been shown to explain frontier models. The gaps, mostly stated by the authors in their Limitations section:

• No human reading data. The features predict difficulty from prior studies; nobody’s eye movements were tracked on this corpus.
• Small seed set. Ten chat topics and five repos, English only, one simulated user (itself a Claude model, which could nudge every conversation’s register).
• Indirect causal link. Experiment 2 uses a 1.5B model on a toy JSON-to-sentence task. Frontier post-training mixes RLHF, RLVR, long contexts and objectives nobody outside the labs can see. The paper argues the compression pressure fits verifiable-reward training better than RLHF, because human-feedback training has been found to reward longer outputs, but the paper offers no measurement of it.
• “Information density” is a contested label. On Hacker News, commenters offered other labels: “high entropy”, or “data dense, information sparse”. The paper itself contrasts its density pressure with uniform information density, the human tendency to spread information evenly, so “dense” here means fewer tokens per idea and says nothing about how useful the idea is.

One popular intuition from the thread also deserves a correction. One comment said: “They’re picking among most likely tokens, which are ones with least entropy and so least information per token.” The paper finds the opposite against human reference text: model output has more unattested word pairs and higher trigram perplexity. A token can be the model’s top choice and still form a pair that rarely occurs in human writing. A token can be probable for the model and improbable in human text, and post-training widens that gap.

FAQ

Why is AI writing so hard to read?

According to this paper, frontier models overuse six grammatical features linked to human processing difficulty: rare word pairs, long dependencies, object relatives without “that”, stacked nouns, nouns used as verbs, and inanimate subjects doing animate things. Each is usually the shorter way to say something, and the authors argue post-training pressure toward fewer tokens selects for them.

Why is AI writing so easy to spot?

Word lists (“delve”, “tapestry”) were the old tell, and Juzek and Ward found results consistent with RLHF playing a role in that overuse. This paper adds grammatical tells that are harder to scrub with find-and-replace: “the docstring promises”, “the papercut people feel”, three-noun compounds. All nine frontier models studied used more inanimate subjects with animate verbs than human writers, though at different rates.

Why does AI writing sound the same across models?

Partly because the same training pressures apply across labs. The paper shows the effect growing with post-training stage inside one model family (Olmo base → Instruct → Think) while the small base models stay closer to human text. Model families still differ, and Claude Opus 5.5 scored better than Opus 5 on 19 of 21 comparisons.

Does telling an LLM to be concise make its writing harder to read?

The paper didn’t test prompts. It tested a training-time length penalty, which pushed a small model to drop optional words in over 98% of trained cases without KL regularization. A concise-style prompt rewards the same kind of shortcut, so it’s reasonable to expect a milder version of the effect. That’s an inference; the paper reports no result on prompts.

Which AI model writes the most readable text?

Among the nine frontier models, the paper doesn’t publish a single ranking, and its figures show variation by setting. The most readable outputs on these measures came from the small Llama 3.1 and Olmo base and Instruct models, which are the least capable models in the study. Among Claude models, Opus 5.5 improved on Opus 5.

Sources

• Why is AI-generated language so hard to understand? (arXiv:2610.29162): Clark, Martin and Mahowald, October 2026; the paper this post explains (CC BY 4.0)
• Hacker News discussion: 256 points, 205 comments
• Do LLMs write like humans? Variation in grammatical and rhetorical styles: Reinhart et al., PNAS 2025, on how instruction tuning moves models away from human grammatical style
• Pi agent toolkit: the open-source harness the authors used to run every model under one scaffold
• Why does ChatGPT “delve” so much?: Juzek and Ward, COLING 2025, on word overuse and RLHF
• A long way to go: Investigating length correlations in RLHF: Singhal et al., COLM 2024
• PERSUADE 2.0 corpus: Crossley et al., Assessing Writing 2024, the student-essay baseline
• Wikipedia featured articles: the reference corpus for the bigram demo above, fetched through the MediaWiki API

Bottom line

The paper turns “AI writing feels off” into six countable features and shows that frontier models lean on all of them, most heavily in chat and coding sessions. The proposed cause is token economics: post-training rewards correct answers in fewer tokens, and English offers plenty of grammatical shortcuts that cost the reader instead of the model. The Opus 5 → 5.5 improvement is the hopeful part. Labs can now measure the problem, which is the first step toward fixing it.