GPT-5.4 vs Claude Opus 4.7 vs Gemini 3.1 Pro for Coding (May 2026)

TL;DR

Put the benchmarks, the prices and the published head-to-head tests side by side, and the picture in May 2026 is messier than the leaderboards suggest. Claude Opus 4.7 wins on raw SWE-bench Verified (83.5% vs 76.9% for GPT-5.4) and is the stronger pick when a refactor crosses three or more files. GPT models win agentic loops on cost: Mindstudio’s testing found GPT-5.5 emits ~72% fewer output tokens than Opus 4.7 on identical loops, and GPT-5.4 shows the same pattern in milder form, which compounds across iteration. Gemini 3.1 Pro is at $4.50 blended per million tokens, and on well-specified problems it lands within a couple of points of the others. There’s no single winner — just a routing decision.

Why the comparison changed in April 2026

Three things shifted the ground:

1. Anthropic shipped Claude Opus 4.7 on April 16. The Adaptive variant pushed the upper bound of SWE-bench Verified to 83.5% (±1.7) on Scale AI’s leaderboard, with Mythos Preview reportedly hitting 93.9% though that one isn’t widely available yet.
2. OpenAI quietly stopped reporting SWE-bench Verified scores because their own audit found that frontier models could reproduce verbatim gold patches from the test set. Opus 4.5 specifically dropped from 80.9% on Verified to 45.9% on SWE-bench Pro — a 35-point gap on the same model. Across other models the Verified→Pro delta tends to land in the 20–35 point range. The benchmark we’ve been quoting for two years is leaking.
3. Gemini 3.1 Pro held its position as the cost leader — half the price of Opus 4.7, an Intelligence Index of 57 (the same as Opus 4.7 and GPT-5.4), and a 1M-token context window that it actually uses well.

The dollar/token math has flipped. Pre-April, “use the best model” meant Claude. Post-April, it means “decide whether you’re paying for raw quality, agentic efficiency, or context-window scale”, which gives three different answers.

Whether you use them inside Cursor or through direct API calls, here’s what’s actually different in practice:

The benchmark scoreboard

These are the public numbers I trust most as of May 1, 2026. Scale AI’s leaderboard and Artificial Analysis are the two I cross-check first because they publish error bars and methodology notes.

Two things to note before drawing conclusions. First, the SWE-bench Verified numbers are reported by the labs themselves with their own scaffolding. Anthropic reports Opus 4.7 with the “Adaptive” runner, OpenAI reports GPT-5.4 at “high” reasoning, Google reports Gemini at the “Preview” tier. They are not running on identical harnesses. Second, the AA Intelligence Index is averaged across 10 benchmarks; tied scores hide huge per-task gaps that show up when you’re choosing for a specific job.

SWE-bench: what a 7-point gap means in practice

Claude Opus 4.7 is 6.6 points ahead of GPT-5.4 on SWE-bench Verified. That sounds modest until you remember the benchmark is “fix this real GitHub issue,” and the gap means Claude resolves roughly 1 in 13 issues that GPT-5.4 doesn’t. On a backlog of 100 bugs that’s eight extra fixes. On a small CI loop that’s the difference between a green run and a stuck queue.

Where the gap tends to show in normal use:

• Multi-file refactors — anything that touches a model, a service, and a controller in the same change. Opus 4.7 is willing to read all three files before editing. GPT-5.4 starts editing earlier and often produces a structurally clean change that compiles but breaks because it inferred the contract instead of reading it.
• Long error chains — a 200-line stack trace through an async pipeline. Claude follows it. GPT-5.4 jumps to the most obvious frame and tries a fix.
• Unfamiliar code — first-touch on a new repo. Claude reads with patience; GPT-5.4 reads enough to act, which is faster but riskier.

The catch: SWE-bench Verified is contaminated. OpenAI’s audit found models could reproduce verbatim patches, and the same Opus 4.5 that scored 80.9% on Verified scored only 45.9% on SWE-bench Pro, a harder, less-leaked benchmark. So the 83.5% on Opus 4.7 is real but possibly inflated; the relative gap to GPT-5.4 is what matters when you’re picking a tool. (For a striking example of how benchmark methodology can change results, see how THINC’s 4B model beat Qwen3-235B by switching the reasoning medium from English to code.)

Token efficiency: the GPT advantage

Reported numbers from Mindstudio’s testing: on identical agentic loops, GPT-5.5 emits roughly 72% fewer output tokens than Opus 4.7 to reach the same answer. GPT-5.4 shows the same direction in milder form. Claude tends to think out loud, with long planning sections, restated assumptions, and “let me first read the file” preambles. GPT cuts straight to the diff.

For a one-shot prompt this barely shows up. For an agentic CLI session that loops 50 times, it’s the entire cost structure. Here’s the arithmetic on one agentic refactor, at list prices (Opus 4.7 at $5/$25 per million tokens, GPT-5.4 at $2.50/$15):

# Worked example at list prices. Token counts are illustrative, not measured.
opus = {"in": 382_400, "out": 88_200, "price_in": 5.00, "price_out": 25.00}
gpt54 = {"in": 341_800, "out": 22_000, "price_in": 2.50, "price_out": 15.00}  # ~1/4 of Opus's output

for name, m in (("Opus 4.7", opus), ("GPT-5.4", gpt54)):
    cost = m["in"] / 1e6 * m["price_in"] + m["out"] / 1e6 * m["price_out"]
    print(f"{name}: ${cost:.2f}")
# Opus 4.7: $4.12
# GPT-5.4: $1.18

Same task, similar input, but a model that writes a quarter of the output at lower rates costs about a third as much per run. If you live in agentic loops, GPT-5.4 is cheaper per finished task even though it loses on raw benchmark quality.

The routing rule, in one line: Claude per query, GPT per loop.

Where Gemini 3.1 Pro fits

Gemini 3.1 Pro is the easiest of the three to underestimate. At $4.50 blended per million tokens, against $5.63 for GPT-5.4 and $10 for Opus 4.7, it’s the cost leader, and it earns a place under three conditions:

• The problem is well-specified. Gemini reads ambiguity as room to interpret, which goes wrong on vague tickets but is fine when the spec is clear.
• The codebase is large. Gemini’s 1M-token context is the same size on paper as Opus 4.7’s, but my read is that Gemini retrieves better from deep in the window. That’s exactly what long-file bugs need.
• You don’t need maximum reliability per call. On agentic chains it’s the most likely of the three to take the wrong fork at step three. For one-shot work it’s fine.

Two jobs suit it best: searching across a whole monorepo for “where does this constant ultimately get used,” and writing the first draft of test fixtures. Both lean more on retrieval than reasoning, and the cost compounds because they’re high-volume. For a hands-on walkthrough of actually using Gemini from the terminal, see our Gemini CLI tutorial. (Note: Google deprecated Gemini CLI in May 2026 and replaced it with Antigravity CLI, which is closed-source with tighter rate limits.)

Cost in real numbers

For a developer doing meaningful work (call it 30M input + 10M output tokens per month), the monthly bill across the three models, before any prompt caching:

Those numbers are list prices ($5/$25 for Opus 4.7, $2.50/$15 for GPT-5.4, $2/$12 for Gemini 3.1 Pro up to 200K-token prompts) applied to that 3:1 input:output mix. Real usage diverges. Claude burns more output tokens, so the gap to GPT-5.4 in practice is wider than the table suggests. Prompt caching narrows it back: with a stable system prompt, cached input on Claude bills at a fraction of the normal rate.

For comparison, the per-seat IDE pricing I covered in Cursor vs Copilot’s real cost ($10/mo for GitHub Copilot Pro, $20/mo for Cursor Pro) looks cheap until you hit the API-overage wall on a busy week. The per-IDE picture is the topic of my OpenCode vs Claude Code vs Cursor piece — and most heavy users mix: an IDE seat for completion + chat, plus a direct API budget for agentic work.

Per-task winners: a routing table

If you need a single default and refuse to route, pick Claude Opus 4.7 for one-shot work and GPT-5.4 if you live in agentic chains. Gemini is the right second model: the one you reach for when you’ve burned the budget on the others.

Five tasks worth testing yourself

Benchmarks don’t cover your codebase, so the most useful comparison is a short one you run on your own tasks. Five that separate these models well:

1. A port to a new idiom. For example, translating a Python helper to Go’s iter package (see the Go iterators guide). Watch for idiomatic naming, not just correctness.
2. A Pandas-to-Polars migration. The Polars vs Pandas piece lists the lazy-execution gotchas; check whether each model remembers the final .collect().
3. A bug reproduction from a long stack trace. This separates models that follow the whole chain from ones that jump to the most obvious frame.
4. An MCP server prototype, the same kind of work as the FastMCP tutorial. Compare how much scaffolding each one writes.
5. Finding every use of a constant in a very large file. This one tests long-context retrieval more than reasoning.

Run each a few times, since single runs vary a lot. It won’t be a controlled experiment, but it answers the question that matters (“which one should I keep paying for next month”) on your own code.

What about GPT-5.5 and Claude Mythos

Two newer models sit at the edge of the comparison:

• GPT-5.5 — released late April 2026 and now generally available on the OpenAI API. The introduction post puts it ahead of GPT-5.4 on Terminal-Bench 2.0 (82.7%) and SWE-Bench Pro (58.6%). Cursor’s default routing hadn’t fully shifted to it at the time of writing, and it needs more independent agentic testing before it earns trust. If the headline numbers hold, GPT-5.5 will be the new default for the agentic-loop slot below.
• Claude Mythos Preview — Anthropic’s restricted release, capped at roughly 50 enterprise partners (12 founding companies plus ~40 critical-infrastructure organisations). Hits 93.9% on SWE-bench Verified per the leaderboard. Until it’s broadly available, it’s not a real option for individual developers.

Both will move the curve, but neither changes the routing logic above for the rest of May 2026.

Honest limitations of this comparison

Three things to weigh before treating this as gospel:

• No two harnesses are equal. Lab-reported SWE-bench numbers use different scaffolding. The 83.5% vs 76.9% gap might shrink to 4 points or grow to 10 on identical infrastructure.
• Benchmark contamination is real. SWE-bench Verified is leaking. The relative ordering survives on SWE-bench Pro, but absolute numbers are inflated across the board.
• A routing table is not a law. The one above is a starting point. Yours will differ, most obviously if you do mostly frontend, mobile, or non-Python/Go work, where the benchmark coverage is thinner.

If you want to dig into the raw data yourself, llm-stats.com’s leaderboard and Scale AI’s SWE-Bench Pro are the two I’d start with. Both publish raw scores, error bars, and harness notes, which lets you cross-check the lab claims.

FAQ

Is Claude better than GPT for coding?

Yes on per-query benchmark quality and multi-file reasoning: Claude Opus 4.7 leads GPT-5.4 by about 7 points on SWE-bench Verified. No on agentic-loop cost-per-task: GPT models output far fewer tokens per finished task (Mindstudio measured ~72% fewer for GPT-5.5 vs Opus 4.7 on identical loops; GPT-5.4 lands in a similar but milder pattern). The right answer is “depends what you’re paying for.”

Which AI model is best for coding in 2026?

For one-shot accuracy: Claude Opus 4.7. For agentic CLI work and autonomous coding: GPT-5.4. For cost-per-token at acceptable quality: Gemini 3.1 Pro. The “best” model depends on whether your bottleneck is query quality, iteration cost, or budget.

What’s the difference between GPT-5.4 and Claude Opus 4.7?

Opus 4.7 is more thorough and reads more of the codebase before editing; GPT-5.4 acts faster with fewer output tokens. Opus 4.7 costs $10 blended per million tokens (3:1 input:output at list prices) versus $5.63 for GPT-5.4. Same Intelligence Index of 57, but Opus 4.7 is ahead on coding-specific benchmarks and AIME math.

Is Gemini 3.1 Pro good for coding?

Yes, with caveats. It’s at $4.50 blended per million tokens (the cheapest of the three frontier models), and matches the others on Intelligence Index. It does well on well-specified problems and large codebases (1M-token context with strong long-window retrieval). It’s the weakest of the three in agentic chains, where it’s most likely to commit to a wrong interpretation early.

Which coding model is cheapest in practice?

Per finished task, GPT-5.4: its lower output-token count beats Claude Opus 4.7 even though Anthropic’s per-token rate is closer than the headline gap suggests. Per raw API call, Gemini 3.1 Pro at $4.50 blended. Per IDE seat, GitHub Copilot Pro at $10/mo or Cursor Pro at $20/mo, though most heavy users top those up with direct API spend.

The frontier has shifted again since this piece — Qwen 3.7 Max vs GPT-5.5 vs Opus 4.8 reruns the comparison with current models.

Sources

• Artificial Analysis — Opus 4.7 deep dive — Intelligence Index, GDPval Elo, pricing baselines
• LM Council benchmarks (April 2026) — SWE-bench Verified scores with error bars for all three models
• Scale AI SWE-Bench Pro leaderboard — the harder, less-leaked benchmark
• MorphLLM — Why 46% beats 81% on SWE-bench Pro — analysis of the Verified-vs-Pro gap and benchmark contamination
• OpenAI — Introducing GPT-5.5 — official agentic numbers for the next-gen GPT
• MindStudio — GPT-5.5 vs Opus 4.7 coding comparison — origin of the 72% output-token efficiency claim
• LLM Stats SWE-bench Verified leaderboard — independent leaderboard with model-by-model scores

Bottom line

The May 2026 reality is that all three frontier models are good enough to ship code, and the pricing has compressed the “use the best model” argument into “match the model to the task.” If I had to keep one, it would still be Claude Opus 4.7. The per-query quality is real and pays back across multi-file work. If I had to drop one, it would be GPT-5.4 only because GPT-5.5 is about to land and reset the agentic side of this comparison. Gemini 3.1 Pro is the one to trust least for hard reasoning and use most for cheap volume — a working contradiction that survives because the three jobs are different jobs.

If you’re picking one for your team, ask which of three constraints binds first: query-quality budget, iteration-cost budget, or per-call dollar budget. Answer that, and the model picks itself. And if budget is binding hard, DeepSeek V4 Pro now enters the picture at 1/7th of Claude’s price with 80% SWE-bench.