Reasoning report / Updated 2026-09-19
Reasoning and Science Benchmarks: Reading Claude Claims Beyond Coding
Claude benchmark discussions are often dominated by coding, but the hardest buyer questions involve research, finance, law, science, and multi-step analysis under uncertainty. The September 2026 evidence is richer here than it was in July, and easier to misread.
Citable baseline
Key facts
"System cards document the capabilities, safety evaluations, and responsible deployment decisions for Claude models."Anthropic system-card index, primary source
Reasoning scores are not procurement answers
Academic reasoning benchmarks such as GPQA Diamond, FrontierMath, ARC-AGI, and Humanity's Last Exam are valuable because they are hard to solve with shallow pattern matching. OpenAI's GPT-5.5 page reports high scores across several of these evaluations, and Anthropic's Fable 5.1 announcement reports its own, while the system-card program covers broader Claude capability and safety evaluations.
The caution is transfer. A model that scores well on a formal benchmark can still fail an internal research workflow if it loses document provenance, mishandles uncertainty, or cannot use tools in the required environment. For Claude Reports, a reasoning score is a starting hypothesis, not a verdict.
Two HLE numbers are not one comparison
Anthropic reports Fable 5.1 at 60.9% on Humanity's Last Exam without tools and 65.0% with tools, against Opus 5 at 56.6% and 63.6%. OpenAI reports GPT-5.5 at 52.2% with tools. The temptation to subtract is immediate and should be resisted.
These are self-reported figures from two vendors, produced at different dates, under undisclosed and almost certainly different harnesses, with different tool sets behind the phrase "with tools." That phrase is doing enormous unstated work: a model given a search index, a code interpreter, and a long attempt budget is running a different evaluation from one given a single retrieval call.
What the numbers do support is within-family movement. Anthropic's table shows Fable 5.1 above Fable 5 and above Opus 5 on both HLE variants, measured by the same party under presumably the same harness. That is a real signal about generational progress and a useful input to a Claude-internal routing decision. It is not a cross-vendor ranking.
Professional and science benchmarks need provenance
GDPval-AA v2 appears in Anthropic's launch table as a rating — 1853 for Fable 5.1, 1824 for Opus 5, 1723 for Fable 5 — rather than an accuracy. Ratings and percentages are different scales and cannot be averaged into a composite. A 29-point rating gap also has no intuitive interpretation without the scale's calibration, which the announcement does not supply.
Terminal-Bench-Science 0.1 is the most striking row in the same table: 52.6% for Fable 5.1 against 29.0% for Opus 5. A gap that large on a new, low-version-number benchmark deserves caution rather than excitement. Early benchmark versions are where harness effects, task-set idiosyncrasies, and unrepresentative difficulty distributions are least well characterised.
Anthropic's announcement also describes public science examples at a high level, including planetary elevation modelling from open Magellan data, protein-binder work validated experimentally under Mythos trusted access, and GPU kernel speedups. These are demonstrations, not evaluations: they have no baseline, no control, and no failure count. Treat them as existence proofs of what the model can sometimes do, and keep them out of comparison tables.
How to evaluate a research workflow
A good private eval should include source retrieval, citation accuracy, uncertainty calibration, spreadsheet or code execution if relevant, and a human review rubric. It should measure whether the model preserves evidence, not only whether the final answer sounds plausible.
Refusal and routing behaviour belongs in the same eval. Anthropic's published policy restricts Fable on professional biology and drug development work and routes biology-related blocks to Opus 5, and describes automatic fallbacks when safety classifiers flag a request. A model that declines or reroutes part of your workload has a different operational profile from one that attempts everything, and no benchmark score captures that.
Claude may be the right choice when long context, document synthesis, and careful uncertainty handling matter. GPT may be the right choice when a specific reasoning benchmark, tool mode, or organization-standard agent performs better. The only defensible answer is dated and task-specific.
FAQ
Are academic reasoning benchmarks enough to choose Claude or GPT?
No. They are useful signals, but private workflow evals should include provenance, tool use, review time, and failure handling. Self-reported scores from two different vendors cannot be compared directly.
Why include safety cards in a benchmark report?
Because refusals, safeguards, fallback routing, and model deployment decisions affect whether a benchmark result transfers to production.
Cite this page
Claude Reports. "Reasoning and Science Benchmarks: Reading Claude Claims Beyond Coding." Updated 2026-09-19. https://claudereports.com/reports/reasoning-science-benchmarks/