Can coding agents finish Paper Mario?
No, but some of them can finish part of it.
The benchmark is the game's opening: from a fresh save file, through Peach's castle, until the first Bowser battle begins. A human clears it in minutes; agents get 90. And this is not a computer-use eval: the agents play with a full toolkit, from an interactive controller and TAS-style controls to the game's decompiled source and walkthroughs.
- G4 finishes
- 176
- Model families
- 30
- Run cap
- 90m
349 comparison runs
The standings
How far each model gets, and how fast it gets there.
Reaching Bowser is 100%; earlier gates earn partial credit. Rows show average progress, the best run, and the fastest finish. Completion consistency and time separate setups that reach equally far.
345 ranked attempts · 176 G4 · 3 runs per effort
Average progress Best runTime: fastest observed finish · 0–90 min · shorter is faster
Select a column heading to sort.
| Progress & completion time | |||||||
|---|---|---|---|---|---|---|---|
 Opus 5.5 [low]September 2026 | 100% | 100% | 3/3 | 17:51 | G4 | $2.11Mean estimated cost per attempt in this effort. Recorded primary and helper input/cache counters and available finalized output, priced at standard API rates as of 2026-09-24. Native cache-write lifetimes are reconciled. Unreported output and final tails are excluded; not a bill. | |
 GPT-6.1 Sol [high]October 2026 | 100% | 100% | 3/3 | 19:32 | G4 | $1.67Mean estimated cost per attempt in this effort. Recorded cumulative usage priced at standard API rates as of 2026-10-09. Native usage reports zero cache writes; cached reads are subtracted from inclusive input before pricing ordinary input. Observed request input stays within the short-context tier. Unreported final usage and orchestration overhead are excluded; not a bill. | |
| GPT-6 Astra [medium]September 2026 | 100% | 100% | 3/3 | 20:05 | G4 | ~$12.29Mean estimated cost per attempt in this effort. Conservative estimate: all recorded non-cached input is priced as cache writes because Codex did not reliably report the split. The recorded-usage range is $11.99 to $12.29. | |
| Fable 5.1 [medium]September 2026 | 100% | 100% | 3/3 | 21:58 | G4 | $5.47Mean estimated cost per attempt in this effort. Recorded primary and helper input/cache counters and available finalized output, priced at current standard API rates as of 2026-09-15. Unreported output and final tails are excluded; not a bill. Recorded primary and helper usage with reconciled cache lifetimes and finalized native model totals, priced at current standard API rates as of 2026-09-15. Not a bill. | |
| GPT-6 Sol [low]October 2026 | 100% | 100% | 3/3 | 22:10 | G4 | $2.45Mean estimated cost per attempt in this effort. Recorded cumulative usage priced at standard API rates as of 2026-10-09. Native usage reports zero cache writes; cached reads are subtracted from inclusive input before pricing ordinary input. Observed request input stays within the short-context tier. Unreported final usage and orchestration overhead are excluded; not a bill. | |
| Opus 5 [medium]August 2026 | 100% | 100% | 3/3 | 24:10 | G4 | $8.07Mean estimated cost per attempt in this effort. Recorded primary and helper usage with reconciled cache lifetimes and finalized native model totals, priced at current standard API rates as of 2026-09-15. Not a bill. | |
| GPT-5.6 Sol [low]August 2026 | 100% | 100% | 3/3 | 24:36 | G4 | $11.04Mean estimated cost per attempt in this effort. Recorded usage repriced at September 15, 2026 standard API rates. Cache-write split is unreported; lower prices non-read input as ordinary input, upper prices it all as writes. Uses short-context rates; historical request-level context tiers could not be verified. Unreported final usage and orchestration overhead are excluded. Underlying request evidence is unavailable; retained aggregate export supplies the tokens. | |
| Fable 5 [low]August 2026 | 100% | 100% | 3/3 | 26:32 | G4 | $18.13Mean estimated cost per attempt in this effort. Recorded input and cache-use subtotal at current standard API rates as of 2026-09-15. Finalized output counts were not retained, so output cost is excluded; not a bill. | |
| Opus 4.7 [high]September 2026 | 100% | 100% | 3/3 | 30:39 | G4 | $22.71Mean estimated cost per attempt in this effort. Recorded primary and helper input/cache counters and available finalized output, priced at current standard API rates as of 2026-09-15. Unreported output and final tails are excluded; not a bill. | |
| Opus 4.8 [high]August 2026 | 100% | 100% | 3/3 | 36:41 | G4 | $13.34Mean estimated cost per attempt in this effort. Recorded primary and helper usage with reconciled cache lifetimes and finalized native model totals, priced at current standard API rates as of 2026-09-15. Not a bill. | |
| GPT-5.5 [high]August 2026 | 100% | 100% | 3/3 | 41:39 | G4 | $15.60Mean estimated cost per attempt in this effort. Recorded usage repriced at September 15, 2026 standard API rates. GPT-5.5 has no additional cache-write charge. Uses short-context rates; historical request-level context tiers could not be verified. Unreported final usage and orchestration overhead are excluded. | |
 DeepSeek V4.1 Flash [low]September 2026 | 100% | 100% | 3/3 | 44:35 | G4 | $1.53Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Missing final usage is excluded. Uses regular peak prices, without off-peak discounts. | |
 Grok 4.6 [high]August 2026 | 96% | 100% | 2/3 | 1:09:32 | G4 | $14.15Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Per-request long-context tiers are unavailable; standard-context rates are assumed. | |
 Gemini 3.8 Flash [high]September 2026 | 93% | 100% | 2/3 | 36:23 | G4 | $6.44Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Includes helper-model tokens; unrecorded final usage, search-service charges and cache storage are excluded. | |
| GPT-5.6 Terra [xhigh]August 2026 | 89% | 100% | 2/3 | 44:14 | G4 | $12.36Mean estimated cost per attempt in this effort. Recorded usage repriced at September 15, 2026 standard API rates. Cache-write split is unreported; lower prices non-read input as ordinary input, upper prices it all as writes. Uses short-context rates; historical request-level context tiers could not be verified. Unreported final usage and orchestration overhead are excluded. Underlying request evidence is unavailable; retained aggregate export supplies the tokens. | |
 GLM-5.2* [high]August 2026 | 89% | 100% | 2/3 | 1:06:46 | G4 | $10.56Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. | |
| GPT-5.6 Luna [high]August 2026 | 85% | 100% | 2/3 | 47:33 | G4 | $1.29Mean estimated cost per attempt in this effort. Recorded usage repriced at September 15, 2026 standard API rates. Cache-write split is unreported; lower prices non-read input as ordinary input, upper prices it all as writes. Uses short-context rates; historical request-level context tiers could not be verified. Unreported final usage and orchestration overhead are excluded. Underlying request evidence is unavailable; retained aggregate export supplies the tokens. | |
| Gemini 3.7 Flash [high]August 2026 | 85% | 100% | 1/3 | 1:09:10 | G4 | $6.18Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Missing final usage is excluded. | |
 Kimi K3 [low]August 2026 - Baseten | 82% | 100% | 2/3 | 44:19 | G4 | $10.38Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. | |
| Sonnet 5 [xhigh]August 2026 | 82% | 100% | 1/3 | 1:08:40 | G4 | $83.09Mean estimated cost per attempt in this effort. Recorded primary and helper usage with reconciled cache lifetimes and finalized native model totals, priced at current standard API rates as of 2026-09-15. Not a bill. | |
| Opus 4.6 [max]September 2026 | 74% | 100% | 1/3 | 1:15:14 | G4 | $23.06Mean estimated cost per attempt in this effort. Recorded primary and helper input/cache counters and available finalized output, priced at current standard API rates as of 2026-09-17. Unreported output and final tails are excluded; not a bill. | |
| Gemini 3.1 Pro [high]August 2026 | 67% | 100% | 1/3 | 47:53 | G4 | $9.05Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Missing final usage is excluded. | |
| Grok 4.5 [low]August 2026 | No finish | 63% | 78% | 0/3 | — | S3 | $5.47Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Per-request long-context tiers are unavailable; standard-context rates are assumed. |
| GLM-5.3-Flash [low]September 2026 | No finish | 59% | 78% | 0/3 | — | S3 | $1.97Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Missing final usage is excluded. |
| Opus 4.5 [low]September 2026 | No finish | 59% | 67% | 0/3 | — | S2 | $28.91Mean estimated cost per attempt in this effort. Recorded primary usage and reported ancillary Haiku usage at September 16, 2026 standard API rates, including cache-write lifetimes. Missing output, unreported ancillary usage and final tails are excluded; not a bill. |
| DeepSeek V4 Flash Vision (exp.) [max]September 2026 | No finish | 52% | 78% | 0/3 | — | S3 | $0.69Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Missing final usage is excluded. Uses Fireworks pricing for the original model; its first-party endpoint is retired. |
| Gemini 3.5 Flash [high]August 2026 | No finish | 52% | 78% | 0/3 | — | S3 | $12.16Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Missing final usage is excluded. |
| Gemini 3.6 Flash [medium]August 2026 | No finish | 52% | 89% | 0/3 | — | S4 | $2.67Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. Missing final usage is excluded. |
| GPT-6 Luna [xhigh]October 2026 | No finish | 52% | 78% | 0/3 | — | S3 | $0.63Mean estimated cost per attempt in this effort. Recorded cumulative usage priced at standard API rates as of 2026-10-09. Native usage reports zero cache writes; cached reads are subtracted from inclusive input before pricing ordinary input. Observed request input stays within the short-context tier. Unreported final usage and orchestration overhead are excluded; not a bill. |
| Kimi K2.7 Code* [high]August 2026 | No finish | 44% | 44% | 0/3 | — | S1 | $10.46Mean estimated cost per attempt in this effort. Recorded tokens at standard API prices as of 2026-09-15. |
*GLM 5.2 and Kimi K2.7 Code were given custom image MCP tools backed by locally hosted Qwen3.6-35B and LocateAnything-3B.
Muse Spark is omitted from these rankings because its 50-image-per-request cap prevented comparable testing. Which versions we tried and what happened.
How the cost estimates are counted
Estimates cover 362 of 362 retained runs using dated standard API pricing snapshots. New models use their audited pricing date; earlier estimates keep their existing snapshot. Leaderboard means use only the selected attempts. Unreported usage is excluded. Two older Gemini runs use modeled estimates because their token records are missing; these carry ~.
These are estimates, not bills. Astra’s ~ estimate prices all recorded non-cached input as cache writes because the input/write split was not reliably reported. Tooltips explain provider choices and incomplete records. Read the cost breakdown and its limits.
The route, in three frames
Save file to Bowser: the ground every run has to cover.
Rendered with the Paper Mario 64 R HD texture pack by MasterKillua; the pack itself is not redistributed here.
The common clock
Same route: 3:47.4 for a lifelong player, 17:51.4 for the fastest agent.
Shared wall clock17:51.4All references have finished. Opus 5.5 low used 4.712 times the practiced human's wall time.
Lifelong player3:47.4
finish
Casual player7:19.2
finish
Opus 5.5 low17:51.4
finish
One route-familiar human on direct controller input, with a casual player as a second reference point. Agents play through a paused harness that freezes the game while they think, so the gap is execution overhead, not a ranking of capability. Human rows are never averaged with each other or with model cells.
Explore the August race by checkpoint
| Gate | Lifelong playerdirect control · real time | GPT-5.6 Sollow · default | Fable 5low · default | Kimi K3high · droid-factory-60tps |
|---|---|---|---|---|
| G1 | 0:38.1 | 2:52.0 | 2:52.3 | 26:57.7 |
| G2 | 1:23.4 | 8:12.7 | 5:12.5 | 32:07.3 |
| G3 | 1:50.1 | 8:44.6 | 5:50.1 | 34:01.7 |
| S1 | 1:57.2 | 11:46.3 | 10:53.7 | 38:10.9 |
| S1u | — | 12:43.0 | 16:24.0 | 51:08.0 |
| S2 | 2:09.6 | 14:57.7 | 17:37.2 | 61:36.9 |
| S3 | 2:29.6 | 18:34.4 | 20:56.6 | 74:22.0 |
| S4 | 3:14.1 | 21:36.8 | 23:28.7 | 85:05.1 |
| G4 | 3:47.4 | 24:36.1 | 26:32.4 | 89:41.7 |
Under the hood