Can coding agents finish Paper Mario?
No, but some of them can finish part of it.
The benchmark is the game's opening: from a fresh save file, through Peach's castle, until the first Bowser battle begins. A human clears it in minutes; agents get 90. And this is not a computer-use eval: the agents play with a full toolkit, from an interactive controller and TAS-style controls to the game's decompiled source and walkthroughs.
- G4 finishes
- 124
- Model families
- 23
- Run cap
- 90m
The standings
How far each model gets, and how fast it gets there.
Reaching Bowser is 100%; earlier gates earn partial credit. Rows show average progress, the best run, and the fastest finish. Completion consistency and time separate setups that reach equally far.
| Progress & completion time | |||||||
|---|---|---|---|---|---|---|---|
 GPT-6 Astra [medium]September 2026 | 100% | 100% | 3/3 | 20:05 | G4 | ~$12.29Mean estimated cost per attempt in this effort. Conservative estimate: all recorded non-cached input is priced as cache writes because Codex did not reliably report the split. The recorded-usage range is $11.99 to $12.29. | |
 Fable 5.1 [medium]September 2026 | 100% | 100% | 3/3 | 21:58 | G4 | $5.44Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| Opus 5 [medium]August 2026 | 100% | 100% | 3/3 | 24:10 | G4 | $7.61Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| GPT-5.6 Sol [low]August 2026 | 100% | 100% | 3/3 | 24:36 | G4 | $13.90Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| Fable 5 [low]August 2026 | 100% | 100% | 3/3 | 26:32 | G4 | $17.05Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| Opus 5 [max]September 2026 | 100% | 100% | 3/3 | 32:45 | G4 | — | |
| Opus 4.8 [high]August 2026 | 100% | 100% | 3/3 | 36:41 | G4 | $10.32Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| GPT-5.5 [high]August 2026 | 100% | 100% | 3/3 | 41:39 | G4 | $15.60Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
 DeepSeek V4.1 Flash [low]September 2026 | 100% | 100% | 3/3 | 44:35 | G4 | $0.76Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
 Grok 4.6 [high]August 2026 | 96% | 100% | 2/3 | 1:09:32 | G4 | $14.14Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
 Gemini 3.8 Flash [high]September 2026 | 93% | 100% | 2/3 | 36:23 | G4 | $6.43Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| GPT-5.6 Terra [xhigh]August 2026 | 89% | 100% | 2/3 | 44:14 | G4 | $6.18Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
 GLM-5.2* [high]August 2026 | 89% | 100% | 2/3 | 1:06:46 | G4 | $10.57Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| GPT-5.6 Luna [high]August 2026 | 85% | 100% | 2/3 | 47:33 | G4 | $0.64Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| Gemini 3.7 Flash [high]August 2026 | 85% | 100% | 1/3 | 1:09:10 | G4 | $6.18Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
 Kimi K3 [low]August 2026 - Baseten | 82% | 100% | 2/3 | 44:19 | G4 | $10.38Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| Opus 4.8 [max]September 2026 | 82% | 100% | 2/3 | 1:03:51 | G4 | — | |
| Sonnet 5 [xhigh]August 2026 | 82% | 100% | 1/3 | 1:08:40 | G4 | $51.41Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| Gemini 3.1 Pro [high]August 2026 | 67% | 100% | 1/3 | 47:53 | G4 | $9.04Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. | |
| Grok 4.5 [low]August 2026 | No finish | 63% | 78% | 0/3 | — | S3 | $5.47Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. |
| GLM-5.3-Flash [low]September 2026 | No finish | 59% | 78% | 0/3 | — | S3 | $1.86Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. |
| DeepSeek V4 Flash Vision (exp.) [max]September 2026 | No finish | 52% | 78% | 0/3 | — | S3 | $0.87Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. |
| Gemini 3.5 Flash [high]August 2026 | No finish | 52% | 78% | 0/3 | — | S3 | $12.16Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. |
| Gemini 3.6 Flash [medium]August 2026 | No finish | 52% | 89% | 0/3 | — | S4 | $5.33Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. |
| Kimi K2.7 Code* [high]August 2026 | No finish | 44% | 44% | 0/3 | — | S1 | $10.47Mean estimated cost per attempt in this effort. API-equivalent estimate from recorded usage. Unreported usage is excluded; this is not a bill. |
*GLM 5.2 and Kimi K2.7 Code were given custom image MCP tools backed by locally hosted Qwen3.6-35B and LocateAnything-3B.
Muse Spark is omitted from these rankings because its 50-image-per-request cap prevented comparable testing. Which versions we tried and what happened.
How the cost estimates are counted
Cost estimates cover 255 of 265 retained runs. They use recorded usage and run-era pricing or client price tables; missing usage can leave an estimate incomplete. Missing costs stay blank.
These are estimates, not bills. OpenRouter uses an unversioned client price table. Astra’s ~ estimate prices all recorded non-cached input as cache writes because the input/write split was not reliably reported. Read the cost breakdown and its limits.
The route, in three frames
Save file to Bowser: the ground every run has to cover.
Rendered with the Paper Mario 64 R HD texture pack by MasterKillua; the pack itself is not redistributed here.
The common clock
Same route: 3:47.4 for a lifelong player, 20:05.3 for the fastest agent.
One route-familiar human on direct controller input, with a casual player as a second reference point. Agents play through a paused harness that freezes the game while they think, so the gap is execution overhead, not a ranking of capability. Human rows are never averaged with each other or with model cells.
Explore the August race by checkpoint
| Gate | Lifelong playerdirect control · real time | GPT-5.6 Sollow · default | Fable 5low · default | Kimi K3high · droid-factory-60tps |
|---|---|---|---|---|
| G1 | 0:38.1 | 2:52.0 | 2:52.3 | 26:57.7 |
| G2 | 1:23.4 | 8:12.7 | 5:12.5 | 32:07.3 |
| G3 | 1:50.1 | 8:44.6 | 5:50.1 | 34:01.7 |
| S1 | 1:57.2 | 11:46.3 | 10:53.7 | 38:10.9 |
| S1u | — | 12:43.0 | 16:24.0 | 51:08.0 |
| S2 | 2:09.6 | 14:57.7 | 17:37.2 | 61:36.9 |
| S3 | 2:29.6 | 18:34.4 | 20:56.6 | 74:22.0 |
| S4 | 3:14.1 | 21:36.8 | 23:28.7 | 85:05.1 |
| G4 | 3:47.4 | 24:36.1 | 26:32.4 | 89:41.7 |
Under the hood