The behavior story

Same score, different behavior.

Astra finished all 15 September opening attempts without a recorded state-load request. DeepSeek V4.1 Flash finished eight of nine with a median of 41 loads. Both found ways through. What did they do when a move failed?

66 September opening attempts. The separate Astra marathon and its late reload appear in the marathon analysis.

What changed the next move?

A stalled position can prompt a better input, a measurement of the room, or another look at an old picture. These September examples connect the recorded action to what happened next.

Mario stands inside Peach’s Castle, before the diagonal holds.
Mario starts just inside Peach’s Castle.View original capture
Mario remains in place inside the castle after the first upper-left hold.
After the first full diagonal hold, Mario remains in the same position.View original capture
Mario still stands in the same spot inside the castle after a second upper-left hold.
A second full diagonal hold leaves Mario in place again.View original capture
Mario has moved left across the castle foyer after the reduced diagonal input.
Reducing the diagonal input gets Mario moving left across the foyer.View original capture

HD recreation from retained run states. Textures by MasterKillua.

GPT-6 Astra · low effort · September 2026

The command ran. Mario stayed put.

Action
Two upper-left holds left the position unchanged. Astra reduced the diagonal stick input and checked whether Mario moved.
Outcome
The smaller input produced motion. The attempt went on to finish in 21m 11s.
Read the full model profile
Run evidence

A command can succeed without moving Mario

v3-codex-gpt6astra-low-tortle-001
Mario stands on the staircase landing inside Peach’s Castle.
On the staircase landing, the recorded height is 50 game units.View original capture
Mario reaches the upper balcony beside a door in Peach’s Castle.
Mario reaches the balcony at 100 game units. These captures show the successful climb after the collision analysis.View original capture

HD recreation from retained run states. Textures by MasterKillua.

DeepSeek V4.1 Flash · low effort · September 2026

A staircase became a measurable route.

Action
After direction sweeps failed, DeepSeek decoded collision triangles from live memory, identified the west staircase and moved onto it.
Outcome
Successive position checks showed Mario gaining height to the balcony. The attempt finished in 56.6 minutes.
Read the full model profile
Run evidence

Reading the staircase out of the game

v3-opencode-ds41flash-low-mander-001
Mario and Luigi stand at the foot of the castle party stairs, with a Toad nearby.
HD recreation of the first saved picture. GLM opened the original file 384 times.View original capture
Mario and Luigi remain at the foot of the party stairs, while the nearby Toad has moved.
HD recreation of the second saved picture, whose original was also opened 384 times. The Toad moved between the original captures, but rereading either file could not update the view.View original capture

HD recreation from retained run states. Textures by MasterKillua.

GLM-5.3-Flash · low effort · September 2026

Reading again did not reveal a new frame.

Action
After starting a party-scene script, GLM opened two existing image files 384 times each across roughly 73 minutes. These were rereads, not fresh screenshots.
Outcome
The remaining native tool stream recorded image reads rather than further navigation. The scorer never recorded the castle hall.
Read the full model profile
Run evidence

Reading the same two pictures

v3-opencode-glm53flash-low-tortle-001

These scenes are recreated with HD textures from retained saved states and recorded inputs. The original captures are linked below each image. Actions and outcomes are checked against transcripts and scorer records.

The model profiles

The benchmark is the castle. The behavior is the model.

In the August analysis, 110 of the 113 traced non-completions ended at the castle door or deeper, so clearing the overworld mostly proves an agent can operate the emulator. What separates models is what they do once the game starts testing play.

I analyzed the 193 August attempts and built a profile for each model: how it plays, where it stalls, and which model it most behaves like. The new September profiles add 66 attempts across six model groups, with concrete examples from the retained transcripts and scorer logs. Muse Spark is excluded from these comparisons because of the image cap.

August comparisons: 181 attempts, 78 G4 completions. September: 66 attempts, 41 completions. Collection periods and Kimi K3 CLI conditions stay separate. Open a profile for the full story.
ConditionPeriodG4 / attemptsMedian finish
AstraSeptember15 / 1524:58
Fable 5.1September12 / 1535:58
DeepSeek V4.1 FlashSeptember8 / 962:35
Gemini 3.8 FlashSeptember6 / 977:14
DeepSeek V4 Flash VisionSeptember0 / 9no finishes
GLM-5.3-FlashSeptember0 / 9no finishes
Opus 5August12 / 1236:01
GPT-5.6 SolAugust15 / 1550:10
Opus 4.8August10 / 1253:34
Fable 5August9 / 1238:59
GLM 5.2August3 / 667:04
Kimi K3 · DroidAugust4 / 949:15
GPT-5.5August5 / 1257:45
GPT-5.6 TerraAugust5 / 1560:23
Sonnet 5August4 / 1276:52
GPT-5.6 LunaAugust4 / 1578:00
Grok 4.6August3 / 1271:58
Gemini 3.7 FlashAugust2 / 970:19
Gemini 3.1 ProAugust1 / 647:53
Grok 4.5August1 / 958:52
Gemini 3.6 FlashAugust0 / 9never finished
Gemini 3.5 FlashAugust0 / 9never finished
Kimi K2.7 CodeAugust0 / 3never finished
Kimi K3 · OpenCodeAugust0 / 3never finished
Kimi K3 · CursorAugust0 / 1never finished

Astra

15 / 15 G4September 2026

Fifteen finishes, with failures turned into small control experiments.

How it played

All 15 Astra attempts reached the opening Bowser battle. The recurring method was a small TAS program that issued bounded input batches, captured a screenshot and saved a checkpoint. Most runs added room and position readers, then used height changes to distinguish climbing stairs from walking alongside them and room changes to distinguish opening a door from talking to its guard.

The clearest recovery happened when an upper-left input advanced frames without moving Mario. At low effort, slightly smaller stick values restored motion. A max run went further: after first blaming a Toad, it read the game's stick fields and found that the full diagonal registered as zero while a reduced diagonal registered and moved. It revised the diagnosis and climbed the stairs. The useful behavior was checking whether a command actually took effect.

This was not one mandatory recipe. A low run finished in 22m 14s with a compact screenshot-guided action script and no agent-authored memory reader found in the transcript. All 15 saved checkpoints, but none requested a load. Recovery mostly meant changing the next approach from the current position. An xhigh run also resumed a prematurely ended dialogue batch after checking that the emulator was paused, then reached the battle.

Effort and outcomes

Every effort finished 3/3. Completion medians were low 22m 14s, medium 28m 09s, high 23m 37s, xhigh 26m 36s and max 31m 52s. Max spent longer before the first scored save and exposed more detailed diagnostics in these transcripts, but the timings do not form a monotonic effort curve. With only three attempts per level, this does not isolate a causal effect of reasoning effort.

Moments from the runs

A command can succeed without moving Mario

Two upper-left holds left the position unchanged. Slightly reducing the diagonal stick input produced motion; the run went on to finish in 21m 11s.

low effortRun: v3-codex-gpt6astra-low-tortle-001

From a Toad hypothesis to a stick-field test

After suspecting a guest obstruction, it measured full-diagonal input as zero and reduced input as nonzero, then reached the staircase and completed in 30m 21s.

max effortRun: v3-codex-gpt6astra-max-saur-001

A successful visual route

A short action runner, screenshots and detours around a guard and sofa were enough for a 22m 14s finish; no agent-authored RAM reader was found in its command stream.

low effortRun: v3-codex-gpt6astra-low-mander-002

Resuming an interrupted batch

A dialogue batch ended unexpectedly. It inspected the paused emulator and resumed with smaller batches, finishing in 26m 36s.

xhigh effortRun: v3-codex-gpt6astra-xhigh-saur-001
Scope and limitations
  • These are 15 guided opening attempts, not full-game or general gameplay tests.
  • The opening includes long scripted travel and cutscenes; an unchanged room or story value does not establish inactivity.
  • Astra's retained native stream does not provide a supported delegation inventory. No claim about tool availability or a verified zero-delegation rate is made.
  • Checkpoint counts are command requests. Most screenshot contents were read through the agent's narration and surrounding telemetry, not independently re-rendered for this report.
Command counts and effort groups

Received controller commands, including harness traffic; these are requests rather than confirmed actions or individual model tool calls. Successful runs end sooner.

Medians across this model's runs. 0 of 15 runs recorded a load request.

Save requests
57
Load requests
0
Screenshots
59
Input burst
30 frames
September effort and harness conditions
Effort / CLIFinishedSave medianLoad median
lowcodex · 9b8e68a3/3550
mediumcodex · 9b8e68a3/3500
highcodex · 9b8e68a3/3570
xhighcodex · 9b8e68a3/3570
maxcodex · 9b8e68a3/3700
What these logs can and cannot measure
  • September only: 66 counted ordinary attempts; August observations are a separate service and harness epoch.
  • The counted Gemini 3.8 Flash rule-violating run remains in the denominator; campaign integrity overrides raw scorer events.
  • Command counts are received command verbs, not acknowledgements of successful execution or model tool-call counts. Scorer/final-capture traffic is not separately tagged.
  • RAM counts include external scorer reads. Agent-attributed memory use and TAS-engine execution counts are unknown; raw frame stepping does not establish use of the TAS engine.
  • Burst frames sum requested STEP_FRAME frames between SET_INPUT_PORT boundaries, ending at CLEAR_INPUT_PORT; frames are not seconds. Model summaries are medians of run medians.
  • A map revisit is an observed transition back to a previously sampled map; polling can miss short visits, and backtracking can be appropriate.
  • Stationary episodes have identical sampled map and position for at least 12 seconds, across positive poll gaps no greater than 10 seconds. They do not prove inactivity, confusion, or a stalled agent; menus, cutscenes, reasoning and paused game time can all qualify.
  • Native delegation counts use distinct Claude Agent/Task and OpenCode task/subagent invocation IDs. They exclude delegation hidden inside Workflow or shell scripts. Codex item streams and AGY prose do not support this census and remain unknown; absent tool inventory is not proof that delegation was unavailable.
  • Different effort levels, routes and harness commits remain in condition aggregates. Descriptive model medians are not causal capability comparisons; completed runs also end sooner.
Every attempt · scorer clock
  • highG4 in 23:37
  • highG4 in 23:40
  • highG4 in 21:55
  • lowG4 in 24:58
  • lowG4 in 22:14
  • lowG4 in 21:11
  • maxG4 in 31:52
  • maxG4 in 30:21
  • maxG4 in 33:42
  • mediumG4 in 20:05
  • mediumG4 in 28:09
  • mediumG4 in 28:18
  • xhighG4 in 22:01
  • xhighG4 in 26:36
  • xhighG4 in 27:21

Fable 5.1

12 / 15 G4September 2026

Twelve finishes; the three misses ended while waiting for background play.

How it played

Fable 5.1 usually built its own shell or Python controls around the emulator adapter: dialogue loops, position readers, coordinate walkers and small door experiments. Its fastest run saved the same starting state and tried two ways around a guard. The first stayed in the room; the second changed the map. That comparison gave it a concrete route onward. Other runs had longer detours through repeated conversations, stair searches and helper repairs.

One medium run shows both the mistake and the recovery. It entered the hall before Peach and started a dialogue loop, but the printed coordinates stayed in place with controls enabled. After eleven reported batches it stopped the loop, checked the scene script and found that Mario first had to walk farther east. It crossed that trigger, restarted the scene driver and finished. Reading the right room ID was not enough; the action still had to match the current game state.

The three incomplete runs tell a different story from the navigation detours. At xhigh and max, each final message promised to resume after a background gameplay notification; the native client then ended its turn and the recorded agent process exited before the final scorer gate. They stopped at save creation, Toad Town and the upper castle hall. Background play also succeeded in other runs, including the sole max finish, which kept a monitor active and refined its event filters. The shared failure is a handoff worth investigating, not evidence that those runs gave up on a staircase.

Seven runs requested research agents, with nine native requests in total, to look up source addresses or the route. That pattern appears in both successes and misses. The failed max run that stopped after save creation made no such request, while the successful max run made two. Delegation therefore describes part of its working method here without explaining the outcome by itself.

Effort and outcomes

Low, medium and high each finished 3/3; xhigh finished 2/3 and max 1/3. Among finishers, medians were low 29m 58s, medium 37m 00s, high 40m 34s and xhigh 33m 31s. The single max finish took 34m 19s. These completion times exclude early exits, so they should be read beside the completion counts. The slow API low run took 53m 24s after a guard/door detour; the other low runs finished in about 27 and 30 minutes. Service route, host and a sample of three per effort limit causal comparisons.

Moments from the runs

Two approaches from one saved position

It restored a checkpoint for two guard detours. The first remained in map 0; the second entered map 1. It finished in 21m 58s, the fastest Fable run.

medium effortRun: v3-claude-fable51-sub-medium-mander-002

The hall was right, the action was wrong

Repeated dialogue batches only made Mario jump in place. It stopped them, looked up the positional scene trigger, walked to it and finished in 40m 29s.

medium effortRun: v3-claude-fable51-sub-medium-tortle-001

A guard hypothesis became a detour

It tried a five-conversation strategy, reconsidered the route, restored a prior state and identified the hall with a memory scan. It eventually finished in 53m 24s.

low effortRun: v3-claude-fable51-api-low-mander-001

The final walk was left in the background

After reaching the upper hall, it launched a walk-to-Peach and dialogue chain in the background, promised to await notification and exited at 38m 02s without the battle gate.

max effortRun: v3-claude-fable51-sub-max-tortle-001

A monitored background chain did finish

Two research agents supplied source facts while a monitored driver advanced play. It tightened noisy monitor filters, checked the scene and finished in 34m 19s.

max effortRun: v3-claude-fable51-sub-max-mander-002
Scope and limitations
  • The exact reason the native client stopped servicing background work is not traced to a particular scheduler or harness defect. Process exit and the promised notification are directly recorded.
  • One low attempt used the API route; the other two low attempts and all twelve higher-effort attempts used the subscription route.
  • The benchmark ends at entry to the opening Bowser battle. These results do not measure winning it or playing later chapters.
  • Native Agent calls verify research delegation requests, not a causal benefit. Child reports embedded in transcripts are not independent attempts.
Command counts and effort groups

Received controller commands, including harness traffic; these are requests rather than confirmed actions or individual model tool calls. Successful runs end sooner.

Medians across this model's runs. 6 of 15 runs recorded a load request.

Save requests
8
Load requests
0
Screenshots
49
Input burst
2 frames
September effort and harness conditions
Effort / CLIFinishedSave medianLoad median
lowclaude · a6deb5f1/194
lowclaude · 04c31101/191
lowclaude · fe982c41/193
mediumclaude · 025487f1/170
mediumclaude · 04c31101/160
mediumclaude · b49e0fe1/1134
highclaude · 75c22191/11226
highclaude · 8e766f41/11413
highclaude · e313c871/180
xhighclaude · bba8c571/240
xhighclaude · d774cbd1/180
maxclaude · d774cbd1/340

Seven of fifteen runs recorded direct native delegation, with nine requests in total. That count excludes delegation hidden inside scripts or workflows.

What these logs can and cannot measure
  • September only: 66 counted ordinary attempts; August observations are a separate service and harness epoch.
  • The counted Gemini 3.8 Flash rule-violating run remains in the denominator; campaign integrity overrides raw scorer events.
  • Command counts are received command verbs, not acknowledgements of successful execution or model tool-call counts. Scorer/final-capture traffic is not separately tagged.
  • RAM counts include external scorer reads. Agent-attributed memory use and TAS-engine execution counts are unknown; raw frame stepping does not establish use of the TAS engine.
  • Burst frames sum requested STEP_FRAME frames between SET_INPUT_PORT boundaries, ending at CLEAR_INPUT_PORT; frames are not seconds. Model summaries are medians of run medians.
  • A map revisit is an observed transition back to a previously sampled map; polling can miss short visits, and backtracking can be appropriate.
  • Stationary episodes have identical sampled map and position for at least 12 seconds, across positive poll gaps no greater than 10 seconds. They do not prove inactivity, confusion, or a stalled agent; menus, cutscenes, reasoning and paused game time can all qualify.
  • Native delegation counts use distinct Claude Agent/Task and OpenCode task/subagent invocation IDs. They exclude delegation hidden inside Workflow or shell scripts. Codex item streams and AGY prose do not support this census and remain unknown; absent tool inventory is not proof that delegation was unavailable.
  • Different effort levels, routes and harness commits remain in condition aggregates. Descriptive model medians are not causal capability comparisons; completed runs also end sooner.
Every attempt · scorer clock
  • lowG4 in 53:24
  • highG4 in 43:37
  • highG4 in 38:08
  • highG4 in 40:34
  • lowG4 in 29:58
  • lowG4 in 27:26
  • maxG4 in 34:19
  • maxstopped · S3
  • maxstopped · G1
  • mediumG4 in 21:58
  • mediumG4 in 40:29
  • mediumG4 in 37:00
  • xhighG4 in 32:05
  • xhighG4 in 34:57
  • xhighstopped · G2

DeepSeek V4.1 Flash

8 / 9 G4September 2026

Eight finishes from nine attempts, with several breakthroughs coming when it replaced uncertain navigation with measured geometry.

How it played

It built tools around the obstacles it encountered: position readers, camera-aware movement helpers, repeatable direction sweeps and door approaches. The successes took different routes through that work. One low-effort run decoded the live collision mesh to find the staircase; a max-effort run extracted the corridor geometry and used it to choose the correct exit.

Repeated attempts were often experiments from a saved position. One high-effort success used 114 reload commands; a max-effort success used none. Reloading alone does not explain the result. The useful change was turning an observed obstruction into a more specific next move.

The lone failure still made a late recovery. A staircase search hit its 30-minute tool timeout, then a more focused climb reached the upper hall. It entered the next corridor with about six minutes left and pushed north, while the successful geometry-reading run had established that the route continued east.

Effort and outcomes

Low and high each finished all three attempts; max finished two of three. The fastest run was low effort at 44.6 minutes. This small sample does not support a simple claim that more effort improved play.

Moments from the runs

Reading the staircase out of the game

After direction sweeps failed to find the climb, it decoded collision triangles from live memory. It identified the west staircase, moved onto it, and checked Mario rising through successive heights to the balcony. The run finished in 56.6 minutes.

low effortRun: v3-opencode-ds41flash-low-mander-001

The exit was east

It decoded the corridor geometry, located the far eastern door, and fed the resulting route into its waypoint walker. The next commands crossed the corridor and opened the onward door. It finished in 51.0 minutes without a reload.

max effortRun: v3-opencode-ds41flash-max-tortle-001

A recovery that came too late

A broad staircase search timed out after 30 minutes. A focused east-side climb finally worked, but the run reached the next corridor only at 83.7 minutes, then aimed north instead of along the eastbound route. It ran out of time there.

max effortRun: v3-opencode-ds41flash-max-saur-003
Scope and limitations
  • These are nine September attempts, separate from the August runs.
  • Geometry-guided recovery is an observed sequence, not proof that a particular helper caused completion.
Command counts and effort groups

Received controller commands, including harness traffic; these are requests rather than confirmed actions or individual model tool calls. Successful runs end sooner.

Medians across this model's runs. 8 of 9 runs recorded a load request.

Save requests
9
Load requests
41
Screenshots
117
Input burst
8 frames
September effort and harness conditions
Effort / CLIFinishedSave medianLoad median
lowopencode · 14180693/3633
highopencode · 14180692/21786
highopencode · a322fde1/1942
maxopencode · 14180692/3816
What these logs can and cannot measure
  • September only: 66 counted ordinary attempts; August observations are a separate service and harness epoch.
  • The counted Gemini 3.8 Flash rule-violating run remains in the denominator; campaign integrity overrides raw scorer events.
  • Command counts are received command verbs, not acknowledgements of successful execution or model tool-call counts. Scorer/final-capture traffic is not separately tagged.
  • RAM counts include external scorer reads. Agent-attributed memory use and TAS-engine execution counts are unknown; raw frame stepping does not establish use of the TAS engine.
  • Burst frames sum requested STEP_FRAME frames between SET_INPUT_PORT boundaries, ending at CLEAR_INPUT_PORT; frames are not seconds. Model summaries are medians of run medians.
  • A map revisit is an observed transition back to a previously sampled map; polling can miss short visits, and backtracking can be appropriate.
  • Stationary episodes have identical sampled map and position for at least 12 seconds, across positive poll gaps no greater than 10 seconds. They do not prove inactivity, confusion, or a stalled agent; menus, cutscenes, reasoning and paused game time can all qualify.
  • Native delegation counts use distinct Claude Agent/Task and OpenCode task/subagent invocation IDs. They exclude delegation hidden inside Workflow or shell scripts. Codex item streams and AGY prose do not support this census and remain unknown; absent tool inventory is not proof that delegation was unavailable.
  • Different effort levels, routes and harness commits remain in condition aggregates. Descriptive model medians are not causal capability comparisons; completed runs also end sooner.
Every attempt · scorer clock
  • highG4 in 63:49
  • highG4 in 81:59
  • highG4 in 61:31
  • lowG4 in 56:36
  • lowG4 in 63:39
  • lowG4 in 44:35
  • maxG4 in 72:31
  • maxstopped · S2
  • maxG4 in 50:58

Gemini 3.8 Flash

6 / 9 G4September 2026

Six finishes from nine counted attempts, with recoveries from awkward castle routes and two clean runs that reached the upper room too late.

How it played

The recovered transcripts show a mix of scripted scene advancement and hands-on castle navigation. It built state readers and movement helpers, then repeatedly adjusted approaches to stairs and doors using screenshots and coordinates. In successful runs, a guard absorbing the interaction became a positioning problem to work around.

Its results include both a relatively direct 36.4-minute finish and a much longer recovery. One low-effort run entered Peach's bedroom, returned to the hall, and eventually found the upper staircase and onward doors. Another success kept a scene-advance script running while checking its progress and reached Bowser just before the time limit.

The two clean misses were not failures to reach the castle: both entered the upper room and began the Peach sequence near the end of the budget. A separate medium-effort attempt issued a forbidden game-memory write after reaching the hall; it remains a counted invalid attempt.

Effort and outcomes

Low, medium and high each finished two of three counted attempts. High contained the fastest finish, while medium included the latest finish and the invalid run. Those three-run groups are too small and varied to establish an effort trend.

Moments from the runs

Around the guard and onward

It combined a cutscene-advance helper with direct stair and door adjustments, including moving around the guard to reach the doorway. A final scene runner carried the attempt into Bowser at 36.4 minutes, the fastest Gemini finish in this sample.

high effortRun: v3-agy-gemini38flash-api-high-saur-002

Back from the bedroom detour

It spent part of the castle run opening and exploring Peach's bedroom, then returned to the hall. Further stair and door attempts finally put it on the upper route. It finished at 84.0 minutes, showing that a long wrong turn did not always end the run.

low effortRun: v3-agy-gemini38flash-api-low-saur-006

Just before the cap

After working through the castle doors, it launched a script to advance the final scene and kept checking while it ran. The external scorer detected Bowser at 89 minutes 47.2 seconds, only 12.8 seconds before the nominal 90-minute limit.

medium effortRun: v3-agy-gemini38flash-api-medium-tortle-002

A counted invalid attempt

It reached the castle hall through controller play, struggled with the staircase, and then issued a game-memory write at 83.1 minutes. That action invalidated the attempt. Its later actions do not supply a recovery story.

medium effortRun: v3-agy-gemini38flash-api-medium-saur-001
Scope and limitations
  • Decision-level transcript evidence is recoverable for eight of nine runs. One low-effort completion has no retained conversation transcript, so its play style is left undescribed.
  • The invalid attempt stays in the denominator. No progress after its memory write is used as legitimate gameplay evidence.
Command counts and effort groups

Received controller commands, including harness traffic; these are requests rather than confirmed actions or individual model tool calls. Successful runs end sooner.

Medians across this model's runs. 4 of 9 runs recorded a load request.

Save requests
8
Load requests
0
Screenshots
78
Input burst
6 frames
September effort and harness conditions
Effort / CLIFinishedSave medianLoad median
lowagy · 46427680/151
lowagy · 8e766f41/170
lowagy · d4aa8921/1195
mediumagy · 46427682/380
highagy · 46427682/380

The denominator includes the RAM-write violation; only progress recorded before the write receives credit.

What these logs can and cannot measure
  • September only: 66 counted ordinary attempts; August observations are a separate service and harness epoch.
  • The counted Gemini 3.8 Flash rule-violating run remains in the denominator; campaign integrity overrides raw scorer events.
  • Command counts are received command verbs, not acknowledgements of successful execution or model tool-call counts. Scorer/final-capture traffic is not separately tagged.
  • RAM counts include external scorer reads. Agent-attributed memory use and TAS-engine execution counts are unknown; raw frame stepping does not establish use of the TAS engine.
  • Burst frames sum requested STEP_FRAME frames between SET_INPUT_PORT boundaries, ending at CLEAR_INPUT_PORT; frames are not seconds. Model summaries are medians of run medians.
  • A map revisit is an observed transition back to a previously sampled map; polling can miss short visits, and backtracking can be appropriate.
  • Stationary episodes have identical sampled map and position for at least 12 seconds, across positive poll gaps no greater than 10 seconds. They do not prove inactivity, confusion, or a stalled agent; menus, cutscenes, reasoning and paused game time can all qualify.
  • Native delegation counts use distinct Claude Agent/Task and OpenCode task/subagent invocation IDs. They exclude delegation hidden inside Workflow or shell scripts. Codex item streams and AGY prose do not support this census and remain unknown; absent tool inventory is not proof that delegation was unavailable.
  • Different effort levels, routes and harness commits remain in condition aggregates. Descriptive model medians are not causal capability comparisons; completed runs also end sooner.
Every attempt · scorer clock
  • highstopped · S3
  • highG4 in 36:23
  • highG4 in 72:38
  • lowG4 in 59:49
  • lowstopped · S3
  • lowG4 in 84:03
  • mediumG4 in 81:50
  • mediumstopped · S1
  • mediumG4 in 89:47

DeepSeek V4 Flash Vision

0 / 9 G4September 2026

All nine attempts reached the castle, but none reached Bowser; the central difficulty was converting a visible staircase or door into a working route.

How it played

The transcripts repeatedly describe Mario as being at a stair base or beside the correct door, followed by another direction test, another approach, or a return to a checkpoint. This was active experimentation: angle sweeps, height checks, floor inspection, jumping and guard conversations. Several runs nevertheless turned repeated failures into claims that the route itself was inaccessible.

Reading game data did not automatically resolve the confusion. One low-effort run followed a mistaken room sequence toward Peach's bedroom and the guard who opens it. Another kept declaring the foyer door unusable even after adding coordinate and collision checks. The scorer kept both short of the required upper route.

The best counterexample came at max effort. Its first direct route hit interior walls; it replaced that line with explicit waypoints along the west side, climbed onto the balcony, crossed the corridor and reached the upper room. That was real progress, but it arrived too late for the final sequence.

Effort and outcomes

Each effort level had three attempts and zero finishes. The furthest run was at max effort; the other two max attempts stopped much earlier. Effort did not produce a consistent progression through the castle.

Moments from the runs

The staircase declared impossible

It tried full-angle sweeps, height and floor checks, jumping, alternate movement inputs and backtracking. It repeatedly concluded that the staircase could not be climbed with the available tools, then continued probing. The scorer recorded the castle hall, but no upper balcony.

high effortRun: v3-opencode-ds4flashvision-high-tortle-003

The wrong room became the objective

A mistaken room sequence led it to the guard who opens Peach's bedroom. It pursued the conversation, entered the room and briefly called it a dead end, then returned to the idea that the story scene should happen there. It never reached the upper balcony gate.

low effortRun: v3-opencode-ds4flashvision-low-mander-003

Waypoints broke the impasse

A direct line to the upper floor failed against the room's walls. Replacing it with west-side waypoints produced an actual climb and an onward door transition. It reached the upper room at 86.1 minutes, then prepared the final scene script without reaching Bowser before the cap.

max effortRun: v3-opencode-ds4flashvision-max-mander-002
Scope and limitations
  • The model's claims that routes were impossible are descriptions of its diagnosis, not accepted findings about the game.
  • This is the experimental vision model's nine-run September sample, not a statement about the later DeepSeek model.
Command counts and effort groups

Received controller commands, including harness traffic; these are requests rather than confirmed actions or individual model tool calls. Successful runs end sooner.

Medians across this model's runs. 9 of 9 runs recorded a load request.

Save requests
11
Load requests
57
Screenshots
151
Input burst
10 frames
September effort and harness conditions
Effort / CLIFinishedSave medianLoad median
lowopencode · 16ba6b60/3985
highopencode · 16ba6b60/225.583
highopencode · 6ffc4780/150203
maxopencode · 16ba6b60/3717
What these logs can and cannot measure
  • September only: 66 counted ordinary attempts; August observations are a separate service and harness epoch.
  • The counted Gemini 3.8 Flash rule-violating run remains in the denominator; campaign integrity overrides raw scorer events.
  • Command counts are received command verbs, not acknowledgements of successful execution or model tool-call counts. Scorer/final-capture traffic is not separately tagged.
  • RAM counts include external scorer reads. Agent-attributed memory use and TAS-engine execution counts are unknown; raw frame stepping does not establish use of the TAS engine.
  • Burst frames sum requested STEP_FRAME frames between SET_INPUT_PORT boundaries, ending at CLEAR_INPUT_PORT; frames are not seconds. Model summaries are medians of run medians.
  • A map revisit is an observed transition back to a previously sampled map; polling can miss short visits, and backtracking can be appropriate.
  • Stationary episodes have identical sampled map and position for at least 12 seconds, across positive poll gaps no greater than 10 seconds. They do not prove inactivity, confusion, or a stalled agent; menus, cutscenes, reasoning and paused game time can all qualify.
  • Native delegation counts use distinct Claude Agent/Task and OpenCode task/subagent invocation IDs. They exclude delegation hidden inside Workflow or shell scripts. Codex item streams and AGY prose do not support this census and remain unknown; absent tool inventory is not proof that delegation was unavailable.
  • Different effort levels, routes and harness commits remain in condition aggregates. Descriptive model medians are not causal capability comparisons; completed runs also end sooner.
Every attempt · scorer clock
  • highstopped · S1u
  • highstopped · S1
  • highstopped · S1
  • lowstopped · S1
  • lowstopped · S1
  • lowstopped · G3
  • maxstopped · S3
  • maxstopped · G3
  • maxstopped · S1

GLM-5.3-Flash

0 / 9 G4September 2026

All nine attempts reached the castle, but none finished. Some made late navigation breakthroughs; one spent most of its run rereading the same captured images.

How it played

The expanded set shows several ways to lose the same 90-minute budget. Runs confused a playable castle scene with an intro that still needed to finish, spent a long time researching the route before moving, or built position readers that needed repeated repairs. Having coordinates available was not the same as interpreting them correctly.

There were useful recoveries. In a new low-effort run, direction probes gave way to a systematic crawl along the left wall. Returned position bytes showed Mario gaining height, and a revised diagonal input carried him to the balcony. It reached the next corridor at 67.0 minutes, then backtracked and kept trying doors without reaching the upper room. The original low-effort run still went furthest, reaching that upper room just before the limit.

Another low-effort run took a very different path. After launching a script to advance the party scene, it repeatedly reopened the same saved images. Two files accounted for 768 distinct read requests. These were reads of existing captures, not fresh screenshots or new controller actions. The external scorer recorded no progress beyond the castle.

Effort and outcomes

Low, high and max now each have three attempts and zero finishes. The furthest low run reached the upper room; high reached the upper balcony, and max reached the corridor. The expanded sample still does not show a consistent improvement with more effort.

Moments from the runs

The intro had already ended

Early mashing had already created the save and reached the party. After treating the scene as a stuck intro, it recognized that Mario was controllable and resumed navigation. It reached the upper room at 88.1 minutes, but not Bowser.

low effortRun: v3-opencode-glm53flash-low-mander-001

Correcting the measuring instrument

Its position helper repeatedly changed as it investigated byte order and candidate addresses. When the authoritative address still appeared wrong, it checked the game version and discovered that it had added the position offset incorrectly. It eventually entered the castle hall at 83.4 minutes.

high effortRun: v3-opencode-glm53flash-high-tortle-002

The map correction helped, late

It first identified the castle using the wrong area names. After correcting that interpretation, it found the east staircase, reached the balcony and opened the onward door. The scorer recorded the corridor at 79.7 minutes; the run ended there.

max effortRun: v3-opencode-glm53flash-max-saur-001

Finding the climb along the wall

After repeated direction probes, it reloaded the hall entrance and alternated left and up inputs along the wall. Position bytes showed the first gain in height. A revised diagonal input then reached the balcony, and the scorer recorded the next corridor at 67.0 minutes. Later backtracking and door attempts did not produce a finish.

low effortRun: v3-opencode-glm53flash-low-saur-002

Reading the same two pictures

After launching a party-scene script, it kept reading previously captured images. Two files were opened 384 times each across roughly 73 minutes. The remaining native tool stream contains image reads rather than further navigation, and the scorer never recorded the castle hall.

low effortRun: v3-opencode-glm53flash-low-tortle-001
Scope and limitations
  • This profile covers nine counted September attempts, including six added in the latest intake. Three runs per effort remain a small sample.
  • The repeated-image case is a verified tool sequence, not a diagnosis of why the model entered the loop. The screenshot files were not re-rendered for this analysis.
Command counts and effort groups

Received controller commands, including harness traffic; these are requests rather than confirmed actions or individual model tool calls. Successful runs end sooner.

Medians across this model's runs. 4 of 9 runs recorded a load request.

Save requests
2
Load requests
0
Screenshots
83
Input burst
2 frames
September effort and harness conditions
Effort / CLIFinishedSave medianLoad median
lowopencode · 13cdf300/29.531.5
lowopencode · f5f1a090/130
highopencode · 13cdf300/21.51
highopencode · 91054e50/155
maxopencode · 13cdf300/261
maxopencode · f5f1a090/120
What these logs can and cannot measure
  • September only: 66 counted ordinary attempts; August observations are a separate service and harness epoch.
  • The counted Gemini 3.8 Flash rule-violating run remains in the denominator; campaign integrity overrides raw scorer events.
  • Command counts are received command verbs, not acknowledgements of successful execution or model tool-call counts. Scorer/final-capture traffic is not separately tagged.
  • RAM counts include external scorer reads. Agent-attributed memory use and TAS-engine execution counts are unknown; raw frame stepping does not establish use of the TAS engine.
  • Burst frames sum requested STEP_FRAME frames between SET_INPUT_PORT boundaries, ending at CLEAR_INPUT_PORT; frames are not seconds. Model summaries are medians of run medians.
  • A map revisit is an observed transition back to a previously sampled map; polling can miss short visits, and backtracking can be appropriate.
  • Stationary episodes have identical sampled map and position for at least 12 seconds, across positive poll gaps no greater than 10 seconds. They do not prove inactivity, confusion, or a stalled agent; menus, cutscenes, reasoning and paused game time can all qualify.
  • Native delegation counts use distinct Claude Agent/Task and OpenCode task/subagent invocation IDs. They exclude delegation hidden inside Workflow or shell scripts. Codex item streams and AGY prose do not support this census and remain unknown; absent tool inventory is not proof that delegation was unavailable.
  • Different effort levels, routes and harness commits remain in condition aggregates. Descriptive model medians are not causal capability comparisons; completed runs also end sooner.
Every attempt · scorer clock
  • highstopped · G3
  • highstopped · S1u
  • highstopped · S1
  • lowstopped · S3
  • lowstopped · S2
  • lowstopped · G3
  • maxstopped · G3
  • maxstopped · S2
  • maxstopped · S1

Opus 5

12 / 12 G4ram-led navigation · screenshot-led · walkthrough-following

The metronome. Every attempt finished, the fastest median in the cohort, and the shortest worst-stalls of any major. Reads RAM, checks a screenshot, moves on.

How it played

Across its twelve attempts, all of which reached the Bowser battle, opus-5 barely spoke. Run after run opens with the same flat sentence about reading the guide and then goes almost silent, with intent carried by terse command labels such as a door sweep, an eight-direction sweep, or a mash toward the battle. What it did instead of talking was build. Most runs replaced ad hoc input with helper scripts: a closed-loop walker that compensated for camera-relative sticks, yaw-calibrated coordinate navigators, wall sweeps, and battle-context pollers that stopped the instant RAM flipped from world to battle. Savestates followed the same engineering instinct. Eight of the twelve attempts logged no load at all and used saves purely as insurance, while four treated a fixed slot as an experimental control, restoring an identical position before each candidate heading and comparing player height. Stalls were answered by widening the search rather than by reflection, sweeping one wall and then another until a door finally gave.

“Closed-loop walker: compensates for camera-relative stick mapping.”

from the session log of v3-opus5-high-mander-001

“10-kkj01 leg1 STALLED at d=71.9”

from the session log of v3-opus5-xhigh-saur-002

In the family

Compared with opus-4.8, opus-5 says much less and scripts much more, and in this sample it converts that into twelve completions out of twelve rather than ten. Fable-5 brings the spoken diagnosis back without matching that reliability, so the visible trade across the three generations is narration for automation.

Calling for backup

The tool inventory was identical to Opus 4.8's, sub-agents included. Opus 5 never touched them once in twelve attempts; the habit vanished in a single model generation.

Effort, and the walk to Bowser

Effort changed nothing about the outcome, since all three attempts at every level finished; the only visible pattern is that its quickest runs sat at low and medium while xhigh runs spent more of their time on front-loaded reading and script writing. It played no battle turns, instrumenting the encounter instead so that its dialogue blitz would halt the moment the battle-context field appeared, then checkpointing that state.

The small figure under each stat is the 78-completer cohort median.

Median finish
36:01
51:19
RAM reads per screenshot
80
58
Screenshots per 1k frames
2.1
3.6
Save/loads per 1k frames
0.41
0.51
Median input burst
3 frames
4 frames
Revisit ratio
0.200
0.200
Median worst stall
2:09
3:42
Every attempt · scorer clock
  • lowG4 in 26:52
  • lowG4 in 26:39
  • lowG4 in 36:37
  • mediumG4 in 24:10
  • mediumG4 in 28:43
  • mediumG4 in 42:38
  • highG4 in 55:13
  • highG4 in 54:53
  • highG4 in 35:24
  • xhighG4 in 54:49
  • xhighG4 in 33:40
  • xhighG4 in 43:13

Median finish by effort: high 54:53 · low 26:52 · medium 28:43 · xhigh 43:13 (cells with fewer than two completions get no median)

Behaves most like:

GPT-5.6 Sol

15 / 15 G4screenshot-led · ram-led navigation · decomp-guided routing

The other perfect record, on the largest sample in the cohort. Leans on screenshots more than the Anthropic models and saves state twice as often, but never once failed to reach Bowser.

How it played

Sol plays like a build-and-verify loop. In most of its fifteen runs the rhythm is to write one narrowly scoped TAS script for one bounded segment, run it, read the trace and the screenshot, state out loud what the map, position, input-lock or context byte now says, and only then choose the next input. It announces each scored milestone as secured rather than assumed. It is the most checkpoint-dense condition in this family, with three of its runs recording more than seventy savestates, and it usually treats those states as insurance instead of rewinds, since nine of the fifteen runs never issued a load at all. Where it does branch, it does so deliberately, probing four cardinal directions from one shared state to settle a question mechanically. When a batch outlived a shell timeout it checked the frame counter and concluded the run was healthy rather than hung.

“I’m branching four cardinal 30-frame probes from one savestate so the working departure direction is established mechanically.”

from the session log of v3-codex-gpt56sol-max-mander-001

“The staged batch is now at Bowser’s “It’s… Mario!!” line (`0053`), leaving exactly four dialogue calls before the battle flag.”

from the session log of v3r-gpt56sol-xhigh-saur-001

In the family

Sol reads as the disciplined version of the shared OpenAI loop, keeping the same read-the-source, steer-by-RAM approach as gpt-5.5 but pairing it with dense checkpointing and an explicit stop condition. It completed 15 of 15 attempts while luna went 4 of 15 and terra 5 of 15; the narrations suggest sol converts uncertainty into a bounded experiment quickly, whereas luna's logs show far more repeated visual probing in the same room. That is a reading of tendencies, not a proven cause.

Effort, and the walk to Bowser

Effort changed style rather than outcome: all three attempts at each of low, medium, high, xhigh and max completed. What visibly shifted is instrumentation, with the low-effort runs narrated as coarse and screenshot-led and recording as few as three savestates, while the max runs saved more than seventy times each. It refused to guess at the battle boundary, reading the pre-battle event source, counting the remaining dialogue calls, and guarding its final confirm loop on the game-context byte so input stopped exactly on the flip from world to battle.

The small figure under each stat is the 78-completer cohort median.

Median finish
50:10
51:19
RAM reads per screenshot
40
58
Screenshots per 1k frames
4.7
3.6
Save/loads per 1k frames
1.21
0.51
Median input burst
6 frames
4 frames
Revisit ratio
0.167
0.200
Median worst stall
2:58
3:42
Every attempt · scorer clock
  • lowG4 in 50:52
  • lowG4 in 24:36
  • lowG4 in 37:13
  • mediumG4 in 63:12
  • mediumG4 in 45:12
  • mediumG4 in 48:46
  • highG4 in 87:27
  • highG4 in 50:10
  • highG4 in 34:30
  • xhighG4 in 59:25
  • xhighG4 in 37:04
  • xhighG4 in 42:09
  • maxG4 in 85:33
  • maxG4 in 51:39
  • maxG4 in 52:07

Median finish by effort: high 50:10 · low 37:13 · max 52:07 · medium 48:46 · xhigh 42:09 (cells with fewer than two completions get no median)

Behaves most like:

Opus 4.8

10 / 12 G4ram-led navigation · screenshot-led · decomp-guided routing · spawns researchers

The RAM-hungriest major, about 123 reads per screenshot, and also the most temperamental: its runs vary more from each other than any other Anthropic model's.

How it played

Across its twelve attempts, ten of which reached the Bowser battle, opus-4.8 was the family's talker. The rhythm was a stated hypothesis, a bounded input batch, a RAM read, and then a sentence about what the numbers meant: a position pinned against a wall, a height stuck at zero beneath a landing, a coordinate unchanged across three chunks meaning the plan needed changing rather than repeating. It also audited itself, rejecting one savestate as a poor mid-transition checkpoint and, in another run, declaring that it had blown a room's budget before escalating to floor collider reads so it could tell stairs from floor mechanically. Savestates were genuinely operational here rather than decorative, with nine of its twelve attempts logging loads, including a reload to escape a guard dialogue loop and scratch slots used for eight-direction height probes. The cost was time. Castle geometry, guards, planters, couches, and a swinging camera absorbed most of every run, and its stall totals are the longest in the family.

“I've blown this room's budget. Let me escalate to a decisive instrument: read which floor collider Mario is standing on, so I can tell stairs from floor mechanically.”

from the session log of v3-opus48-low-tortle-001

“Rather than guess the approach angle, let me run a savestate-based 8-direction probe and pick whichever actually raises height.”

from the session log of v3-opus48-xhigh-tortle-001

In the family

This is the generation where the reasoning is loudest and the savestate use is most active, and also the slowest and the one that leaves two attempts short of Bowser. Opus-5 keeps the instrumentation but drops the commentary and the wandering, and fable-5 restores a compact version of the commentary while spending fewer reloads.

Calling for backup

Delegation is an opening ritual here. Eleven of twelve attempts spawned a researcher inside the first minutes, a background agent that read the decomp source for the RAM address map while the parent kept playing, and the addresses went straight into the telemetry probe. Later spawns fired at concrete blockers, and the reports earned their keep. One opened by telling the parent 'your main hypothesis is wrong' about a staircase that was actually a Toad guard's hitbox. The run took the correction.

Effort, and the walk to Bowser

Effort mainly moved speed rather than success: its three low attempts all finished but took roughly four thousand to five thousand scorer seconds, while its high and xhigh completions landed closer to two to three thousand, and its two failures sat at medium and xhigh, so no level was uniformly safe. It reached the fight without fighting it, advancing Bowser's dialogue in bounded A pulses while polling the game context field for the battle value, then checkpointing and screenshotting the confirmation.

The small figure under each stat is the 78-completer cohort median.

Median finish
53:34
51:19
RAM reads per screenshot
123
58
Screenshots per 1k frames
2.8
3.6
Save/loads per 1k frames
1.08
0.51
Median input burst
3 frames
4 frames
Revisit ratio
0.200
0.200
Median worst stall
4:05
3:42
Every attempt · scorer clock
  • lowG4 in 67:24
  • lowG4 in 82:59
  • lowG4 in 83:41
  • mediumG4 in 29:14
  • mediumstopped · S1 · castle hall
  • mediumG4 in 62:39
  • highG4 in 36:41
  • highG4 in 41:18
  • highG4 in 50:53
  • xhighG4 in 49:15
  • xhighstopped · S2 · stairs
  • xhighG4 in 56:16

Median finish by effort: high 41:18 · low 82:59 · medium 45:57 · xhigh 52:45 (cells with fewer than two completions get no median)

Behaves most like:

Fable 5

9 / 12 G4ram-led navigation · screenshot-led · walkthrough-following · spawns researchers

Reads the world more than it looks at it: about 92 RAM reads per screenshot, input bursts of two frames, among the smallest in the cohort, and almost no backtracking.

How it played

Across its twelve attempts, nine of which reached the Bowser confrontation, fable-5 narrated in short declarative bursts that named the obstacle and the fix in the same breath. It described the game in physical terms: a Toad absorbing a press, a cake table in the way, a pillar base pinning Mario, a camera that had rotated its stick axes. Nearly every run built RAM telemetry early and then reached for the decompiled map source the moment visual guessing failed, usually announcing the switch out loud rather than drifting into it. Savestates were mostly milestone insurance; six of its nine narrated runs never loaded one, and the runs that did spent a single reload retrying a hallway or castle route from a known entry instead of replaying the opening. Its most characteristic move was turning a stall into an experiment, restoring one scratch slot before six directional probes and comparing coordinates to see which heading actually gained staircase height.

“Let me stop guessing at the camera and probe systematically”

from the session log of v3-fable5-high-mander-003

“The guard is stealing my A presses (classic door-line NPC). Approaching the door from its right side instead.”

from the session log of v3-fable5-low-mander-001

In the family

Set beside opus-4.8, fable-5 keeps the habit of saying what went wrong but spends far less of the run saying it, and it reloads savestates much less often while still finishing most attempts. Against opus-5 it is the more talkative of the two and the less reliable in this sample, completing nine of twelve where opus-5 completed all twelve, though its narration makes its reasoning far easier to follow.

Calling for backup

One attempt of twelve delegated: the xhigh run sent two background readers into the decomp source during its opening minutes while it kept playing, and finished.

Effort, and the walk to Bowser

Effort visibly bought nothing here: all three low attempts and all three medium attempts reached Bowser, while two of three high and two of three xhigh attempts did not, with the failures stopping at Toad Town, the first castle hall, and the upper castle. It never fought Bowser, treating the encounter as a scripted destination reached by batched A presses while polling the RAM battle-context byte to confirm entry.

The small figure under each stat is the 78-completer cohort median.

Median finish
38:59
51:19
RAM reads per screenshot
92
58
Screenshots per 1k frames
2.3
3.6
Save/loads per 1k frames
0.26
0.51
Median input burst
2 frames
4 frames
Revisit ratio
0.200
0.200
Median worst stall
3:48
3:42
Every attempt · scorer clock
  • lowG4 in 44:09
  • lowG4 in 38:59
  • lowG4 in 26:32
  • mediumG4 in 68:59
  • mediumG4 in 43:43
  • mediumG4 in 38:47
  • highG4 in 33:32
  • highG4 in 62:06
  • highstopped · G2 · Toad Town
  • xhighG4 in 35:30
  • xhighstopped · S1 · castle hall
  • xhighstopped · S3 · upper castle

Median finish by effort: high 47:49 · low 38:59 · medium 43:43 · xhigh n=1 (cells with fewer than two completions get no median)

Behaves most like:

GLM 5.2

3 / 6 G4ram-led navigation · screenshot-led · walkthrough-following · spawns researchers

The looper. Long nine-frame input bursts, the loopiest completed run in the cohort (revisit 0.368), and heavy save/load traffic on the way to a slow finish.

Condition note: This endpoint takes no images, so sight came from two tools we built for it: a describe call backed by qwen3.6-35b and a LocateAnything-3B locator, served from a dedicated RTX 5090 vision host that never ran evals.

How it played

Across six attempts under the droid harness, three finished at the Bowser battle and three did not. Because its endpoint takes no images, every look at the screen went through a describe call over the screenshot, backed by a captioner and a locator on a separate vision host. The narrations show that rhythm plainly: an input or frame-step batch, a RAM read of position and map, a screenshot followed by a vision description, then a narrated correction. It issued roughly 6,000 to 8,000 core memory reads per run against 81 to 101 screenshots, so coordinates carried most of the load and pictures explained what coordinates could not. It was also the heaviest reloader among the conditions here that finished anything, with 7 to 25 loads against 6 to 10 saves per run, branching direction tests from a stable checkpoint instead of replaying rooms. Castle stairs and doors absorbed the bulk of every run, with 44 to 71 stall episodes and single stalls as long as 795 seconds. The three failures stopped at G3-castle, S1u-kkj01-upper and S2-kkj02-stairs, two of them idling more than 2,400 seconds after their last gate.

“So the whole run is really just the OPENING.”

from the session log of v3-glm52-high-metapod-004

“The stairs are on the LEFT side of the screen, not the right! They go from middle-left to top-left. With the camera mapping, LEFT = South-West. I've been going the wrong way the entire time!”

from the session log of v3-glm52-max-metapod-001

In the family

Within this group of six conditions it is one of only two that scored any completions, and it did so without image input at the model endpoint, working from described screenshots plus dense RAM telemetry. Its willingness to reload checkpoints separates it from the kimi-k3 droid runs, which placed savestates constantly and almost never loaded them.

Calling for backup

One attempt delegated, twice over: a blocking worker for RAM telemetry early on, then a second at the castle staircase. Both reports were transcribed into its route notes, and the run finished.

Effort, and the walk to Bowser

Three attempts at high yielded two completions and three at max yielded one, with median g4 times of 4,034 and 4,024 seconds respectively. Three runs per level is too thin to read anything into that difference. It treated the opening Bowser fight as a trigger target, watching the context value for the transition into battle rather than planning any combat.

The small figure under each stat is the 78-completer cohort median.

Median finish
67:04
51:19
RAM reads per screenshot
84
58
Screenshots per 1k frames
3.7
3.6
Save/loads per 1k frames
1.14
0.51
Median input burst
9 frames
4 frames
Revisit ratio
0.250
0.200
Median worst stall
4:52
3:42
Every attempt · scorer clock
  • highstopped · S2 · stairs
  • highG4 in 66:46
  • highG4 in 67:43
  • maxstopped · G3 · castle door
  • maxG4 in 67:04
  • maxstopped · S1u · upper hall

Median finish by effort: high 67:14 · max n=1 (cells with fewer than two completions get no median)

Behaves most like:

Kimi K3 · Droid

4 / 9 G4ram-led navigation · screenshot-led · walkthrough-following · spawns researchers

The most Anthropic-looking non-Anthropic profile: RAM-heavy, screenshot-light, its nearest behavioral neighbor is Opus 4.8. When it stalls, it stalls long.

How it played

Nine attempts produced four completions, and all four narration files for this model come from this harness. The signature behavior is instrumentation. Every narrated run derived RAM fields for area, map, position, yaw, input lock and battle context, and two of them wrote small helper scripts to turn raw state into navigation feedback, including a closed-loop navigator that corrected itself from measured position deltas. Screenshots were sparse and memory reads dense, ranging from 69 to 489 reads per screenshot across the nine runs. Savestates were placed but essentially never used: eight of nine runs recorded zero load commands and the ninth recorded one, so checkpoints acted as forward insurance while mistakes were corrected in place. The castle interior is where time went, with camera rotation, switchback stairs, doorframe ledges and a red Toad standing on the door line all producing pinned-coordinate stalls. The five non-completions stopped at G3-castle, S1u-kkj01-upper, or S1-kkj01-hall, four of them containing a single stall longer than 570 seconds.

“This manual navigation is burning lots of cycles.”

from the session log of v3-kimik3-high-tortle-011

“The camera is rotated ~24°, making mental math error-prone. Let me build a closed-loop navigator that self-corrects from measured movement deltas.”

from the session log of v3-kimik3-low-tortle-002

In the family

Holding the model fixed and changing only the harness changed the outcome completely: droid finished four of nine, cursor finished none of one, and opencode finished none of three. The two harnesses that never finished did not fail by playing badly so much as by barely playing, with opencode's runs stepping 8,100 to 9,200 frames against droid's 10,300 to 26,100. The near-absence of load commands held across all three harnesses, so that habit is the one thing the CLI swap did not disturb.

Calling for backup

Kimi's delegation habit travels with the model; the same decomp research brief shows up on every CLI it ran. On Droid the blocking calls returned cleanly, and the low-effort run that leaned on them finished.

Effort, and the walk to Bowser

Completions appeared at every effort level tested, two of three at low, one of three at high, one of three at max, and median completion time rose from 2,755 seconds at low to 3,059 at high to 4,732 at max. On three attempts per level that is suggestive at best, but nothing here shows more effort buying more completions. No combat menu or action-command play appears in any narrated run, because the target was battle entry rather than winning the fight, so it handled the lead-in by pulsing A through Bowser's dialogue while polling map and input-disabled state.

The small figure under each stat is the 78-completer cohort median.

Median finish
49:15
51:19
RAM reads per screenshot
106
58
Screenshots per 1k frames
2.6
3.6
Save/loads per 1k frames
0.28
0.51
Median input burst
5 frames
4 frames
Revisit ratio
0.222
0.200
Median worst stall
6:17
3:42
Every attempt · scorer clock
  • lowG4 in 44:19
  • lowG4 in 47:31
  • lowstopped · S1 · castle hall
  • highstopped · G3 · castle door
  • highG4 in 50:59
  • highstopped · S1 · castle hall
  • maxstopped · S1 · castle hall
  • maxG4 in 78:52
  • maxstopped · S1u · upper hall

Median finish by effort: high n=1 · low 45:55 · max n=1 (cells with fewer than two completions get no median)

Behaves most like:

GPT-5.5

5 / 12 G4ram-led navigation · screenshot-led · walkthrough-following

A middle path in every column. Its nearest neighbor is its successor Sol, with the same walkthrough-following, decomp-guided habits and about half the completion rate.

How it played

Across its twelve attempts, gpt-5.5 always wanted to read before it played. Every one of the five narrated runs opens the same way, working through the harness guide, the playbook, the TAS API, the opening walkthrough and the decompiled source before the emulator is even booted. Then it goes in by eye and gets thoroughly humbled by Peach's Castle: stairs it walks under, railings, columns, couches, and Toads standing directly on the door line. The signature moment in this family is the instrument upgrade partway through, where it stops trusting the camera and starts steering by Mario's coordinates in RAM. It is candid about its own inefficiency, narrating that its tiny press-and-release batches were costing more than they bought. Savestates were mostly banked rather than rewound in the successful runs; one narrated run reloaded a clean lobby state after deciding its position had gone bad.

“This has become a bad local position. I’m reloading the clean lobby checkpoint at slot 6”

from the session log of v3-gpt55-high-mander-001

“The shell loop is taking too long because it pays adapter startup overhead for every tiny press. I’m still letting it finish, and I’ll avoid this pattern for the rest of the run.”

from the session log of v3-gpt55-medium-tortle-001

In the family

gpt-5.5 sets the pattern the whole line inherits: study the documentation, play by screenshot, then convert to RAM and decompiled-source navigation once the castle geometry defeats visual steering. The three 5.6 variants all keep that loop, and what separates them is not the loop but the discipline around it, with sol completing 15 of 15 attempts while luna managed 4 of 15 and terra 5 of 15. Judged only from these runs, gpt-5.5 sits closest to terra, capable of the full route but inconsistent about reaching it inside the clock.

Effort, and the walk to Bowser

Effort mattered here: all three high-effort attempts completed, while none of the three low-effort attempts did, and those low runs also ended with the shortest sessions of the twelve. Medium and xhigh split one completion out of three each, so the middle of the range is too noisy to read confidently from twelve attempts. It treated the Bowser confrontation purely as a transition-control problem, shrinking its dialogue batches as the fight-start line approached and then withholding input entirely so the battle could load.

The small figure under each stat is the 78-completer cohort median.

Median finish
57:45
51:19
RAM reads per screenshot
76
58
Screenshots per 1k frames
2.3
3.6
Save/loads per 1k frames
0.45
0.51
Median input burst
7.5 frames
4 frames
Revisit ratio
0.177
0.200
Median worst stall
3:52
3:42
Every attempt · scorer clock
  • lowstopped · S1u · upper hall
  • lowstopped · G3 · castle door
  • lowstopped · G3 · castle door
  • mediumstopped · S2 · stairs
  • mediumstopped · S2 · stairs
  • mediumG4 in 57:45
  • highG4 in 84:07
  • highG4 in 65:50
  • highG4 in 41:39
  • xhighstopped · S1 · castle hall
  • xhighstopped · S3 · upper castle
  • xhighG4 in 44:29

Median finish by effort: high 65:50 · medium n=1 · xhigh n=1 (cells with fewer than two completions get no median)

Behaves most like:

GPT-5.6 Terra

5 / 15 G4ram-led navigation · walkthrough-following · screenshot-led

Ten failures, and half of them never got inside: the castle door is Terra's wall more than any other OpenAI model's.

How it played

Terra at its best is the most mechanically inventive condition in this family. Its higher-effort runs decode word-swapped RAM by hand, navigate by the live collision mesh, and run bounded savestate sweeps and a six-offset grid search to find a door plane rather than shove at it again. One run picked a purely mechanical success signal for a staircase, treating a rise in Mario's height from zero as proof of ascent instead of arguing with the camera. It also produced the family's sharpest stall diagnosis, proving a castle-lift scene was genuinely dead by watching its map variable never update, then reloading the pre-Peach checkpoint and replaying the sequence by hand. That care does not show up everywhere. Seven of its ten non-completions ended well short of the time ceiling, several at the castle gate with few stalls and light tool use, reading as short, thin attempts rather than long grinds.

“Telemetry shows the castle-lift scene’s map variable never begins updating, so this instance is genuinely stalled rather than merely rendering black.”

from the session log of v3-gpt56terra-high-tortle-001

“The collision sweep located the staircase’s true entrance. Mario is now on its first step (height rose from 0 to 10), so I can follow the measured ascending vector to the north door.”

from the session log of v3-gpt56terra-xhigh-tortle-001

In the family

Terra keeps the family's decomp-and-telemetry instincts and pushes them furthest, into collision meshes and checkpoint-backed vector searches, but only when it is running at higher effort. It completed 5 of 15 attempts against sol's 15 of 15 and luna's 4 of 15, and its failures split differently from luna's: luna's cluster in the upper hall after long grinding sessions, while most of terra's ended early and shallow. The rows suggest a consistency gap rather than a capability gap, though these runs alone cannot prove that.

Effort, and the walk to Bowser

Effort is the clearest signal for terra in this set: zero of three completed at low and zero of three at medium, against one of three at high and two of three at each of xhigh and max. The low and medium non-completions were also the shortest sessions, ending under about 2,800 seconds of session time. It handled Bowser as a trigger it did not want to overshoot, tapering from batched confirms to single presses separated by neutral verification frames while it watched the battle context flip.

The small figure under each stat is the 78-completer cohort median.

Median finish
60:23
51:19
RAM reads per screenshot
38
58
Screenshots per 1k frames
4.0
3.6
Save/loads per 1k frames
1.04
0.51
Median input burst
3 frames
4 frames
Revisit ratio
0.200
0.200
Median worst stall
3:29
3:42
Every attempt · scorer clock
  • lowstopped · G3 · castle door
  • lowstopped · G3 · castle door
  • lowstopped · G3 · castle door
  • mediumstopped · G3 · castle door
  • mediumstopped · S1u · upper hall
  • mediumstopped · G3 · castle door
  • highstopped · S2 · stairs
  • highstopped · S1u · upper hall
  • highG4 in 45:21
  • xhighstopped · S2 · stairs
  • xhighG4 in 44:14
  • xhighG4 in 60:36
  • maxstopped · S2 · stairs
  • maxG4 in 68:02
  • maxG4 in 60:23

Median finish by effort: high n=1 · max 64:12 · xhigh 52:25 (cells with fewer than two completions get no median)

Behaves most like:

Sonnet 5

4 / 12 G4screenshot-led · ram-led navigation · walkthrough-following · spawns researchers

The photographer. Almost nine screenshots per thousand frames and the fewest RAM reads per look of any Anthropic model, with savestate branching its narrations describe explicitly. Slow but occasionally through.

How it played

Across its twelve attempts, four of which reached the Bowser battle, sonnet-5 played by looking. Its screenshot counts are the highest in the family, with several runs past two hundred captures, and its rhythm was a small input batch, a picture, an interpretation, and another small batch. That served it well through the opening and broke down inside Peach's Castle, where the transcripts become long crawls: doors tried and retried, a sleeping Toad sitting on the door line, a kitchen route that turned out to be the wrong branch, pillars and bushes backed away from for a wider view. It was candid about the trouble, saying plainly that it had made confirmed progress and hit a genuine navigation wall, and its recoveries were concrete, reloading a courtyard checkpoint to go find the north double door or sweeping x offsets until the real door gap appeared. The pivot that helped most, when it came at all, was abandoning pixels and reading Mario's position out of memory.

“I've made confirmed progress and hit a genuine navigation wall.”

from the session log of v3-sonnet5-low-saur-002

“This blocked navigation is slow. Let me check the decompiled source for Mario's position address to build proper telemetry.”

from the session log of v3-sonnet5-xhigh-saur-002

In the family

As the smaller sibling in this set it shows the same instincts as the Opus line, decompilation lookups, RAM telemetry, savestate branching, but arrives at them later in each run and after much more visual trial and error. Where opus-5 finished all twelve and opus-4.8 ten, sonnet-5 finished four, and eight of its attempts ended at or inside the castle rather than at Bowser.

Calling for backup

Sonnet 5 delegates under pressure. Nine of twelve attempts spawned researchers, 28 spawns in all, and nearly every spawn past the halfway mark is stuck-recovery aimed at the castle chokepoint. One low-effort run handed the whole game to a single background agent in its first minute and yielded; the harness's background-wait ceiling ended that run inside ten minutes.

Effort, and the walk to Bowser

No effort level rescued it: its completions are scattered across low, high, and xhigh with none at medium, and its longest single stall and its highest stall total both occur at xhigh, so the sample shows effort changing where it got stuck more than whether it got stuck. It showed no combat technique at all, advancing the Peach and Bowser cutscenes with repeated A presses and neutral gaps until the screen faded into the battle transition.

The small figure under each stat is the 78-completer cohort median.

Median finish
76:52
51:19
RAM reads per screenshot
31
58
Screenshots per 1k frames
8.8
3.6
Save/loads per 1k frames
0.32
0.51
Median input burst
5 frames
4 frames
Revisit ratio
0.211
0.200
Median worst stall
5:36
3:42
Every attempt · scorer clock
  • lowstopped · S1u · upper hall
  • lowG4 in 85:17
  • lowstopped · G1 · playground save
  • mediumstopped · G3 · castle door
  • mediumstopped · S1 · castle hall
  • mediumstopped · S1u · upper hall
  • highstopped · G3 · castle door
  • highG4 in 76:48
  • highG4 in 76:56
  • xhighstopped · S2 · stairs
  • xhighstopped · S3 · upper castle
  • xhighG4 in 68:40

Median finish by effort: high 76:52 · low n=1 · xhigh n=1 (cells with fewer than two completions get no median)

Behaves most like:

GPT-5.6 Luna

4 / 15 G4screenshot-led · ram-led navigation · walkthrough-following

The upper hall is Luna's trap: six of eleven failures die at S1u, the most concentrated failure point of any condition. Uses savestates least of the OpenAI family.

How it played

Luna is the one that keeps looking. Its runs carry the heaviest screenshot counts in this family, up to 269 captures in a single non-completing attempt, and the loop is usually input, neutral frames, screenshot, narrate, try a slightly different angle. It is also the most savestate-averse of the four, with five of its fifteen runs never issuing a single save, so when a probe went wrong there was often nothing to fall back to. The stuck place is the upper castle hall: six of its eleven non-completions ended at that gate, and those rows show two thousand to three thousand seconds parked in the hall map with stall episodes running past five hundred seconds. Its completed runs narrate the same room's obstacles, a guard whose interaction radius steals the door press and reopens dialogue on contact, and a wraparound landing that pins Mario against a wall.

“I’ve confirmed the castle’s main stairs are navigable, but the upper landing is a wraparound corridor and the first long push pinned against its wall.”

from the session log of v3-gpt56luna-high-saur-001

“The guard’s interaction is taking priority over the door because Mario is still inside the NPC trigger.”

from the session log of v3-gpt56luna-xhigh-saur-001

In the family

Luna shares the family's reflexes, reading the decompiled exit scripts and switching to RAM position when the camera misleads it, but it commits to those tools later and less consistently than sol does. It completed 4 of 15 attempts against sol's 15 of 15, and nine of its eleven non-completions ran all the way to the roughly 5,370-second session ceiling, so these were runs that kept trying rather than runs that quit. The non-completions carry no narration, so the link between the upper-hall guard described in its completions and the failures at that gate is an inference from dwell and stall rows.

Effort, and the walk to Bowser

No clean effort gradient shows up here: across fifteen attempts it completed none of three at low, one of three at medium, two of three at high, one of three at xhigh, and none of three at max. With three runs per level that pattern is too thin to call anything but noisy. Its battle handling was the loosest of the four, mostly mashing confirm through the lead-in and, in one run, pressing A and then repeated B after entry with no narrated action-command plan.

The small figure under each stat is the 78-completer cohort median.

Median finish
78:00
51:19
RAM reads per screenshot
41
58
Screenshots per 1k frames
4.6
3.6
Save/loads per 1k frames
0.08
0.51
Median input burst
4 frames
4 frames
Revisit ratio
0.214
0.200
Median worst stall
5:16
3:42
Every attempt · scorer clock
  • lowstopped · S1u · upper hall
  • lowstopped · S1u · upper hall
  • lowstopped · G3 · castle door
  • mediumG4 in 82:08
  • mediumstopped · S1u · upper hall
  • mediumstopped · S2 · stairs
  • highG4 in 73:52
  • highG4 in 47:33
  • highstopped · S1u · upper hall
  • xhighG4 in 84:31
  • xhighstopped · S2 · stairs
  • xhighstopped · S4 · Bowser approach
  • maxstopped · S1u · upper hall
  • maxstopped · S2 · stairs
  • maxstopped · S1u · upper hall

Median finish by effort: high 60:42 · medium n=1 · xhigh n=1 (cells with fewer than two completions get no median)

Behaves most like:

Grok 4.6

3 / 12 G4ram-led navigation · screenshot-led · walkthrough-following

The heaviest screenshot rate of any completing condition plus near-constant save/load traffic. Looks hard, saves often, still usually stops in the castle hall.

How it played

Twelve attempts and three finishes, at 71:57, 69:32, and 75:55. It is the most screenshot-hungry of the completing conditions, a median of about 7.7 screenshots per thousand frames and one low run at 30.5, paired with constant save and load traffic, including 396 loads in a single xhigh run. The three narrated runs all open with research, reading the walkthrough, the TAS API, and decompiled sources before touching the emulator, and one of them corrects its own scope early after assuming the target was the endgame Bowser fight. Castle navigation absorbs most of the run in every case: camera-relative stick directions that shift as Mario moves, a star door that ignored A presses until the agent realized it was pressing before Mario was pushing into the wall, and a hunt for side staircases guided by collision reads. Slots are used as checkpoints and scratch branches, and at least one run corrupted its own good slot in an unstick loop.

“The interact prompt appeared, but A was pressed before Mario was pushing into the wall.”

from the session log of v3-cursor-grok46-high-mander-002

“We're in Peach's Castle with control. Next I'll calibrate movement and route through the halls to Peach, then Bowser.”

from the session log of v3-cursor-grok46-high-saur-004

In the family

Between the two xAI conditions here, 4.6 finishes more often, three of twelve against one of nine, and looks at the screen far more, with a median screenshot rate roughly double that of 4.5. Both share the same bottleneck, camera-relative movement in the first castle hall, and both lean on savestates as checkpoints, though 4.5 does so more compulsively.

Effort, and the walk to Bowser

Across three runs at each of low, medium, high, and xhigh, the completions came two at high and one at medium, with none at low or xhigh, which is suggestive but well inside what three runs per cell can produce by chance. Because the scored battle is entered by a scripted scene, the narrated runs spend their effort on spaced A presses and memory checks until the context flips to battle, and none attempt offensive action commands.

The small figure under each stat is the 78-completer cohort median.

Median finish
71:58
51:19
RAM reads per screenshot
71
58
Screenshots per 1k frames
7.7
3.6
Save/loads per 1k frames
3.79
0.51
Median input burst
9 frames
4 frames
Revisit ratio
0.211
0.200
Median worst stall
4:59
3:42
Every attempt · scorer clock
  • lowstopped · S1 · castle hall
  • lowstopped · S1 · castle hall
  • lowstopped · S1 · castle hall
  • mediumG4 in 75:55
  • mediumstopped · G3 · castle door
  • mediumstopped · S2 · stairs
  • highstopped · S4 · Bowser approach
  • highG4 in 71:58
  • highG4 in 69:32
  • xhighstopped · S1 · castle hall
  • xhighstopped · S2 · stairs
  • xhighstopped · S1 · castle hall

Median finish by effort: high 70:45 · medium n=1 (cells with fewer than two completions get no median)

Behaves most like:

Gemini 3.7 Flash

2 / 9 G4background task automation · stepped navigation scripts · batched cutscene and dialogue advancing

The first Gemini Flash to finish. It launches background navigation tasks and grinds chokepoints by small variation: one completer rephrased the same ballroom-staircase climb eleven times, the other ran counted, timed dialogue cycles on Peach's guard and scanned game memory for the map id when the door would not open.

How it played

Nine attempts and two finishes, at 71:28 and 69:10, the first completions in this Flash line. Both completer narrations run on launched tasks: start a background navigation or dialogue batch, wait, review the captures, then adjust. The medium completer narrates in terse one-line status updates and beat the ballroom staircase by rephrasing the same climb eleven lines in a row; the high completer is more deliberate, checkpointing at Mario's house and the top-floor couch, running counted, timed conversation cycles on the guard outside Peach's chamber, and scanning game memory for the map identifier when the door would not open. The seven failures all reach the castle: two stop on the approach, and five get inside, one in the first hall, one on its upper landing, two in the stairs room, and one on the Bowser approach, two rooms from the battle. Several failed runs then idle for most of an hour past their last gate.

“Climbing the ballroom stairs up to the balcony double doors.”

from the session log of v4-agy-gemini37flash-medium-mander-001

“I am searching game memory for the current map identifier and player state to map the exact scene and trigger. I will check the findings once the scan finishes.”

from the session log of v4-agy-gemini37flash-high-mander-002

In the family

The Flash line in this cohort moves from zero completions at 3.5 and zero at 3.6 to two of nine at 3.7, and the failure shape shifts with it, from stalling outside the castle to stalling inside it. Gemini 3.1 Pro sits apart from that line: it is the only Google condition here with a completion before 3.7, but its six runs scatter across five different stopping depths rather than settling into one profile.

Effort, and the walk to Bowser

The two completions came one at medium and one at high, with none at low, but at three runs per level that is too thin to order the efforts. Both completer narrations reach and trigger the prologue Bowser battle but record no attacks, defensive timing, or action commands, so the battle method cannot be read from these logs.

The small figure under each stat is the 78-completer cohort median.

Median finish
70:19
51:19
RAM reads per screenshot
22
58
Screenshots per 1k frames
6.1
3.6
Save/loads per 1k frames
0.54
0.51
Median input burst
4 frames
4 frames
Revisit ratio
0.222
0.200
Median worst stall
1:59
3:42
Every attempt · scorer clock
  • lowstopped · G3 · castle door
  • lowstopped · S1u · upper hall
  • lowstopped · S1 · castle hall
  • mediumG4 in 71:28
  • mediumstopped · G3 · castle door
  • mediumstopped · S2 · stairs
  • highstopped · S2 · stairs
  • highG4 in 69:10
  • highstopped · S4 · Bowser approach

Behaves most like:

Gemini 3.1 Pro

1 / 6 G4screenshot-led · scripted frame-stepping · camera-relative navigation

One clean screenshot-led completion and five stops at five different depths. Too scattered to call a tendency beyond inconsistency itself.

How it played

Six attempts, one of them a clean finish in 47:53, and five stops at five different depths: file creation, the castle exterior, the first hall, the upper hall, and the stairs. The completing run is the tidiest picture available, and it is the only run in this condition with a narration file. Its narration is screenshot-led rather than memory-led, with long scripted input batches for cutscenes, short four-direction probes when a wall stopped progress, and the discovery that the red carpet in the final hallway is a raised surface Mario had fallen off. The failures show the opposite habits in the derived rows. Three of the six runs never wrote a savestate at all, one burned 808 seconds in a single stall in the first castle hall, and another logged 74 loads against 13 saves with a revisit ratio of 0.806. The condition is consistent mainly in being inconsistent.

“The diagnostic showed that Mario was stuck against the wall under the stairs. Pushing down (towards the camera) successfully freed him onto the red carpet.”

from the session log of v3-gemini31pro-high-tortle-002

“Okay, looking closely at the screenshots, I realized the red carpet is actually a slightly raised platform, and Mario had fallen off of it onto the lower black-and-white tiles.”

from the session log of v3-gemini31pro-high-tortle-002

In the family

The Flash line in this cohort moves from zero completions at 3.5 and zero at 3.6 to two of nine at 3.7, and the failure shape shifts with it, from stalling outside the castle to stalling inside it. Gemini 3.1 Pro sits apart from that line: it is the only Google condition here with a completion before 3.7, but its six runs scatter across five different stopping depths rather than settling into one profile.

Effort, and the walk to Bowser

The six runs split three high and three low, and the single completion is a high run, which is too thin a sample to separate an effort effect from run-to-run variance. In the one narrated run no battle or action command is executed; the log ends with repeated A presses intended to carry the Peach and Bowser cutscene into the first battle.

The small figure under each stat is the 78-completer cohort median.

Median finish
47:53
51:19
RAM reads per screenshot
42
58
Screenshots per 1k frames
2.6
3.6
Save/loads per 1k frames
0.02
0.51
Median input burst
3 frames
4 frames
Revisit ratio
0.199
0.200
Median worst stall
2:19
3:42
Every attempt · scorer clock
  • lowstopped · G1 · playground save
  • lowstopped · G3 · castle door
  • lowstopped · S2 · stairs
  • highstopped · S1 · castle hall
  • highG4 in 47:53
  • highstopped · S1u · upper hall

Behaves most like:

Grok 4.5

1 / 9 G4ram-led navigation · screenshot-led · walkthrough-following

The savestate extremist: the highest save/load rate in the cohort, about 4.8 state actions per thousand frames. One finish; most failures stop in the castle hall.

How it played

Nine attempts and one finish, at 58:52. The defining number is state traffic: this condition runs the highest save and load rate in the 193-attempt set, a median near 4.8 state actions per thousand frames, and one low run logged 954 loads against 139 saves. The single narrated run shows what that traffic is for. It polls memory for area, map, position, and battle context, checks screenshots around uncertain transitions, and treats numbered slots as route checkpoints, saving at castle control and at each good stair landing and reloading whenever pathfinding drifted into a wall. It also caught itself assuming it had control while Luigi was still talking, and reloaded to clear the dialogue first. The failures are concentrated where that loop is most expensive: seven of the eight non-completions stopped inside the castle, most of them in the first hall or on the stairs above it, with worst stalls of five to seven minutes.

“Handed off mid-intro (Luigi still talking; stick cal invalid). Reloading and clearing dialogue before navigating.”

from the session log of v3-grok45-medium-tortle-001

“Pathfinding drifted east into a wall. Killing it and walking with corrected stick selection (favor `-Z`, correct X with left).”

from the session log of v3-grok45-medium-tortle-001

In the family

Between the two xAI conditions here, 4.6 finishes more often, three of twelve against one of nine, and looks at the screen far more, with a median screenshot rate roughly double that of 4.5. Both share the same bottleneck, camera-relative movement in the first castle hall, and both lean on savestates as checkpoints, though 4.5 does so more compulsively.

Effort, and the walk to Bowser

Three runs at each of low, medium, and high, with the only completion at medium and the heaviest state churn at low and medium, which is too thin to call an effort effect. The narrated completion confirms entry into the Bowser battle context after switching from held A to press and release pulses, but it plays no battle turns and attempts no action commands.

The small figure under each stat is the 78-completer cohort median.

Median finish
58:52
51:19
RAM reads per screenshot
79
58
Screenshots per 1k frames
3.4
3.6
Save/loads per 1k frames
4.84
0.51
Median input burst
10 frames
4 frames
Revisit ratio
0.222
0.200
Median worst stall
4:57
3:42
Every attempt · scorer clock
  • lowstopped · S2 · stairs
  • lowstopped · S3 · upper castle
  • lowstopped · S1 · castle hall
  • mediumstopped · S1 · castle hall
  • mediumstopped · G3 · castle door
  • mediumG4 in 58:52
  • highstopped · S1 · castle hall
  • highstopped · S1u · upper hall
  • highstopped · S2 · stairs

Behaves most like:

Gemini 3.6 Flash

0 / 9 G4

Five attempts ended at the castle door, and one stood still outside it for 80 minutes, the stationary-stall record the trace figure shows.

How it played

Nine attempts, no completions, and no narration files, so the account rests on derived rows, one of which produced no usable metrics at all. Five of the nine ended at the castle door. The signature failure is stationary rather than restless: one high run spent about 4,828 seconds, roughly 80 minutes, on the castle exterior map with essentially no gate progress after arriving, the stationary-stall record in this dataset, and two more runs sat outside for a comparable stretch. Across the condition the median tail after the last gate is over 76 minutes. Revisit ratios stay low, near 0.25, so these runs were not looping, they were waiting. The profile is also the most screenshot-forward of the Google conditions, with 216 to 443 screenshots in several runs and about 23 memory reads per screenshot, and one low run pushed input bursts to a median of 180 frames. Two runs did get inside, reaching the upper room and the Bowser approach.

In the family

The Flash line in this cohort moves from zero completions at 3.5 and zero at 3.6 to two of nine at 3.7, and the failure shape shifts with it, from stalling outside the castle to stalling inside it. Gemini 3.1 Pro sits apart from that line: it is the only Google condition here with a completion before 3.7, but its six runs scatter across five different stopping depths rather than settling into one profile.

Effort, and the walk to Bowser

Three runs at each of low, medium, and high all failed, and the deepest run of the nine was a medium one, which is not enough separation to read an effort effect. No run reached a battle; the deepest gate across the nine attempts was the Bowser approach room, reached once at medium effort, with five attempts ending outside the castle door.

The small figure under each stat is the 78-completer cohort median.

Median finish
no median (n<2)
51:19
RAM reads per screenshot
23
58
Screenshots per 1k frames
3.0
3.6
Save/loads per 1k frames
0.14
0.51
Median input burst
3 frames
4 frames
Revisit ratio
0.250
0.200
Median worst stall
0:59
3:42
Every attempt · scorer clock
  • lowstopped · no gate reached
  • lowstopped · S1u · upper hall
  • lowstopped · G3 · castle door
  • mediumstopped · S4 · Bowser approach
  • mediumstopped · G3 · castle door
  • mediumstopped · G3 · castle door
  • highstopped · S3 · upper castle
  • highstopped · G3 · castle door
  • highstopped · G3 · castle door

Gemini 3.5 Flash

0 / 9 G4

Also walled at the castle door, but restless about it: the highest revisit ratio of any multi-attempt condition, and the only cohort model that ever fired WRITE_CORE_MEMORY.

How it played

Nine attempts, no completions, and no narration files, so everything here comes from the derived rows. Five of the nine stopped at the castle gate without getting inside, and the deepest any run reached was the upper room past the stairs, once. The failures were busy rather than frozen. This condition holds the highest revisit ratio of any multi-attempt condition in the 193-attempt set, 0.814 in one low run that also logged 43 map transitions across nine maps, and its save and load traffic is heavy and lopsided, including one run with 148 loads against 31 saves and another with 37 saves and 58 loads. Long dead tails are the other signature: four runs recorded more than an hour of wall clock after their last gate. It is also the only condition in the cohort recorded firing WRITE_CORE_MEMORY, a note carried in the deepdive audit rather than in the toolmix counters.

In the family

The Flash line in this cohort moves from zero completions at 3.5 and zero at 3.6 to two of nine at 3.7, and the failure shape shifts with it, from stalling outside the castle to stalling inside it. Gemini 3.1 Pro sits apart from that line: it is the only Google condition here with a completion before 3.7, but its six runs scatter across five different stopping depths rather than settling into one profile.

Effort, and the walk to Bowser

Three runs at each of low, medium, and high all failed, and while the low runs stalled less in total, they stopped no deeper, so effort does not separate anything in this condition. No run reached a battle; the deepest gate across the nine attempts was the upper castle room past the stairs, reached once, with five attempts ending outside the castle door.

The small figure under each stat is the 78-completer cohort median.

Median finish
no median (n<2)
51:19
RAM reads per screenshot
86
58
Screenshots per 1k frames
2.8
3.6
Save/loads per 1k frames
1.46
0.51
Median input burst
3 frames
4 frames
Revisit ratio
0.400
0.200
Median worst stall
2:47
3:42
Every attempt · scorer clock
  • lowstopped · G3 · castle door
  • lowstopped · G3 · castle door
  • lowstopped · G3 · castle door
  • mediumstopped · S2 · stairs
  • mediumstopped · S1 · castle hall
  • mediumstopped · G3 · castle door
  • highstopped · S1 · castle hall
  • highstopped · G3 · castle door
  • highstopped · S3 · upper castle

Kimi K2.7 Code

0 / 3 G4

Three attempts, three stops in the same castle hall, the most screenshot-hungry profile in the cohort, eleven per thousand frames.

How it played

Three attempts, all at high effort on the same host, none finished, and all three stopped at exactly the same place, S1-kkj01-hall. It played by eye more than any other condition here, taking 191 to 292 screenshots per run at 8 to 12 per thousand frames, with the lowest ratios of memory reads to screenshots in this group at 27 to 38. Two of the three leaned hard on rewinding, at 54 and 41 load commands, and made no headway regardless: the idle tail after the last gate ran 2,211 to 3,099 seconds in every run. With no narration files, this is a thin but uniform failure.

In the family

It is the only condition in this group whose runs all halted at the identical gate, and the most screenshot-driven of the six. It shares the droid harness with glm-5.2 and kimi-k3, both of which produced completions under it.

Effort, and the walk to Bowser

Only high effort was sampled, three attempts, so there is no effort comparison available. All three runs stopped at S1-kkj01-hall, well short of any battle.

The small figure under each stat is the 78-completer cohort median.

Median finish
no median (n<2)
51:19
RAM reads per screenshot
27
58
Screenshots per 1k frames
11.0
3.6
Save/loads per 1k frames
2.34
0.51
Median input burst
2 frames
4 frames
Revisit ratio
0.222
0.200
Median worst stall
4:21
3:42
Every attempt · scorer clock
  • highstopped · S1 · castle hall
  • highstopped · S1 · castle hall
  • highstopped · S1 · castle hall

Kimi K3 · OpenCode

0 / 3 G4spawns researchers

The stall champion: median longest stall of 37 minutes, and about 231 RAM reads per screenshot while going nowhere.

How it played

Three attempts, all at max effort, none finished. Two stopped at G3-castle and one at S1-kkj01-hall. The stall shapes are the most extreme of any condition reviewed here: single uninterrupted stalls of 2,227 and 3,287 seconds, and 4,682 to 5,011 seconds of stall in total per run. The activity counts agree, with 47 to 174 input commands, 15 to 45 screenshots and 8,103 to 9,192 stepped frames per run, the lowest frame totals in this group. Savestates were rare and loads nearly absent. No narration files exist, so the stalls can be described from the derived timings but not explained.

In the family

Same model as the droid and cursor kimi-k3 conditions with only the harness changed, and the change is stark: the droid runs stepped two to three times as many frames and produced four completions, while these three barely accumulated activity at all. The long frozen intervals, rather than wrong choices, are what these runs consist of.

Calling for backup

Kimi's standing research brief ran here as one synchronous task mid-run; the report arrived intact and its addresses went straight into the telemetry loop.

Effort, and the walk to Bowser

All three attempts sat at max effort, so this condition carries no within-condition effort comparison. Its deepest gate across three runs was G3-castle, reached twice, so no battle was reached.

The small figure under each stat is the 78-completer cohort median.

Median finish
no median (n<2)
51:19
RAM reads per screenshot
231
58
Screenshots per 1k frames
4.0
3.6
Save/loads per 1k frames
0.33
0.51
Median input burst
3 frames
4 frames
Revisit ratio
0.222
0.200
Median worst stall
37:07
3:42
Every attempt · scorer clock
  • maxstopped · G3 · castle door
  • maxstopped · G3 · castle door
  • maxstopped · S1 · castle hall

Kimi K3 · Cursor

0 / 1 G4spawns researchers

A single attempt that stopped in the castle hall. One run is a data point, not a profile.

How it played

One attempt only, at high effort, and it did not finish. It reached S1-kkj01-hall and then spent most of its budget not moving, accumulating 4,288 seconds of stall across 50 episodes with a single longest stall of 925 seconds. It read core memory 12,268 times against 51 screenshots, about 241 reads per picture, while issuing only 411 input commands and stepping 14,842 frames. It wrote three savestates and loaded none, matching the pattern seen under droid. There is no narration file for this run, so nothing about the reasoning behind those stalls is recoverable from this evidence.

In the family

This is the same underlying model as the droid and opencode kimi-k3 conditions, differing only in harness, and it is the thinnest of the three at a single attempt. Where droid converted similar telemetry habits into four completions, this run stalled for a quarter of an hour at a stretch inside the first castle hall and never got past it.

Calling for backup

The research brief Kimi sends on every CLI ran here too, but this harness backgrounded the researcher and the report never came back. The run redid the work itself fifteen minutes later.

Effort, and the walk to Bowser

A single attempt at a single effort level. There is no effort signal to read. Its one run stopped at S1-kkj01-hall, so no battle was reached.

The small figure under each stat is the 78-completer cohort median.

Median finish
no median (n<2)
51:19
RAM reads per screenshot
no median (n<2)
58
Screenshots per 1k frames
no median (n<2)
3.6
Save/loads per 1k frames
no median (n<2)
0.51
Median input burst
no median (n<2)
4 frames
Revisit ratio
no median (n<2)
0.200
Median worst stall
no median (n<2)
3:42
Every attempt · scorer clock
  • highstopped · S1 · castle hall

August profiles retain their original statistics and 78-finisher reference medians. September profiles use their own run counts and command summaries. Gate timing comes from the external scorer; the two periods are not pooled into a single behavioral baseline.

August 2026 · 193 attempts, 78 completions

The route is table stakes. The score is hesitation.

That's the strange thing the poll traces showed me. Every model that finishes Paper Mario Bench finishes it the same way: a median of 15 room transitions among the runs that finish, the same castle ladder, the same doors. As far as the completers' transition counts can show, there is one route through this game, and that turns the transition count into a mistake counter: the minimum path is the minimum, and every door-crossing above it is a wrong turn, a backtrack, or a re-derivation. What separates a 24-minute run from a 90-minute run isn't the route. It's hesitation, and how each agent works its way out of it.

Ten stops from the playground save to the Bowser fight. Read 01 to 10; panels are placed for reading order, not geography.

Opening zone · overworld

  1. G101Mario's house + playgroundplayground-save
  2. G202Toad Towntoad-town-entry
  3. →G303Castle approachcastle-approach-ext
  4. G304Castle doorthreshold

also dwelled: goomba-village-outskirts

Castle zone · interior · up from here

  1. S105Castle hallcastle hall
  2. S1u06Upper hallupper hall
  3. S207Stairsstair room
  4. S308Upper castleupper floor
  5. S409Bowser approachbowser approach
  6. G410Bowser battlefinal gate · battle

Failed runs dwell longest in the castle hall and on the castle approach.

scored checkpoint (G/S) · waypoint, not scoredReading aid for the trace plates · single route, read 01 → 10 · no run population; this is not a trace

The command logs

Two schools of play under identical trace shapes.

The routes converge; the methods don't. Same trace shapes, and the command logs still split into two schools of play.

School one · read the RAM

Ask the console what's true.

Step a few frames, read game memory, step again. Position, map, and battle state come straight from RAM; the screenshot is a sanity check, not the eyes. Inputs go out as short taps because the next read will say whether they worked.

  • 12.3 RAM reads per screenshot, median
  • 3-frame input taps
  • 2.4 screenshots per 1k frames

Favored by Anthropic models

School two · watch the screen

Look at the game like a player.

Screenshot, judge, commit. RAM barely gets read; the picture is the source of truth, taken far more often, and inputs go out in longer committed bursts before the next look.

  • 0.8 RAM reads per screenshot, median
  • 8-frame input bursts
  • 4.5 screenshots per 1k frames

Favored by OpenAI models

Ranking the 35 Anthropic and 29 OpenAI completions by RAM-reads-per-screenshot separates the families on 899 of 1015 pairs. That's overwhelming, with real counterexamples, and the loudest is Sonnet 5 itself: all four of its completers sense like OpenAI runs (0.2–0.9 reads per screenshot).

The RAM school's addresses have a supply line. 27 of the 193 runs spawned sub-agents, 61 spawns in all, and nearly every one ran the same errand, reading the decompiled source for RAM addresses and door coordinates while the main agent kept playing. The habit is lopsided: Opus 4.8 delegated in 11 of its 12 runs, Opus 5 in none of its 12 on an identical tool list, and Kimi K3 carried the same research brief across all three of its harnesses. It reads as temperament, not advantage; 13 of the 27 spawning runs finished. Not every harness offers the tool: the Codex CLI has none, so its models never had the choice, and the Antigravity logs cannot confirm either way.

12 traces, one route

Same route. Different pace.

Getting to the right room is only part of the task. These traces show where each run moved, doubled back, or spent time in one place. The September additions are the fastest completed opening runs for Astra and Fable 5.1, alongside the earlier August examples. They show what those runs achieved, rather than a typical run for either model. A pause in the trace can include dialogue, cutscenes, tool waits, or reasoning.

Selected examples, not typical runs. Collection dates and input methods differ; the human used a direct controller.

GPT-6 Astra · medium20:05Finished at Bowser battleSeptember 2026 · 125 whole-run observations
Fable 5.1 · medium21:58Finished at Bowser battleSeptember 2026 · 150 whole-run observations

Inside the castle

Castle gameplay rooms only. Cutscene contexts and input-disabled samples are excluded, including the house and castle-approach scenes.

Sampled gameplay timeGPT-6 Astra · mediumFable 5.1 · medium

Estimates sum gaps whose two consecutive endpoints show control enabled in the same room. They are observed segments, not total active play time; control can change between polls.

Castle upper hall

Both movement plots share one coordinate scale. Height over time appears below each plot. Lines stop at excluded samples and room changes.

GPT-6 Astra · medium
~2:18 sampled gameplay time1 observed segments18 position samples
-742-230282-474395519:20 · x 0, y 0, z 4809:28 · x 0, y 0, z 4809:36 · x -91.8, y 0, z 257.19:44 · x -91.8, y 0, z 257.19:52 · x -272.9, y 0, z 343.610:00 · x -272.9, y 0, z 343.610:07 · x -334.1, y 0, z 118.610:15 · x -334.1, y 0, z 118.610:24 · x -422.2, y 0, z -4410:32 · x -422.2, y 0, z -4410:41 · x -422.6, y 60, z -260.710:49 · x -422.6, y 60, z -260.710:58 · x -422.6, y 60, z -260.711:06 · x -400.9, y 80, z -293.111:15 · x -299.5, y 100, z -34411:22 · x -299.5, y 100, z -34411:31 · x -40.5, y 100, z -336.211:38 · x -40.5, y 100, z -336.2
Height after first controllable observation11100:003:01
1 repeated-position intervalControllable samples at the same position for at least 12 seconds, with no gap over 10 seconds. They do not establish why the character stayed still.
See intervals (run clock)
  1. 10:4110:58 0:17
Read position samples
SegmentRun clockxyz
19:2000480
19:2800480
19:36-91.80257.1
19:44-91.80257.1
19:52-272.90343.6
110:00-272.90343.6
110:07-334.10118.6
110:15-334.10118.6
110:24-422.20-44
110:32-422.20-44
110:41-422.660-260.7
110:49-422.660-260.7
110:58-422.660-260.7
111:06-400.980-293.1
111:15-299.5100-344
111:22-299.5100-344
111:31-40.5100-336.2
111:38-40.5100-336.2

Run sampled about every 8.2s (median).

Fable 5.1 · medium
~3:01 sampled gameplay time1 observed segments24 position samples
-742-230282-4743955110:27 · x -71.4, y 0, z 440.810:35 · x -71.4, y 0, z 440.810:43 · x -71.4, y 0, z 440.810:50 · x -200.9, y 0, z 39810:58 · x -265.7, y 0, z 376.611:06 · x -468.5, y 0, z 225.511:13 · x -468.5, y 0, z 225.511:21 · x -476.2, y 0, z 20.911:28 · x -487, y 10, z -60.311:36 · x -400, y 50, z -135.511:44 · x -382.8, y 50, z -170.211:51 · x -330.2, y 100, z -329.411:59 · x -297.3, y 100, z -396.712:07 · x -236.5, y 100, z -402.912:15 · x -236.5, y 100, z -402.912:23 · x -236.5, y 100, z -402.912:31 · x -83.4, y 100, z -38712:38 · x -83.4, y 100, z -38712:46 · x -83.4, y 100, z -38712:55 · x -83.4, y 111.1, z -38713:04 · x -83.4, y 100, z -38713:12 · x -83.4, y 100, z -38713:20 · x 27, y 100, z -380.113:28 · x 27, y 100, z -380.1
Height after first controllable observation11100:003:01
3 repeated-position intervalsControllable samples at the same position for at least 12 seconds, with no gap over 10 seconds. They do not establish why the character stayed still.
See intervals (run clock)
  1. 10:2710:43 0:16
  2. 12:0712:23 0:16
  3. 12:3112:46 0:15
Read position samples
SegmentRun clockxyz
110:27-71.40440.8
110:35-71.40440.8
110:43-71.40440.8
110:50-200.90398
110:58-265.70376.6
111:06-468.50225.5
111:13-468.50225.5
111:21-476.2020.9
111:28-48710-60.3
111:36-40050-135.5
111:44-382.850-170.2
111:51-330.2100-329.4
111:59-297.3100-396.7
112:07-236.5100-402.9
112:15-236.5100-402.9
112:23-236.5100-402.9
112:31-83.4100-387
112:38-83.4100-387
112:46-83.4100-387
112:55-83.4111.1-387
113:04-83.4100-387
113:12-83.4100-387
113:2027100-380.1
113:2827100-380.1

Run sampled about every 8.2s (median).

An open circle marks the first retained sample and a square the last. Unobserved movement stays unknown; these are not room maps.

Filtering, limitations, and original downloads

Room plots retain samples in the four castle gameplay rooms only when the recorded context is normal and input is enabled. Excluded samples break lines and observed segments. Each time estimate sums only intervals with two consecutive eligible endpoints in the same room; it is not total active play time. Control can change between polls.

Checkpoint arrivals come from the external scorer. Those timings and observed run totals still include the intro and cutscenes. Repeated-position intervals use consecutive retained samples no more than 10 seconds apart. The height charts share a scale and show elapsed time since the first retained observation, including gaps where samples were excluded. Sparse samples cannot establish a continuous path or the reason for a delay.

These original figures are unfiltered archival downloads and include cutscene positions:

Same model, two conditions

One reached the castle sooner. The other finished it.

Both conditions shown here reached the castle. The Cursor run got there in 13:18; the Droid run took 2.6× longer. Then they told opposite stories: the Cursor run stalled for over an hour at the first castle subgate, and the Droid (Factory, 60 TPS) run kept going through every retained subgate to G4. They're distinct harness conditions, so I report them side by side and never pool them. Four Droid-route K3 completions stand in the cohort.

Fast openingMoonshot AICursor CLIKimi K3 Cursor high
Reached G3 in 13:17.9, then spent 71:05.3 reaching the first castle subgate.
G3 → S1, then stopped
Slower opening, complete routeMoonshot AIDroid (Factory)Kimi K3 Droid high
Reached G3 in 34:01.7, then continued through every retained castle subgate to G4.
G3 → S1 → S1u → S2 → S3 → S4 → G4

These lines are explanatory diagrams of retained gate progression, not exact movement traces.

The failure spectrum

Failure has styles.

Endpoint scores collapse different behaviors into the same number:

Papercraft illustration: a sprint trail halts at a rigid figure before an untouched blue doorway.

The grinder

A Gemini 3.6 Flash run reached the castle at 10:16, solid pace, then stood at the approach for 79 minutes and 36 seconds until the cap. One room short of the hall, motionless, to the buzzer.

Papercraft illustration: a tomato ribbon loops repeatedly through nine rooms without reaching the sealed exit.

The wanderer

A Gemini 3.5 Flash run made 43 room transitions, three times what a completer needs, across only nine unique rooms: 81% of its door-crossings re-entered a room it had already seen. Lots of motion, no progress. It also died at G3.

Papercraft illustration: a diminishing trail quietly ends before a distant yellow step.

The quiet stopper

Sonnet 5 low completed 1 of its 3 attempts, at about 85 minutes. The interesting part is how the other two ended: no thrashing, no grinding, just moderate stalls (their median longest stall ~4 minutes) and a short ~8-minute median tail after the last gate. It just stopped, quietly. (Three runs, so an anecdote, not a ranking.)

Papercraft illustration: three repeated jump poses trigger a blank button while the yellow panel stays unresponsive.

The superstition

A Gemini 3.6 Flash run learned the opening's one true lesson, that pressing A advances everything (cutscenes, file select, naming), and never learned another. Its own plan says it: "advance through the opening cutscenes," then "trigger the opening story events." After reaching the castle steps it stood at a single point and jumped, pressing A in unbroken identical script-emitted bursts, 400 presses, then 900, for 39 minutes, position frozen to the decimal while its height ticked up and down in little jump arcs. Then it walked to a second spot and mashed A into a dialogue box for 30 more, and hit the time cap mid-burst, 693 presses into a sequence nothing was ever going to answer. The belief lives in the plan text, not in per-press deliberation; the bursts were scripted loops. But the world had stopped being a cutscene an hour earlier. Nobody told the agent. It is essentially a reward hack: the plan made pressing A the measure of progress, so the agent optimized the pressing instead of the progress.

Same benchmark, different problems. This is what the benchmark actually measures once the route is table stakes: what a model does when the next step isn't obvious. Three of these failures are drawn room by room in the trace comparison above: click through Gemini 3.1 Pro low, GPT-5.6 Terra low, and Fable 5 xhigh to see a run that never left the intro, one that stopped at the castle door, and one that died two rooms from Bowser.