RSIGame — Table Artifacts

Everything behind a published number: for each row of Table 1 and Table 2, the base game, the build that was actually scored, and the judge's own records for that build.

Nothing here is a summary. For every row, the per-task reward column of manifest.csv, averaged over the 140 benchmark tasks (a task without a reward counts as 0, as in the paper), is the Overall published in Table 1 or Table 2, and the per-family means are the appendix breakdown tables. All nineteen rows reproduce to the second decimal (last checked 2026-09-26).

Layout

<table>/<generator>/<method>/
    games.tar.gz      <task>/…            base project, or the build that was scored
    scores.tar.gz     <task>/p1..p3/      breakdown.json, judge_log.json, ctrf.json,
                                          reward.txt, build.log
                      <task>/summary.json the 3-pass rewards
    manifest.csv      task, reward, champion round, and where each came from
    games_addendum.tar.gz   (some rows) game trees added after the first release

Rows

tasks counts tasks with a reward; the rest of the 140 count as 0 in the paper (Phaser tasks that shipped no demo script, and one Phaser base game never scored).

champion = round 0 means the Global Quality Monitor never promoted a new build: development ran its budget and nothing beat the game it started from. The scored build for those tasks is the base game, and that is what the archive holds — not a missing file. 49 of the Qwen3.8-27B RSIGame tasks end that way, which is a result, not a gap.

What is deliberately not here

• Intermediate rounds. A thirty-round run is published at one version: the one the score was read at.
• Frames and videos. The judge watches recorded demos; those recordings are 2 MB a pass against 40 KB for the record that decided the score, and they replay from the published project. The judgment itself is here verbatim, including the rationale written for every rubric item.
• dist/ in Phaser projects. A build output; npm run build regenerates it from the src/ and public/ that are here. Keeping it doubled the Phaser rows.

Where the numbers come from, and one trap

The trap: r0_scores/<arm>.csv in the working tree is a stale by-product of a scoring run that failed (missing libXcursor, partial imports). For the Qwen arm it carries 18 zeros for games the rescored directory scores at 50 on average. The per-task directories are authoritative; that CSV is not, and nothing in this release was built from it.

Scoring protocol

Three independent replay-and-score passes by a Qwen3.8-27B judge against each task's own rubric (12–21 items, most often 18), Q = BUILD × (0.15M + 0.35D + 0.15V + 0.35A). judge_log.json holds one entry per (demo, rubric item) with the rationale — which is how one can see an item scored 0 because the recording never showed the screen it asks about.

Cleaning

1. Godot's .godot/ import cache is dropped — it stores the toolchain path of the machine that built it and regenerates on first open.
2. Every text member is rewritten: absolute machine paths, user and host names, internal cluster and project names, the model-gateway host, e-mail addresses, non-loopback IP literals.
3. Archive metadata normalised: every member user:user, uid/gid 0.
4. Each archive is re-scanned with the release patterns before it is written. Zero hits across all nineteen rows.

Game content, rubric text, judge rationales, rewards and timings are untouched.