H3 acceleration arena

Human-judged ranking of ~26 MiniMax-H3 acceleration variants over a 200-prompt corpus, from blind pairwise votes on pre-generated clips, with confidence intervals, cost and slice breakdowns. The design and its reasoning are in arena/DESIGN.md; the app's own notes are in arena/README.md.

This Space is private and must stay private until deliberately flipped. It streams ~3,700 clips out of the private dataset multimodalart/h3-pre-gen-arena. See Going public below.

Sign in with Hugging Face

hf_oauth: true above creates the OAuth app and injects OAUTH_CLIENT_ID, OAUTH_CLIENT_SECRET, OAUTH_SCOPES and OPENID_PROVIDER_URL. arena/space_auth.py implements the flow by hand (this is a Docker Space, so there is no Gradio integration to lean on): /login → https://huggingface.co/oauth/authorize → /login/callback → token exchange → userinfo, then an HMAC-signed session cookie. Endpoints are read from /.well-known/openid-configuration rather than hardcoded.

Every route is gated — voting and media alike. /media/ serves the private corpus, so it is not "just static files": it is exactly as sensitive as the vote endpoint and is gated exactly as hard. Only /login, /login/callback, /logout and /healthz are open.

The session cookie is SameSite=None; Secure, because a Space renders in an iframe on huggingface.co and a Lax cookie is simply not sent there — that is the single most common way this integration fails, and it fails as an infinite sign-in loop rather than as an error. The sign-in button opens in a new tab for the same reason.

Authorization: Bearer <hf_token> is accepted as an equivalent proof of the same identity. It is not a bypass — an access token proves exactly what the browser flow proves, and anyone holding one could complete that flow anyway — and it is what makes the durability guarantee testable without a browser. The Hub also puts a Bearer token on requests it proxies to a private Space, so resolving it to the real user beats showing a sign-in page to someone already past it.

Each vote carries the authenticated identity: userKey is hf:<sub> (stable across username changes), and h3.hfUser records the sub and the username. Anything the client sends as a user key is ignored. Votes used to be anonymous, which made one enthusiastic voter indistinguishable from consensus.

Votes survive a restart — the load-bearing part

A Space's filesystem is wiped on restart, sleep and rebuild, so the local fsync that is the right primitive on a real box buys nothing here on its own. arena/hub_votes.py:

• pulls the vote log from the dataset at boot, before VoteStore is constructed — the server's whole resume story is "replay votes.jsonl", and this is what makes that file complete on a machine that kept nothing;
• writes through on every vote, inside the request, before the response returns — not on a timer, so a Space killed one second after a vote has lost nothing;
• queues and retries loudly on a Hub failure: an fsync'd outbox, a backoff retry thread, the failure returned to the page as a red banner and reported by /api/stats and /admin.

Loudly on a FAILURE, and silent on a queue. Health is failing (the last push actually failed) or stuck (a vote unconfirmed past STUCK_AFTER_S = 45 s) — never "the queue is non-empty". The push is one whole-file commit measured at 1.4–3.8 s and it starts the moment the vote lands, so a queue depth of one for a few seconds is what correct operation looks like: the first version reported that as unhealthy and put a red "NOT yet saved to the dataset · Error: unknown" banner in front of a voter whose last push had returned ok: true, error: null, six votes uploaded. A banner that cries wolf on normal behaviour is worse than no banner. The error line renders only when there is an error, and nothing asks the voter to keep the tab open — the outbox is fsync'd server-side and the retry thread owns the upload, so closing the tab changes nothing.

Shards live at arena/votes/log/<session>.jsonl in the same private dataset — one whole-file commit per vote, so a partial write can never corrupt the remote copy, and a retry is idempotent because it always re-uploads the complete session. Deduplication at boot is on h3.matchupId.

Blind comparison

Clips are served under an opaque per-matchup token (/media/clip/<24 random bytes>) that maps to (prompt, variant) server-side; the mapping is revealed only in the vote response. /api/next carries no variant id at any depth, and a tripwire refuses to serve a matchup whose payload names either leg. selection and per-clip loudness are deliberately absent — the first announces a forced baseline-anchor pair, the second is a per-leg fingerprint with a 20 dB spread. The by-name media route is admin-only, because fetching every leg for a prompt and comparing Content-Length would undo the blinding.

Admin

multimodalart (ARENA_ADMINS) gets /admin: campaign state and durability health, every leg with its clip count and why it is or is not votable, ratings recomputed over task/emphasis/style/family/NFE/contract/leg filters with the honest n, every vote with its voter, per-voter aggregates, prompt search, and direct clip browsing that does not go through the matchmaker. Plus inspect mode in the voting UI: step through pairs and reveal them without voting — no impression recorded, no sampling counter moved, and the server refuses a vote on such a pair.

Configuration

Going public

Flipping visibility is one setting, and nothing in the app assumes a trusted viewer. These are the questions that change answer when it happens, listed rather than pre-solved:

1. Media egress. Any signed-in HF user could then stream the whole private corpus through /media/. The gate stops anonymous access, not enumeration by a signed-in account. Consider hf_oauth_authorized_org, or a per-session cap on distinct clips served.
2. Unbounded voting by one account. Nothing limits how many votes one identity casts. The per-voter panel makes it visible; it does not make it bounded.
3. Vote weighting. The fit treats every vote as equally informative. One account with 300 votes moves the table as far as 300 people agreeing.
4. Rate limiting. There is none. Each matchup fetches and normalises two clips on a cache miss, so a scripted client is a real cost.
5. Prompt text. context_ir — the full expansion fed to the model — is shown in the UI.
6. Session lifetime. 24 h, signed with OAUTH_CLIENT_SECRET; sessions die on a rebuild.