Ouroboros on OSWorld-Verified: best published result on Claude Sonnet 4.6

Status: Self-reported result over all 361 task packages. Prompts, manifests, outcomes and feasibility records are public for every task. The official evaluator produced 360 score files; one unscored task is counted as zero.

Start here

Read the technical report, Browse all Ouroboros benchmark runs, inspect individual tasks without an API key, or read the public evidence page.

83.27% (300.59 / 361) on OSWorld-Verified with claude-sonnet-4.6, screenshot-only, one rollout and a 100-step budget. This repository contains all 361 task packages, including 360 official score files and the one unscored task that is counted as zero.

Result

Ouroboros leads every published Sonnet 4.6 system on OSWorld-Verified by +7.37 raw points over the next best, using a stricter base: our denominator is the full 361 tasks, while the other two rows are computed over 360 and 356.

Our companion run with claude-opus-5, 90.69% and the highest result reported on OSWorld-Verified to date, is at razzant/ouroboros-osworld-verified-opus5.

Competitor figures are read from the official leaderboard workbook osworld_verified_results.xlsx.

Per domain

Method

Ouroboros is a general-purpose autonomous agent runtime, not an OSWorld-specific scaffold. The benchmark adapter binds it to the OSWorld VM through a computer-use skill and hands scoring to the official evaluator.

• Task set: evaluation_examples/test_nogdrive.json, all 361 tasks in the standard set excluding the 8 Google Drive tasks. The evaluator produced 360 score files, and the remaining task is kept in the denominator as zero.
• Observation: screenshot only. No accessibility tree, no set-of-marks.
• Rollouts: 1 (pass@1). No best-of-N, no retry after scoring.
• Step budget: 100. One step = one policy turn = one agent call returning a batch of actions, matching lib_run_single.py, which increments step_idx once per predict() regardless of how many primitive actions that prediction emits. Steps used: median 28, p90 86, max 100. Zero tasks exceeded the cap; the audit is in every task's task_outcome.json under step_budget_audit.
• Feasibility pre-pass: before acting, a separate read-only task inspects the environment and returns INFEASIBLE / PROCEED / UNDETERMINED against a fixed rubric. It cannot modify the VM, its turns count inside the same 100-step budget, and the acting phase may override its verdict. OSWorld-Verified contains 27 infeasible-by-design tasks, and this makes that judgement explicit and auditable.
• Network: tasks whose OSWorld config sets "proxy": true ran through a residential proxy with a distinct session per task.
• Scoring: the official OSWorld evaluator at commit 091f5ef, unmodified. Our only changes to that checkout are in VM provisioning (desktop_env/providers/docker/provider.py: port-lock timeout, and one port scan per VM start instead of four); no task, evaluator or metric was touched.
• One task returned no score because of a fail-closed safety in our adapter and is counted as zero. The reported number is therefore conservative.

Contents

<domain>/<task_id>/
  result.txt                 official per-task score
  task_outcome.json          reward, status, step-budget audit
  ouroboros_task_final.json  acting task record; empty when no acting phase ran
  prompt.txt                 the exact prompt for that task
  feasibility_gate.json      pre-pass verdict and its tool trace
  reset_verification.json    evidence the task setup was verified
  task_run_manifest.json     provenance: model, effort, seed commit, dataset commit
results_summary.json         machine-readable scores, per-domain and per-task

The scaffold prompt is published for every task deliberately: it is what lets a reader judge whether the system is engineered against the evaluator or against the task.

Record coverage: prompt, manifest, outcome and feasibility files are present for all 361 tasks, while official score files are present for 360. The acting task record is non-empty for 341 tasks, an empty object for 19 feasibility-only tasks, and absent for the one fail-closed gate cancellation that returned no score and was counted as zero.

Not included: OSWorld task definitions (they carry evaluator.expected, i.e. the gold answers) and the evaluator's downloaded reference files. Publishing those alongside trajectories would turn this repository into an answer key.

Provenance and licensing

This is an evidence package produced by the Ouroboros v6.86.0 run identified above. viewer_index.jsonl is a derived index of the unchanged per_task records in results_summary.json; it exists only so the Hugging Face Dataset Viewer can display one stable row per task. The exact evidence snapshot remains 0e8ad516a4eeaa586607ead400429885814e7633.

The task identifiers, benchmark prompts and evaluator outputs originate from OSWorld at the pinned commit. OSWorld is Apache-2.0, as is the original material in this evidence package. The repository does not include OSWorld task definitions, gold answers or downloaded evaluator reference files. See THIRD_PARTY_NOTICES.md for the exact scope and upstream attribution.

Ouroboros

• Source: https://github.com/razzant/ouroboros
• This run: tag v6.86.0, commit a947a931

Citation

@misc{ouroboros_osworld_sonnet46,
  title  = {Ouroboros on OSWorld-Verified with Claude Sonnet 4.6},
  author = {Razzhigaev, Anton},
  year   = {2026},
  url    = {https://github.com/razzant/ouroboros}
}