RigorPilot Skills

Run research repositories from their README, with bounded execution and auditable evidence. RigorPilot adds section-level results without rewriting the original README. Trusted reproduction is the default; candidate exploration requires explicit authorization.

English · 简体中文

Real examples · Install & use · Skill index · Validation · Engineering roadmap

📄 Real repositories, inspectable results

Original commands, prose, badges, images, videos and HTML stay in the source file. RigorPilot splits that file into sections and inserts one evidence-linked card per section. Removing its insertion blocks restores the retained original README byte for byte.

Each card below opens a full annotated README beside the original README in a retained repository checkout. Supporting repository files are kept so relative links and media retain their original context.

🟢 selected checks passed · 🔵 not executed · ⚪ read only · 🟡 partial · 🔴 blocked · 🟣 decision needed. Green does not automatically mean paper-result reproduction; blue is not an execution failure.

All four cases and upstream links · Recorded suite · Case definitions · Methodology

These are historical, commit-pinned deterministic runs: 4/4 case protocols passed in 251.0 s, with a peak workspace of 98.67 MiB and 0 model API calls. The zero-API count applies only to that suite. Selection-only and partial cases are not completed evaluations, converged training or reproduced paper scores.

New: installed-skill micrograd trial, with before/after command reports, a retained failed attempt and independent checks—not a model-quality comparison.

Current acceptance: explicit named-skill and fresh-client natural-language AUTO both pass end to end; earlier failures remain retained and A/B/model uplift remain unrun. Compact status · real-client evidence

🚀 Install and use

The installer needs Node.js/npm. Tested with [email protected] and Node 22.20.0; that installer requires Node ≥22.20.0. If you see EBADENGINE, check the requested version.

Install the self-contained reproduction skill:

npx skills add lllllllama/RigorPilot-Skills --skill ai-research-reproduction

Install all skills for the companion research entrypoints:

npx skills add lllllllama/RigorPilot-Skills --all

These commands use the current GitHub repository name.

Open the target repository in a Skills-capable agent, then ask:

Use ai-research-reproduction: run the smallest README-documented evaluation, preserve the source and write evidence to repro_outputs/, plus an annotated copy beside the original README. Ask before large downloads or long training.

The main skill works alone; choose all skills for companion and leaf entrypoints. Your existing agent loads the skill. The standalone model runner is optional. Client compatibility

Recommended first run:

Start with the RIGORPILOT_README.md reported in source_adjacent_readme.path, then follow its command and log links. If a conflicting file blocks the extra copy, that file stays intact; inspect repro_outputs/SUMMARY.md for the outcome and next action.

What it does—and does not do

README → documented target → reviewed setup → bounded execution → verification → evidence.

• Preserves source meaning; records assumptions, deviations, failures and blockers.
• Records process state, logs and attempt lineage; supports explicit cancellation, recovery and retry through the persistent runtime.
• Separates trusted reproduction from explicitly authorized, candidate-only exploration.
• Checks execution criteria independently of the model's completion claim.

This is local execution, not an OS sandbox. Approved commands can access the host and network; use trusted repositories. Resource admission and between-action budget checks are not hard OS quotas or subscription-balance monitoring.

The optional model loop currently supports Anthropic Messages and reviewed command IDs, not unrestricted source repair. This standalone runner has no successful live-model acceptance recorded yet: three provider attempts returned HTTP 502. Other model profiles are metadata, not proof of working transports or equivalent model performance. Runner and recovery contract · Implementation evidence and limits

🎯 Skill index

Two helpers support orchestration: repo-intake-and-plan and paper-context-resolver. Exploration requires a durable current_research anchor and a frozen comparison contract. Candidate results never become trusted baseline results by declaration. Routing · Research loop · Campaign inputs

📦 Evidence bundle

🟢 success · 🔵 not executed · ⚪ read only · 🟡 partial · 🔴 blocked · 🟣 decision required

Standard evidence stays under repro_outputs/. Both main runners accept --source-adjacent-readme to also write RIGORPILOT_README.md beside the original, preserving the context of its relative media/file links. Only inserted evidence links are rebased. The same output directory may refresh its unchanged owned copy, never an unrelated or manually edited file. Retain supporting repository files and the evidence directory's readme_delivery.json. Output contract · Rigor principles

For agent clients, the shortest normal path is --plan-only --agent-output, then run a returned cmd-XX with its --plan-fingerprint, then --verify-output --agent-output. The fingerprint binds execution to the reviewed README command set; setup/download commands are excluded from reproduction candidates. Routine success should not require reading orchestrator, bundle, writer, or runtime source. When --output-dir is omitted, the orchestrator writes to the target repository's repro_outputs/ directory, not the caller's current working directory. The evidence manifest detects later changes against the retained manifest; it is not a digital signature, external attestation or substitute for an OS sandbox.

Short tool-call hosts can opt in with --plan-only --include-agent-handoff to receive identity-bound start/status/cancel argv. Ordinary plans omit this extra control path. Short-call contract

✅ Offline validation

For normal development, run the high-signal core suite first:

python -m pip install -r requirements-dev.txt
python scripts/run_all_tests.py --core

Before commit/release, run the complete automatically discovered suite:

python scripts/run_all_tests.py
python scripts/check_publication.py

The latest version-bound counts and timing are in the acceptance ledger. Reports and complete per-script logs are written outside the repository by default; use --report PATH to choose a location. The core suite does not replace the full suite or CI.

For installed-layout functional acceptance without model calls:

python benchmarks/run_skill_acceptance.py --output tmp/skill-check

It covers positive/negative evidence behavior and can optionally include pinned micrograd when PyTorch/pytest already exist. Functional acceptance

For installation/environment problems (replace the skill path after installation):

python skills/ai-research-reproduction/scripts/doctor.py --repo /path/to/target

This read-only check reports the actual Python, Git, bundled-file integrity and README availability. Add --require-module torch --require-module pytest for dependency discovery; it does not install anything or execute target code.

Deeper checks are intentionally off the main path: benchmark methodology · agent runtime/fault evidence · paired A/B kit · controlled trials. Local tests do not substitute for live-model or held-out evaluation.

Engineering and contributions

Engineering roadmap · Contributing · Security and reporting · CI workflow · Reproduction feedback · MIT license

Keep acceptance checks independent, retain failed evidence and review traces before publication. Do not publish credentials or unreviewed private repository data. Agent guidance · Operating principles · Personalization policy

Historical interface illustration—not execution evidence

First attempt · After setup. This older MiniSeg preview illustrates error, metric and authorization displays. Its execution provenance is not independently verified; it is excluded from benchmarks.