Agent Lightning Starter Kit

Train ANY AI agent with reinforcement learning — zero code changes.

This is your fork-it starter kit for Microsoft's Agent Lightning v1.0, the 3,500-line lightweight agentic RL framework that's changing how teams train coding agents, search agents, and tool-using AI systems.

Why This Matters

Agent Lightning v1.0 just dropped (August 2026) with results that should make every AI engineering lead pay attention:

• Qwen3.5-9B on SWE-bench Verified: 41.8% → 56.4% (+14.6 points)
• Only 6K training samples needed
• Full repository test feedback in the loop
• Runs on Kubernetes natively
• Zero changes to your existing agent harness

If you're building AI agents that write code, search the web, query databases, or use tools — this is your training pipeline.

Quick Start

# Clone and install
git clone https://github.com/Varritech/agent-lightning-starter
cd agent-lightning-starter
uv sync

# Install verl GPU stack (CUDA 13.0 example)
bash scripts/setup_verl.sh 0.8.0 cu130

# Run the Calc-X POC (math reasoning with MCP calculator)
python examples/calc_x/train.py

That's it. You now have a working RL training loop for agents.

What's Inside

├── README.md              # This file
├── GUIDE.md               # Deep dive: architecture, training loop, production patterns
├── examples/
│   ├── calc_x/           # Math reasoning agent (MCP tools)
│   └── coding_agent/     # SWE-bench style code fixer
├── scripts/
│   └── setup_verl.sh     # GPU environment bootstrap
├── kubernetes/
│   └── agent-job.yaml    # K8s job spec for distributed rollouts
└── requirements.txt       # Core deps (uv managed)

Architecture at a Glance

Agent Lightning keeps things stupid simple with three components:

1. Trainer — Runs verl + vLLM, builds training samples, updates policy
2. API Gateway — Proxies model requests, captures trajectories
3. Rollout Controller — Launches agents locally or as K8s Jobs

Your agents talk to the Gateway like normal. Gateway records everything. Trainer turns it into RL gold.

Results That Matter

Full benchmarks in GUIDE.md and the technical report.

When to Use This

✅ You have an agent that uses tools (code exec, search, SQL, APIs)
✅ You can define a reward signal (tests pass, query runs, task complete)
✅ You want to improve success rates without manual prompt engineering
✅ You're running Qwen, Llama, Mistral, or any open model locally

❌ You only use closed APIs (no gradient updates possible)
❌ Your tasks have no clear success/failure signal
❌ You need >100K step training (consider distributed setup first)

Next Steps

1. Read GUIDE.md for full architecture breakdown
2. Run the Calc-X example to see the training loop
3. Adapt the coding_agent example to your repo's test suite
4. Deploy rollouts to Kubernetes for scale

Resources

• Agent Lightning v1.0 Docs
• Technical Report (arXiv)
• Original GitHub Repo
• vLLM Blog: Token ID Return

License

MIT — fork it, ship it, own it.

---

Built by Varritech | christian@varritech.com
Bold ideas wait for no one.