README
Self-Improving Agents
Classical ML optimization ideas, transferred to the agent scaffold around a frozen frontier model.
A living brainstorm. The thesis: you can't train Claude or GPT, but everything around the model is trainable — system prompts, CLAUDE.md, SKILL.md files, tool descriptions, workflows, subagent topology, harness code, sampling params. Treat those as the weights, an eval suite as the loss function, and an LLM as the optimizer, and most of the classical ML toolbox transfers.
The field has names for this now: harness engineering (OpenAI, Feb 2026), loop engineering (Osmani / Steinberger, Jun 2026), and academically Meta-Harness / Self-Harness / Agentic Harness Engineering (Mar–Jun 2026). The best single synthesis is Lilian Weng's Harness Engineering for Self-Improvement (Jul 2026).
Every approach in here is a variation on one meta-loop:
propose an edit → run an eval → keep / discard / archive → repeat
What varies is how you propose (random, reflective, evolutionary, gradient-like), how many candidates you keep (one, a population, a Pareto front, an archive), and how you spend eval budget (minibatches, cascades, pruners, held-out splits).
Status: research notes as of September 2026. Claims link to primary sources where possible. Things that couldn't be verified are flagged inline. Numbers reported by vendors about their own products are self-reported.
The map
| Traditional concept | Agent equivalent | What's actually happening | Flagship project(s) | Build this |
|---|---|---|---|---|
| Hill climbing | Modify skill → eval → keep if better | One candidate file, one metric, git as accept/reject. The agent is the mutation operator. | autoresearch, AutoSkill, SkillOpt | skill-climb: a Claude Code skill that mutates one section of a SKILL.md, reruns 30 tasks × 3, commits or resets |
| Evolutionary algorithms | Population of skill variants; mutate / crossover / select; Pareto front | Genome = text or code. Mutation = LLM rewrite after reading failure traces. Selection = per-instance Pareto (keep anything that wins on any task). | GEPA, EvoSkill, OpenEvolve, ShinkaEvolve, Darwin Gödel Machine | skill-evolve: 8 SKILL.md variants, fitness = (pass rate, tokens, wall time), Pareto front, GEPA's optimize_anything does the loop |
| Bayesian optimization | Search expensive agent configs (model × prompt × temp × context) | Surrogate model (TPE) over a mixed categorical/continuous space; pruners kill bad trials early; multi-objective front. | DSPy MIPROv2, syftr, HARBOR paper | agent-tuner: Optuna over {model tier, thinking budget, which CLAUDE.md sections load, tool-description variant}; objectives = pass rate, $/task, p50 latency |
| Population-based training | Parallel agents; losers copy winners' state + mutate | The copyable "state" is accumulated context (playbooks, skill banks); the "hyperparameters" are instructions and tool configs. Least mature mapping. | ACE, island models in OpenEvolve/ShinkaEvolve, CODESKILL | pbt-skills: 6 parallel sessions on 200 issues; every 25, bottom 2 copy top 2's CLAUDE.md + playbook, then mutate |
| Exploration vs exploitation | Try new approaches vs reuse known-good skills | Two levels: which candidate to mutate next (Pareto sampling, novelty rejection) and which mutator to use (UCB over LLMs). | ShinkaEvolve, AFlow, Meta-Harness | mutator-bandit: UCB over 5 mutation types (tighten, add example, add checklist, remove, reorder); learn which edits move your metric |
| Gradient descent | "Textual gradients": critique flows backward through the pipeline | Build a graph of prompt → LLM → tool → judge; backward() asks an LLM how each node's input should change; optimizer rewrites the variable. | TextGrad, Microsoft Trace, OPRO | textgrad-tools: tool descriptions as tg.Variables, loss = judge comparing tool-call transcript to a rubric |
| Reinforcement learning | Tool calls get rewards from outcomes | Weight-free "verbal RL": rollout → outcome → LLM turns the scalar into a lesson → lesson appended to a skill/memory → next rollout reads it. | EvoSkill, Reflexion, SkillRL, Live-SWE-agent | EvoSkill-lite for AWS: 30 LocalStack scenarios with deterministic checkers; Proposer writes SKILL.md diffs from failures; keep only if held-out pass rate rises |
| Experience replay | Store trajectories, retrieve them later | Three maturity levels: raw transcripts → refined notes (Claude Code auto memory) → abstracted procedures (AWM workflows, ACE playbooks). Retrieval = sampling from the buffer. | Agent Workflow Memory, Claude Code auto memory, Mem0, Letta sleep-time compute | Stop hook writes a 10-line structured session summary to SQLite FTS; UserPromptSubmit hook injects top-3 matches |
| Imitation learning | Turn expert sessions into skills | Expert trajectory → SKILL.md (trigger, steps, scripts, pitfalls). Anthropic's skill-creator is a demonstration-to-skill compiler with an eval loop. | Agent Skills spec, skill-creator, Voyager, Trace2Skill | "Session → skill": subagent reads a successful incident transcript, emits SKILL.md + 3 replay evals, benchmark vs baseline before merge |
| Credit assignment | Which step / tool caused the failure? | Per-step verdicts on a 40-call trajectory before you can write a useful lesson. Judge reads trace, ideally re-executes read-only checks at each step. | Agent-as-a-Judge, AgentPRM, ACE, credit-assignment survey | "Blame" step: judge subagent re-runs describe-* calls at each step's state, emits {step, verdict, lesson}; only cited lessons can edit a skill |
| Curriculum learning | Progressively harder tasks | With frozen weights, the curriculum shapes which experiences get written to memory first. Sample tasks with pass rate in [0.3, 0.7]. | SWE-smith, WebRL, SEAgent, Voyager's auto-curriculum | Terraform/CFN stacks with injected misconfigs at 3 severity tiers; nightly run samples the middle band; promote a tier at 90% |
| Self-play | Breaker creates challenges, Fixer solves them | Proposer emits task + verifier; solver attempts; proposer rewarded for medium difficulty. What accumulates: a task bank and skills distilled from solver failures. | Self-Play SWE-RL, Absolute Zero, Self-Challenging Agents, CoEvoSkills | Two worktrees: Breaker mutates a healthy repo/infra and ships a reproducing test; Fixer gets only the test; accept only if Fixer's 3-try pass rate is 1/3–2/3 |
| Reflection / self-critique | Agent writes its own lessons | Intra-episode (Self-Refine) vs inter-episode (Reflexion, ExpeL, ACE). Only useful when anchored to a test result or diff. | ExpeL, Dynamic Cheatsheet, Self-Refine | Post-session "retro" subagent: every lesson must cite a tool-output line, test name, or diff hunk; writes to .claude/rules/<topic>.md with paths: frontmatter |
| Multi-armed bandits | Pick prompt / model / tool / skill from historical reward | Arms = skill variants or model tiers; context = task features; Thompson sampling; you only observe the reward of the arm you pulled. | LLM Bandit, BaRP, OPTS, Prompt Duel Optimizer | Session-start hook: bucket task (bugfix/feature/refactor/infra), SQLite of Beta(α,β) per (bucket, SKILL.md variant), Thompson-sample, reward = tests green + no revert in 24h |
| Model / tool routing | Dynamic Claude / GPT / Gemini / local router | Classifier before each call (or each turn); newest routers are trajectory-aware: cheap model explores, then escalate. | RouteLLM, Not Diamond Code, SWE-Router, vLLM Semantic Router | "Trajectory router": start subagents on Haiku with maxTurns: 8; SubagentStop hook scores partial transcript; re-spawn on Opus with the summary if low |
| Ensembling | Best-of-N, mixture-of-agents, debate, judge/select/merge | Sample N independent trajectories (seeds, prompts, models, worktrees), aggregate by vote / judge / tournament. Needs decorrelated failures. | Mixture-of-Agents, Self-MoA, Scaling TTC for Agentic Coding, Agent-as-a-Judge | "Best-of-3 PR": 3 worktree subagents (plan-first / TDD / minimal-diff), tests in each, read-only judge does pairwise comparison on diff + test log, opens only the winner |
| Active learning | Only bother the human on high-uncertainty decisions | Estimate uncertainty (committee disagreement, detector agent, information gain) before irreversible steps; ask, proceed, or hand off. Reward = success minus per-question cost. | Ask or Assume, Learning to Ask, IG-reward clarification | "Query-by-committee gate": before terraform apply, 3 cheap subagents predict the intended change; disagreement → hook returns ask with a generated question |
| Hierarchical RL / options | Orchestrator selects specialized skills / subagents | Option = subagent: initiation set ≈ description, internal policy ≈ instructions + tools, termination ≈ maxTurns. You optimize the library, not the policy. | Building Effective Agents, Claude Code subagents, HiAgent, HiSkill | "Option library with termination": strict description, required status: done|blocked|needs_human output, SubagentStop hook logs (option, turns, status); prune/split badly terminating options |
| Test-time search (MCTS, beam) | Tree search over agent actions | Nodes = repo state + partial trajectory; value = tests / PRM / self-eval; UCT select → expand → rollout → backprop. | LATS, SWE-Search, Tree of Thoughts, SWE-TRACE | "Beam-of-plans": 4 read-only plan subagents → value agent scores → execute top 2 in worktrees → re-score on real tests → keep 1 |
Deep dives, with the classical version, the agent version, verified projects, build ideas, and gotchas for each:
- concepts/01-search-and-optimization.md — hill climbing, evolutionary, Bayesian opt, PBT, explore/exploit, textual gradients
- concepts/02-learning-from-experience.md — RL, experience replay, imitation, credit assignment, curriculum, self-play, reflection
- concepts/03-decisions-and-orchestration.md — bandits, routing, ensembling, active learning, hierarchical RL, test-time search
- concepts/04-evals-the-loss-function.md — eval harnesses, trajectory logging, evaluator design, anti-overfitting
- landscape-2026.md — who is doing "agent R&D by agents": frameworks vs papers vs products
- failure-modes.md — reward hacking, eval overfitting, diversity collapse, cost, and what people do about them
- projects/autoskill.md — the flagship build: learn an AWS troubleshooting skill from experience
The progression ladder
Normal agent
→ human writes skills (Agent Skills spec, SKILL.md)
→ agent reflects and updates skills (Claude Code auto memory, Reflexion, ACE)
→ eval-driven hill climbing (autoresearch, AutoSkill, SkillOpt, skill-creator loop)
→ evolutionary skill search (GEPA, EvoSkill, CoEvoSkills)
→ skills learned from trajectories (Trace2Skill, AWM, SkillEvolver)
→ agent architecture / harness evolves (Meta-Harness, AHE, Self-Harness, DGM, SICA)
→ population of agents competes / learns (OpenEvolve, ShinkaEvolve, Hyperagents)
→ autonomous agent R&D (AI Scientist v2, AIDE², AlphaEvolve)
Each rung is roughly one order of magnitude more eval budget and one more failure mode (see failure-modes.md).
Study / build order
- autoresearch — the loop in 3 files. Port it to a SKILL.md instead of
train.py. Readprogram.mdclosely; it's a primitive skill. - skill-creator +
claude plugin eval— the official eval loop for skills. Learn what a grader, an ablation, and a held-out split look like before automating anything. - GEPA — reflective evolutionary search with a Pareto front. Its
gskillpipeline (SWE-smith tasks → evolved SKILL.md → transfer to Claude Code) is the closest existing thing to "AutoSkill". - EvoSkill — same idea but starting from failed trajectories of Claude Code / Codex / OpenHands / Goose.
- Bandit model / tool routing — cheap to build, immediately useful, teaches explore/exploit with real reward noise.
- Trajectory-based skill learning — Trace2Skill, AWM, ACE, SkillEvolver; plus a
Stophook that captures your own sessions. - Multi-agent evolutionary search — OpenEvolve / ShinkaEvolve driving Claude Code or Codex as the mutator; Meta-Harness for evolving the scaffold itself.
The one-sentence versions
- Hill climbing is
git commitif the number went up,git resetif it didn't. - Evolution is hill climbing with a population and a memory of what didn't work.
- Bayesian optimization is evolution that keeps a model of the landscape so it can guess before it runs.
- Bandits are Bayesian optimization for the one decision you make over and over.
- RL without weights is "write down why it failed, read it next time."
- Experience replay is a searchable pile of those notes.
- Imitation is compressing a good session into a checklist.
- Credit assignment is finding which line of the checklist mattered.
- Curriculum is only learning from tasks you fail about half the time.
- Self-play is having one agent write the exam and another take it.
- Ensembling is running it three times and letting a judge pick.
- Active learning is only asking the human when three runs disagree.
- Hierarchical RL is subagents.
- Test-time search is a beam of plans, pruned by the test suite.
How to add to this doc
This is meant to be appended to. When you find a new paper, tool, or blog post:
- Put it under the concept it belongs to in
concepts/, with a date, a link, and one line on what makes it different. - If it's a framework or product rather than a paper, also add it to landscape-2026.md.
- If it reports a failure mode, add it to failure-modes.md.
- Update the map above only if it changes the flagship project for a row.
Related repos: awesome-autoresearch · Awesome-Self-Evolving-Agents · awesome-Self-Improving-Agents · LLM-Agent-Optimization · awesome-ai-model-routing
Contributors
claude
1 commits
Similar repos
README
Self-Improving Agents
Classical ML optimization ideas, transferred to the agent scaffold around a frozen frontier model.
A living brainstorm. The thesis: you can't train Claude or GPT, but everything around the model is trainable — system prompts, CLAUDE.md, SKILL.md files, tool descriptions, workflows, subagent topology, harness code, sampling params. Treat those as the weights, an eval suite as the loss function, and an LLM as the optimizer, and most of the classical ML toolbox transfers.
The field has names for this now: harness engineering (OpenAI, Feb 2026), loop engineering (Osmani / Steinberger, Jun 2026), and academically Meta-Harness / Self-Harness / Agentic Harness Engineering (Mar–Jun 2026). The best single synthesis is Lilian Weng's Harness Engineering for Self-Improvement (Jul 2026).
Every approach in here is a variation on one meta-loop:
propose an edit → run an eval → keep / discard / archive → repeat
What varies is how you propose (random, reflective, evolutionary, gradient-like), how many candidates you keep (one, a population, a Pareto front, an archive), and how you spend eval budget (minibatches, cascades, pruners, held-out splits).
Status: research notes as of September 2026. Claims link to primary sources where possible. Things that couldn't be verified are flagged inline. Numbers reported by vendors about their own products are self-reported.
The map
| Traditional concept | Agent equivalent | What's actually happening | Flagship project(s) | Build this |
|---|---|---|---|---|
| Hill climbing | Modify skill → eval → keep if better | One candidate file, one metric, git as accept/reject. The agent is the mutation operator. | autoresearch, AutoSkill, SkillOpt | skill-climb: a Claude Code skill that mutates one section of a SKILL.md, reruns 30 tasks × 3, commits or resets |
| Evolutionary algorithms | Population of skill variants; mutate / crossover / select; Pareto front | Genome = text or code. Mutation = LLM rewrite after reading failure traces. Selection = per-instance Pareto (keep anything that wins on any task). | GEPA, EvoSkill, OpenEvolve, ShinkaEvolve, Darwin Gödel Machine | skill-evolve: 8 SKILL.md variants, fitness = (pass rate, tokens, wall time), Pareto front, GEPA's optimize_anything does the loop |
| Bayesian optimization | Search expensive agent configs (model × prompt × temp × context) | Surrogate model (TPE) over a mixed categorical/continuous space; pruners kill bad trials early; multi-objective front. | DSPy MIPROv2, syftr, HARBOR paper | agent-tuner: Optuna over {model tier, thinking budget, which CLAUDE.md sections load, tool-description variant}; objectives = pass rate, $/task, p50 latency |
| Population-based training | Parallel agents; losers copy winners' state + mutate | The copyable "state" is accumulated context (playbooks, skill banks); the "hyperparameters" are instructions and tool configs. Least mature mapping. | ACE, island models in OpenEvolve/ShinkaEvolve, CODESKILL | pbt-skills: 6 parallel sessions on 200 issues; every 25, bottom 2 copy top 2's CLAUDE.md + playbook, then mutate |
| Exploration vs exploitation | Try new approaches vs reuse known-good skills | Two levels: which candidate to mutate next (Pareto sampling, novelty rejection) and which mutator to use (UCB over LLMs). | ShinkaEvolve, AFlow, Meta-Harness | mutator-bandit: UCB over 5 mutation types (tighten, add example, add checklist, remove, reorder); learn which edits move your metric |
| Gradient descent | "Textual gradients": critique flows backward through the pipeline | Build a graph of prompt → LLM → tool → judge; backward() asks an LLM how each node's input should change; optimizer rewrites the variable. | TextGrad, Microsoft Trace, OPRO | textgrad-tools: tool descriptions as tg.Variables, loss = judge comparing tool-call transcript to a rubric |
| Reinforcement learning | Tool calls get rewards from outcomes | Weight-free "verbal RL": rollout → outcome → LLM turns the scalar into a lesson → lesson appended to a skill/memory → next rollout reads it. | EvoSkill, Reflexion, SkillRL, Live-SWE-agent | EvoSkill-lite for AWS: 30 LocalStack scenarios with deterministic checkers; Proposer writes SKILL.md diffs from failures; keep only if held-out pass rate rises |
| Experience replay | Store trajectories, retrieve them later | Three maturity levels: raw transcripts → refined notes (Claude Code auto memory) → abstracted procedures (AWM workflows, ACE playbooks). Retrieval = sampling from the buffer. | Agent Workflow Memory, Claude Code auto memory, Mem0, Letta sleep-time compute | Stop hook writes a 10-line structured session summary to SQLite FTS; UserPromptSubmit hook injects top-3 matches |
| Imitation learning | Turn expert sessions into skills | Expert trajectory → SKILL.md (trigger, steps, scripts, pitfalls). Anthropic's skill-creator is a demonstration-to-skill compiler with an eval loop. | Agent Skills spec, skill-creator, Voyager, Trace2Skill | "Session → skill": subagent reads a successful incident transcript, emits SKILL.md + 3 replay evals, benchmark vs baseline before merge |
| Credit assignment | Which step / tool caused the failure? | Per-step verdicts on a 40-call trajectory before you can write a useful lesson. Judge reads trace, ideally re-executes read-only checks at each step. | Agent-as-a-Judge, AgentPRM, ACE, credit-assignment survey | "Blame" step: judge subagent re-runs describe-* calls at each step's state, emits {step, verdict, lesson}; only cited lessons can edit a skill |
| Curriculum learning | Progressively harder tasks | With frozen weights, the curriculum shapes which experiences get written to memory first. Sample tasks with pass rate in [0.3, 0.7]. | SWE-smith, WebRL, SEAgent, Voyager's auto-curriculum | Terraform/CFN stacks with injected misconfigs at 3 severity tiers; nightly run samples the middle band; promote a tier at 90% |
| Self-play | Breaker creates challenges, Fixer solves them | Proposer emits task + verifier; solver attempts; proposer rewarded for medium difficulty. What accumulates: a task bank and skills distilled from solver failures. | Self-Play SWE-RL, Absolute Zero, Self-Challenging Agents, CoEvoSkills | Two worktrees: Breaker mutates a healthy repo/infra and ships a reproducing test; Fixer gets only the test; accept only if Fixer's 3-try pass rate is 1/3–2/3 |
| Reflection / self-critique | Agent writes its own lessons | Intra-episode (Self-Refine) vs inter-episode (Reflexion, ExpeL, ACE). Only useful when anchored to a test result or diff. | ExpeL, Dynamic Cheatsheet, Self-Refine | Post-session "retro" subagent: every lesson must cite a tool-output line, test name, or diff hunk; writes to .claude/rules/<topic>.md with paths: frontmatter |
| Multi-armed bandits | Pick prompt / model / tool / skill from historical reward | Arms = skill variants or model tiers; context = task features; Thompson sampling; you only observe the reward of the arm you pulled. | LLM Bandit, BaRP, OPTS, Prompt Duel Optimizer | Session-start hook: bucket task (bugfix/feature/refactor/infra), SQLite of Beta(α,β) per (bucket, SKILL.md variant), Thompson-sample, reward = tests green + no revert in 24h |
| Model / tool routing | Dynamic Claude / GPT / Gemini / local router | Classifier before each call (or each turn); newest routers are trajectory-aware: cheap model explores, then escalate. | RouteLLM, Not Diamond Code, SWE-Router, vLLM Semantic Router | "Trajectory router": start subagents on Haiku with maxTurns: 8; SubagentStop hook scores partial transcript; re-spawn on Opus with the summary if low |
| Ensembling | Best-of-N, mixture-of-agents, debate, judge/select/merge | Sample N independent trajectories (seeds, prompts, models, worktrees), aggregate by vote / judge / tournament. Needs decorrelated failures. | Mixture-of-Agents, Self-MoA, Scaling TTC for Agentic Coding, Agent-as-a-Judge | "Best-of-3 PR": 3 worktree subagents (plan-first / TDD / minimal-diff), tests in each, read-only judge does pairwise comparison on diff + test log, opens only the winner |
| Active learning | Only bother the human on high-uncertainty decisions | Estimate uncertainty (committee disagreement, detector agent, information gain) before irreversible steps; ask, proceed, or hand off. Reward = success minus per-question cost. | Ask or Assume, Learning to Ask, IG-reward clarification | "Query-by-committee gate": before terraform apply, 3 cheap subagents predict the intended change; disagreement → hook returns ask with a generated question |
| Hierarchical RL / options | Orchestrator selects specialized skills / subagents | Option = subagent: initiation set ≈ description, internal policy ≈ instructions + tools, termination ≈ maxTurns. You optimize the library, not the policy. | Building Effective Agents, Claude Code subagents, HiAgent, HiSkill | "Option library with termination": strict description, required status: done|blocked|needs_human output, SubagentStop hook logs (option, turns, status); prune/split badly terminating options |
| Test-time search (MCTS, beam) | Tree search over agent actions | Nodes = repo state + partial trajectory; value = tests / PRM / self-eval; UCT select → expand → rollout → backprop. | LATS, SWE-Search, Tree of Thoughts, SWE-TRACE | "Beam-of-plans": 4 read-only plan subagents → value agent scores → execute top 2 in worktrees → re-score on real tests → keep 1 |
Deep dives, with the classical version, the agent version, verified projects, build ideas, and gotchas for each:
- concepts/01-search-and-optimization.md — hill climbing, evolutionary, Bayesian opt, PBT, explore/exploit, textual gradients
- concepts/02-learning-from-experience.md — RL, experience replay, imitation, credit assignment, curriculum, self-play, reflection
- concepts/03-decisions-and-orchestration.md — bandits, routing, ensembling, active learning, hierarchical RL, test-time search
- concepts/04-evals-the-loss-function.md — eval harnesses, trajectory logging, evaluator design, anti-overfitting
- landscape-2026.md — who is doing "agent R&D by agents": frameworks vs papers vs products
- failure-modes.md — reward hacking, eval overfitting, diversity collapse, cost, and what people do about them
- projects/autoskill.md — the flagship build: learn an AWS troubleshooting skill from experience
The progression ladder
Normal agent
→ human writes skills (Agent Skills spec, SKILL.md)
→ agent reflects and updates skills (Claude Code auto memory, Reflexion, ACE)
→ eval-driven hill climbing (autoresearch, AutoSkill, SkillOpt, skill-creator loop)
→ evolutionary skill search (GEPA, EvoSkill, CoEvoSkills)
→ skills learned from trajectories (Trace2Skill, AWM, SkillEvolver)
→ agent architecture / harness evolves (Meta-Harness, AHE, Self-Harness, DGM, SICA)
→ population of agents competes / learns (OpenEvolve, ShinkaEvolve, Hyperagents)
→ autonomous agent R&D (AI Scientist v2, AIDE², AlphaEvolve)
Each rung is roughly one order of magnitude more eval budget and one more failure mode (see failure-modes.md).
Study / build order
- autoresearch — the loop in 3 files. Port it to a SKILL.md instead of
train.py. Readprogram.mdclosely; it's a primitive skill. - skill-creator +
claude plugin eval— the official eval loop for skills. Learn what a grader, an ablation, and a held-out split look like before automating anything. - GEPA — reflective evolutionary search with a Pareto front. Its
gskillpipeline (SWE-smith tasks → evolved SKILL.md → transfer to Claude Code) is the closest existing thing to "AutoSkill". - EvoSkill — same idea but starting from failed trajectories of Claude Code / Codex / OpenHands / Goose.
- Bandit model / tool routing — cheap to build, immediately useful, teaches explore/exploit with real reward noise.
- Trajectory-based skill learning — Trace2Skill, AWM, ACE, SkillEvolver; plus a
Stophook that captures your own sessions. - Multi-agent evolutionary search — OpenEvolve / ShinkaEvolve driving Claude Code or Codex as the mutator; Meta-Harness for evolving the scaffold itself.
The one-sentence versions
- Hill climbing is
git commitif the number went up,git resetif it didn't. - Evolution is hill climbing with a population and a memory of what didn't work.
- Bayesian optimization is evolution that keeps a model of the landscape so it can guess before it runs.
- Bandits are Bayesian optimization for the one decision you make over and over.
- RL without weights is "write down why it failed, read it next time."
- Experience replay is a searchable pile of those notes.
- Imitation is compressing a good session into a checklist.
- Credit assignment is finding which line of the checklist mattered.
- Curriculum is only learning from tasks you fail about half the time.
- Self-play is having one agent write the exam and another take it.
- Ensembling is running it three times and letting a judge pick.
- Active learning is only asking the human when three runs disagree.
- Hierarchical RL is subagents.
- Test-time search is a beam of plans, pruned by the test suite.
How to add to this doc
This is meant to be appended to. When you find a new paper, tool, or blog post:
- Put it under the concept it belongs to in
concepts/, with a date, a link, and one line on what makes it different. - If it's a framework or product rather than a paper, also add it to landscape-2026.md.
- If it reports a failure mode, add it to failure-modes.md.
- Update the map above only if it changes the flagship project for a row.
Related repos: awesome-autoresearch · Awesome-Self-Evolving-Agents · awesome-Self-Improving-Agents · LLM-Agent-Optimization · awesome-ai-model-routing
Similar repos
Contributors
claude
1 commits