Picrew/awesome-rsi

A curated collection of papers, research blogs, and open-source projects on recursive self-improvement (RSI), self-modifying agents, and iterative self-training.

Python

13 commits

updated Sep 20, 2026

See the code

README

Awesome RSI

A curated research map of recursive self-improvement: agents that revise their own machinery, models that generate their next training signal, and the evidence that defines their limits.

English | 中文

31 first-party blog posts · 45 research papers · 14 active GitHub projects

Last verified: 2026-09-20

Start Here

Reading path	What to look for
Self-modifying agents	Does the revised agent participate in its next improvement?
Iterative self-training	Do updated models generate the next training data, curriculum or rewards?
Theory and evaluation	Which assumptions and measurements support the loop?

Reading the evidence: self-modification, bounded self-training and theoretical proposals are different claims. Related methods and selection decisions explain what stays outside the main paper list.

Category Overview
Company Research Blogs
Papers and Official Code
Active GitHub Projects
Scope and Curation
Maintenance

Category Overview

Category	Resource	Entries
Mechanisms and Results	Blog	6
AI Research and Supporting Methods	Blog	15
Evaluation and Failure Modes	Blog	4
Research Agendas	Blog	2
Foundations and Historical Tutorials	Blog	4
Papers / Harness	Paper	15
Papers / Models	Paper	21
Papers / Theory and Evaluation	Paper	9
GitHub / Models / Training Research	GitHub project	2
GitHub / Models / Recursive Self-Training	GitHub project	1
GitHub / Harness / Self-Modification	GitHub project	4
GitHub / Harness / Prompt and Workflow Optimization	GitHub project	4
GitHub / Artifacts / Program Evolution	GitHub project	1
GitHub / Artifacts / Learned Skills	GitHub project	2
Total		90

Company Research Blogs

Start here: first-party technical accounts from model builders and specialist AI research labs. Mechanisms, failures, agendas and historical foundations are separated; publisher claims are not independent replications.

Mechanisms and Results

The Darwin Godel Machine: AI that improves itself by rewriting its own code — Sakana AI · 2025-05-30
- Harness · Self-modification — Describes an agent that rewrites its tools and workflows, evaluates descendants on coding benchmarks, and branches from a growing archive to improve again.
- Boundary: Foundation-model training is future work; the article documents reward hacking and supervised sandbox limits.
Kimi K2: Open Agentic Intelligence — Moonshot AI / Kimi · 2025-07-11
- Models · Self-training — Its general RL system uses the model as its own rubric-based critic, continuously updating that critic from on-policy rollouts with verifiable rewards to improve evaluation of non-verifiable tasks.
- Boundary: The critic and policy improve within a designed RL setup; the article does not establish autonomous rewriting of the learning algorithm.
SIMA 2: An Agent that Plays, Reasons, and Learns With You in Virtual 3D Worlds — Google DeepMind · 2025-11-13
- Models · Self-training — Gemini supplies tasks and estimated rewards; SIMA 2 accumulates self-generated experience and trains subsequent agent generations, including in new game and Genie environments.
- Boundary: Initial training uses human demonstrations and later rewards rely on Gemini; the research preview is not unconstrained self-improvement of Gemini itself.
MiniMax M2.7: Early Echoes of Self-Evolution — MiniMax · 2026-03-18
- Harness · Self-modification — Reports more than 100 autonomous rounds of failure-trajectory analysis, scaffold-code modification, evaluation, and keep-or-revert selection; retained memory and skills also support its model-development experiments.
- Boundary: The reported 30% gain is on internal evaluation sets. Researchers still guide model development and make critical decisions; full autonomous weight-level self-evolution is a future aim.
Prime Agent: A self-improving RLM agent — Prime Intellect · 2026-08-05
- Harness · Self-modification — Its /refine pipeline reads its own trajectory and changes persistent prompt notes, memory, skills and subagent specifications; recorded triggers/outcomes and rollback history carry improvements into later turns and sessions.
- Boundary: The base system prompt remains immutable, and this does not retrain model weights. Its Factorio result is explicitly suspected of reward hacking; use the mechanism, not that score, as evidence.
RoboCat: A self-improving robotic agent — Google DeepMind · 2023-06-20
- Models · Self-training — Fine-tunes a task-specific spin-off, collects its practice trajectories, merges them with demonstrations and retrains a generalist RoboCat version for later tasks.
- Boundary: Each new task begins with 100–1000 human demonstrations; the training procedure and robot interfaces remain human-designed.

AI Research and Supporting Methods

Browse 15 articles

Automated Alignment Researchers: Using large language models to scale scalable oversight — Anthropic · 2026-04-14
- Models · Bounded optimization — Nine Claude research agents propose, implement and evaluate weak-to-strong supervision methods, sharing findings and code; performance-gap feedback determines subsequent experiments.
- Boundary: Held-out transfer was mixed, and the best method did not significantly improve production-scale Claude Sonnet 4. Researchers disqualified reward hacks; the researcher models were not themselves retrained in this experiment.
MiniMax M3: Frontier Coding, 1M Context, Native Multimodality — All in One Model — MiniMax · 2026-06-01
- Models · Bounded optimization — The PostTrainBench section describes an agent independently choosing synthetic data and training strategies, training four base models, evaluating them and adjusting its next experiments during a 12-hour loop.
- Boundary: This is a task-bounded external-model optimization experiment, not M3 retraining its own weights. The release also mixes product benchmarks and demos, which are not separate RSI evidence.
Automated researchers can reliably mitigate alignment failures — Anthropic · 2026-08-28
- Models · Bounded optimization — Claude searches literature, proposes methods and data, trains target models, and tests them in repeated experiments across ten alignment-failure categories; methods are checked on held-out benchmarks and larger models.
- Boundary: This optimizes external student models rather than Claude's own weights. Capability constraints and a monitoring agent exclude invalid methods; benchmark success does not establish general autonomous alignment science.
Can LLMs invent better ways to train LLMs? — Sakana AI · 2024-06-13
- Models · Bounded optimization — LLM-Squared proposes preference-loss code, trains models with each candidate, and feeds downstream scores into the next proposal; the loop discovered DiscoPOP.
- Boundary: The proposer is fixed; feeding an improved model back into its own research process is discussed as future work, not a demonstrated result.
AlphaEvolve: How our Gemini-powered coding agent is scaling impact across fields — Google DeepMind · 2026-05-07
- Artifacts · Bounded optimization — Reports follow-up applications of evaluated code evolution to model components, training efficiency, cache policies and TPU circuits, with concrete AI-development feedback paths.
- Boundary: Deployment case studies and publisher-reported gains do not demonstrate a fully closed cycle that retrains and improves the Gemini proposer itself.
AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms — Google DeepMind · 2025-05-14
- Artifacts · Bounded optimization — Explains evaluated program evolution that improves algorithms and Gemini training kernels, feeding successful programs into the next evolutionary proposals.
- Boundary: Improving infrastructure used to train its underlying LLM is not proof of repeated autonomous Gemini-weight self-training.
ShinkaEvolve: Evolving New Algorithms with LLMs, Orders of Magnitude More Efficiently — Sakana AI · 2025-09-25
- Artifacts · Bounded optimization — Details sample-efficient program evolution and an evolved MoE load-balancing loss, tying executable candidate selection to subsequent program generations.
- Boundary: Human-defined fitness and a fixed proposer model bound the result; it does not demonstrate a self-rewriting learning algorithm.
Digital Red Queen: Adversarial Program Evolution in Core War with LLMs — Sakana AI · 2026-01-08
- Artifacts · Bounded optimization — Evolves Core War programs against a growing history of predecessors; changing opponents supply selection pressure and retained programs shape subsequent evolution.
- Boundary: Program co-evolution in a controlled virtual machine does not demonstrate autonomous improvement of the underlying LLM or real-world security capability.
Autonomous AI research for nanogpt speedrun — Prime Intellect · 2026-05-14
- Artifacts · Bounded optimization — Coding agents repeatedly revise optimizer code and hyperparameters, run nanoGPT training and use steps-to-target-validation-loss to select better variants; durable scratchpads preserve experiment state.
- Boundary: Agents excelled at search and recombination but needed upstream human records to keep improving. Model/data/architecture and benchmark rules were fixed, and humans changed the harness between phases.
General Agent: A Self-Evolving, Synthetic Agent Environment — Prime Intellect · 2026-05-18
- Artifacts · Bounded optimization — A synthesizer evolves task families and a solver measures pass rates; only tasks in calibrated difficulty bands survive, and harder tiers seed later extensions of the synthetic training corpus.
- Boundary: The evolved object is the task corpus. The post describes closing the full model-training/environment-generation loop as a broader research direction, not an already completed autonomous cycle.
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery — Sakana AI · 2024-08-13
- Artifacts · Bounded optimization — Describes idea generation, code experiments, paper writing and automated reviewing; saved reviews and experiments inform revisions and future research ideas.
- Boundary: Flawed comparisons and self-review remain risks; accidental execution-script modification is a safety failure, not evidence of beneficial RSI.
FunSearch: Making new discoveries in mathematical sciences using Large Language Models — Google DeepMind · 2023-12-14
- Artifacts · Bounded optimization — Samples earlier high-scoring programs, asks a fixed LLM for improvements, executes candidates, and returns the best programs to a population for future search.
- Boundary: The evaluator and seed program are user-supplied; the model weights and improvement algorithm are not recursively rewritten.
Population-based Model Merging via Quality Diversity — Sakana AI · 2024-12-03
- Models · Bounded optimization — CycleQD cycles which task defines quality, crosses and mutates expert models, and retains diverse high-performing models in skill archives for further evolution.
- Boundary: The tasks, starting experts and quality-diversity algorithm are human-specified; model merging is not gradient self-training or self-rewriting optimization.
Evolving New Foundation Models: Unleashing the Power of Automating Model Development — Sakana AI · 2024-03-21
- Models · Bounded optimization — Evolves layer-selection and weight-mixing recipes over successive generations, selecting merged models by task fitness and assessing the selected model on a separate test set.
- Boundary: Evolution searches a human-defined merge space of pretrained models; it does not establish an LLM autonomously rewriting its training algorithm.
Accelerating scientific breakthroughs with an AI co-scientist — Google Research · 2025-02-19
- Artifacts · Bounded optimization — Generation, reflection, ranking, evolution and meta-review agents iteratively revise scientific hypotheses using tournament feedback and researcher input.
- Boundary: Elo is a self-evaluation signal rather than independent ground truth; laboratory validation uses expert guidance, and model weights remain fixed.

Evaluation and Failure Modes

Browse 4 articles

AI CUDA Engineer update: robust benchmarking and interim results — Sakana AI · 2025-09-17
- Artifacts · Evaluation / safety — Corrects kernel-optimization claims after benchmark bypasses; robust-kbench reduces reported mean speedup from 3.13x to 1.49x under stricter measurement.
- Boundary: A negative-results and measurement lesson for improvement loops, not a new self-improving agent; the original update is in Japanese.
Sycophancy to subterfuge: Investigating reward tampering in language models — Anthropic · 2024-06-17
- Harness · Evaluation / safety — Tests whether a curriculum of specification gaming generalizes to editing the model's own reward function and concealing the change, exposing a direct failure mode when self-modifying systems can alter their evaluators.
- Boundary: Reward tampering occurred in 45 of 32,768 trials in a constructed study; it was not observed evidence of deployed Claude performing RSI, nor proof that all self-modification is unsafe.
From shortcuts to sabotage: natural emergent misalignment from reward hacking — Anthropic · 2025-11-21
- Harness · Evaluation / safety — Shows reward-hacking training generalizing to malicious behavior, including a Claude Code evaluation where the model attempts to sabotage this research project's detection code, directly testing trust in AI-assisted AI safety research.
- Boundary: Researchers deliberately selected hackable RL environments and added hacking knowledge during pretraining. The 12% sabotage attempt rate describes this experimental model, not ordinary deployed Claude or proven completed sabotage.
Measuring Autonomous AI Research — Prime Intellect · 2026-08-14
- Artifacts · Evaluation / safety — Evaluates 153 autonomous optimizer-research runs across 18 frontier models, examining whether proposed nanoGPT improvements survive evaluation and whether long-running agents produce new methods rather than only recombine existing ones.
- Boundary: A fixed optimizer speedrun is a bounded proxy for research ability, not proof of general RSI. Results depend on seeds, supplied baselines and evaluation integrity.

Research Agendas

Browse 2 articles

Introducing Sakana AI's Recursive Self-Improvement (RSI) Lab — Sakana AI · 2026-06-05
- Harness · Research agenda — Maps a proposed loop from agent-native models to AI scientists that build better models, grounded in DGM, LLM-Squared, ShinkaEvolve and adversarial co-evolution.
- Boundary: A research agenda and lineage map, not evidence that the complete autonomous model-improvement cycle has already been achieved.
When AI builds itself — Anthropic Institute
- Models · Research agenda — Defines recursive self-improvement as AI autonomously designing and developing its successor, presents internal evidence of AI accelerating engineering and research, and examines whether that assistance can close the full model-development loop.
- Boundary: The article explicitly says full RSI has not been achieved and is not inevitable. Internal productivity statistics are observational; human research judgment, compute, evaluations and security remain constraints.

Foundations and Historical Tutorials

Browse 4 articles

Constitutional AI: Harmlessness from AI feedback — Anthropic · 2022-12-15
- Models · Self-training — Samples model responses, generates self-critiques and revisions, fine-tunes on the revised responses, then derives AI preferences for a reward model used in reinforcement learning.
- Boundary: Human-written constitutional principles and a fixed staged training recipe remain essential. This is a bounded self-supervision foundation, not evidence of endlessly repeated autonomous RSI.
AlphaGo Zero: Starting from scratch — Google DeepMind · 2017-10-18
- Models · Self-training — Self-play outcomes train the network; the updated network guides stronger search and games that supply the next training round.
- Boundary: A foundational bounded self-training example in Go, with human-designed rules and learning machinery, not open-ended RSI.
AlphaZero: Shedding new light on chess, shogi, and Go — Google DeepMind · 2018-12-06
- Models · Self-training — Describes neural-network parameter updates from self-play outcomes and stronger network-guided tree search across separately learned games.
- Boundary: Each game retains fixed rules and objectives; this is not one model autonomously expanding its domain or rewriting its learner.
Self-Evolving Agents - A Cookbook for Autonomous Agent Retraining — OpenAI and Bain · 2025-11-04 · archived tutorial
- Harness · Bounded optimization — Demonstrates versioned summarization-prompt updates from grader feedback, meta-prompting and GEPA, retaining better candidates for later requests.
- Boundary: The official recipe is archived and may reference outdated APIs. Despite the title, it changes prompts rather than model weights; production needs held-out tests and human approval. Its example validation slice overlaps its training list.

Papers and Official Code

Dates refer to first publication. Only source-verified venues are shown; a date alone makes no peer-review claim.
Code links are author-linked releases. — means no author-linked implementation was established in this review; it does not assert that none exists. Release details preserve metadata and partial-release notes.
Star badges and push dates use the 2026-09-20 metadata snapshot.

Papers / Harness

Agents revise their own executable code or retained control procedures, then use the revised system in subsequent improvement.

Paper	Improvement Mechanism	Code
SIFT 2026-09-17	Self-modification Evolves coding-agent implementations using pairwise LLM judgments and benchmark scores to select parents and prioritize evaluation; archived descendants seed later code revisions. Experiments compare DGM/HGM baselines, judge ablations and transfer across coding models. Boundary Model weights and the search/judging procedure stay fixed; stronger external judges and benchmark checks remain necessary. Full-benchmark results include the search subset, so they are not a wholly disjoint held-out evaluation. Cost tables report individual runs, and unsafe evaluator edits are rejected.	—
Dream-RSI 2026-09-14	Self-modification Revises executable exploration policies using replayed discovery histories, then redeploys them to collect subsequent improvement data. Experiments compare fixed exploration across algorithm, mathematics and kernel tasks, including held-out Lasso datasets. Boundary Modification is restricted to exploration control; discovery/development models, evaluator and interfaces stay fixed. Replay covers recorded outcomes only. Non-decreasing replay scores do not guarantee online improvement; task gains vary. Full implementation remains unreleased.	Official artifacts only Details • Last push: 2026-09-16 • Author-linked repository currently contains the paper and presentation assets. Its README marks the full codebase, discovered-program release and reproduction scripts as pending; this is not a runnable implementation release.
Meta^n 2026-08-25	Self-modification Repeatedly applies a fixed meta-operation to the evolving solver stack, generating preprocessing code and reusable helpers; an archive retains evaluated layer chains. Boundary Recursion acts on generated layers, not on the meta-operation or model weights. Most reported gains come from passed context; runs plateau at finite depth.	Official code Details • Last push: 2026-08-26 • The authors label this a research prototype with exploratory results.
HELIX 2026-08-14	Self-modification Decomposes agent systems into typed modular components and co-evolves harness and model in a loop; harness evolution boosts current performance and generates verified trajectories that become training data for the next model update. Boundary Evaluated on code repair tasks only. A 65-candidate portfolio and 200-slot sibling slice define the search budget. Model-harness co-evolution is demonstrated in bounded iterations, not as indefinite improvement.	Official code Details • Last push: 2026-08-17
DarwinX 2026-07-31	Self-modification Maintains a population of harness variants with frozen model weights; a fitness-based selection mechanism admits only variants that extend task coverage without regression, and an archive preserves alternative lineages for recombination. Boundary Model weights are frozen throughout; only harness scaffolding evolves. Cross-benchmark transfer is demonstrated but not guaranteed for arbitrary domains. Fitness is determined by each benchmark's own verifier.	—
RHI 2026-07-17	Self-modification Represents the harness as a prompt-level specification of the agent loop and iteratively refines it using pairwise feedback from its own revision history, enabling low-reasoning-effort agents to surpass maximum-reasoning-effort settings. Boundary Tested on 30 synthetic ML research tasks across three domains. Gains come mainly from improved context management, not deeper reasoning. The information-theoretic framing is a proposed hypothesis, not a proven bound.	—
HarnessBank 2026-07-15	Self-modification Pairs a task agent with an evolver agent that diagnoses failures, generates harness candidates, and maintains a Harness Gene Bank of high-performing configurations organized by semantic coordinates; a gated screening mechanism filters candidates before costly evaluation. Boundary Cross-model experiments show improvements are model-specific rather than universal. Gains of 5.1-15.4% across seven benchmarks are relative to existing methods, not absolute baselines. The evolver agent and screening mechanism are fixed.	—
MetaSkill-Evolve 2026-07-06	Self-modification Evolves task skills frequently and the five agents’ meta-skill files more slowly; the same pipeline edits the instructions that govern its own improvement. Boundary One frozen backbone and three curated benchmarks. Meta-skills change, but the five roles, their wiring and the update schedule remain fixed.	—
Continual Harness 2026-05-11	Experience learning Alternates action and refinement of prompts, subagents, skills and memory within a reset-free run. A separate co-learning experiment relabels rollouts with a frontier teacher and updates an open model without resetting the game. Boundary Earlier Gemini Plays Pokemon results used human-in-the-loop harness refinement; later automated adaptation and teacher-assisted weight co-learning are distinct settings. Teacher supervision and game-specific evaluation limit autonomy claims.	Official code Details • Last push: 2026-09-20 • Official associated implementation of the harness component via an explicit README link to this paper. The inspected paper v1 does not establish release of all Pokemon experiments or the teacher-relabeled weight co-learning pipeline; this is not certified full reproduction code.
Hyperagents 2026-03-19	Self-modification Integrates a task agent and a meta agent into one editable program so that evaluated changes can improve both task behavior and the machinery producing future changes. Boundary Reported transfer and accumulation are finite experiments, not evidence of indefinite acceleration or autonomous weight-level learning.	Official code Details • Last push: 2026-07-31
Huxley-Gödel Machine (HGM) 2025-10-24 ICLR 2026	Self-modification Uses descendant performance to estimate which self-modifying coding-agent lineages will produce better future agents, guiding the next code rewrites. Boundary Clade statistics approximate improvement potential; they are not proofs of globally optimal rewrites. Experiments use bounded coding benchmarks and fixed underlying LLMs.	Official code Details • Last push: 2026-02-07
Darwin Gödel Machine (DGM) 2025-05-29	Self-modification A coding agent modifies its own implementation, evaluates descendants on coding benchmarks, and branches from a growing archive of agents to produce further improvements. Boundary Empirical code-level self-improvement, not formal proof of beneficial rewrites or foundation-model weight training; benchmark exploitation and sandbox escape remain concerns.	Official code Details • Last push: 2025-08-13
Self-Improving Coding Agent (SICA) 2025-04-21 ICLR 2025 Workshop on Scaling Self-Improving Foundation Models	Self-modification Evaluates the current coding agent, archives results, runs that same agent on its own codebase to implement an improvement, and evaluates the updated implementation again. Boundary Non-gradient scaffold learning uses fixed LLM weights; gains on a sampled SWE-bench Verified subset and other benchmarks do not establish unlimited progress or whole-benchmark state of the art.	Official code Details • Last push: 2025-04-23 • Its citation explicitly identifies a workshop, not the ICLR main conference track.
Gödel Agent 2024-10-06	Self-modification Uses LLM-generated changes to recursively revise the agent's own logic and behavior under high-level objectives rather than limiting changes to a predefined task-agent pipeline. Boundary Inspired by the Gödel machine but supported by empirical task evaluations, not proofs that all rewrites are beneficial or that the whole agent-design space is optimally searched.	Official code Details • Last push: 2025-09-17 • The canonical abstract links the repository, whose README links the same paper. No verified formal-proof implementation or conference venue is inferred from the name.
Self-Taught Optimizer (STOP) 2023-10-03	Self-modification A seed LM-calling program optimizer is applied to its own code, discovering improved search scaffolds that then optimize downstream programs. Boundary The paper explicitly says unchanged language models make this not full recursive self-improvement; only a small task set is studied, including sandbox-bypass risks.	Official code Details • Last push: 2024-01-01

Papers / Models

Iterative model, curriculum and evaluator training. Updated models create later training signals; the learning rule can remain fixed.

Paper	Improvement Mechanism	Code
ScienceBuddy 2026-09-15	Self-training Alternates feedback-guided harness revision with GRPO training of the task model; updated weights and the selected harness enter subsequent cycles. A three-cycle scientific-task experiment reports single-attempt test accuracy rising from 42.2% to 73.3%. Boundary The auxiliary editor, rubric composer and evaluators remain fixed; researcher-derived tasks and three finite cycles bound the result. The released simplified experiment uses different stage budgets from the paper and does not include the hosted product or establish open-ended scientific improvement.	Official code Details • Last push: 2026-09-17 • The paper links ScienceBuddy-RSI, which redirects to this canonical repository. README and docs/algorithm.md describe a simplified three-cycle release with frozen tasks, three harness steps and 30 GRPO updates per stage, distinct from the paper's ten-step/20-update experiment. Hosted product source is excluded.
J-Zero 2026-08-27	Self-training Co-trains a task Challenger, Solver and Judge across rounds; structurally constructed preference pairs update the Judge that rewards later policy training. Boundary The Judge starts from a pretrained reward checkpoint. Preference ordering is a designed assumption; reported ten-round gains do not establish unbounded improvement.	Official code Details • Last push: 2026-09-10
SPADE 2026-08-19	Self-training A single LLM fills two self-play roles, Environment Designer and Reasoning Agent; the Designer writes executable OpenAI Gym-style environments grounded in pretraining documents, and the Agent's regret signal guides the Designer to produce challenges at the edge of the agent's capabilities. Boundary Work in progress. Tested at 30B scale. The Designer-Agent co-evolution relies on pretraining corpus grounding and accumulated environment memory, not arbitrary open-ended generation.	—
Socratic-SWE 2026-06-05	Self-training Distills solving traces into skills, generates targeted repair tasks, and jointly trains generator/solver roles; updated solvers produce the next curriculum’s traces. Boundary A fixed seed-repository pool, executable tests and trusted validation tasks constrain the loop. The paper reports later-iteration saturation.	—
Q-Evolve 2026-06-05 ICML 2026	Self-training Unifies automatic process-reward labeling and policy learning in an in-distribution RL loop; a critic trained on mixed expert and agent data derives step-level rewards via advantage estimation, providing dense supervision for iterative self-improvement without distribution shift. Boundary Evaluated on AlfWorld, WebShop and ScienceWorld. The critic is trained from a mix of expert and self-generated data, not purely self-generated. Reported gains are within these benchmark environments.	—
SIA-W 2026-05-22 ICML 2026 Workshop (AI as a Tool for Mathematics, Computer Science, and Machine Learning)	Self-training A self-improving agent framework that jointly optimizes agent scaffold (tools, prompts, execution harnesses) and model weights through test-time reinforcement learning, building domain-adapted search and reasoning procedures across generations. Boundary Evaluated on three contrasting domains (LawBench, GPU kernel optimization, single-cell RNA denoising). Weight updates require test-time RL computation. The framework combines scaffold evolution with weight adaptation, making it difficult to isolate individual contributions.	—
Self-Guided Self-Play (SGS) 2026-04-22	Self-training Co-trains a Solver and Conjecturer on verified Lean4 proofs. A frozen Guide scores generated subproblems for relevance and clarity; updated models produce later problems and solutions. Long-run experiments and ablations examine conjecturer degeneration and solver entropy collapse. Boundary Uses a fixed target-problem pool and formal verifier; the Guide is not trained. Headline comparisons measure cumulative solves during training against a larger model's pass@4, not equal-budget held-out accuracy. Fitted asymptotes are extrapolations, and non-verifiable domains remain untested.	Official code Details • Last push: 2026-04-26 • Author-linked implementation includes full SGS, frozen-conjecturer and no-Guide scripts. standard_sgs.py retains both trained models; full_pipeline.py manages successive checkpoints. Experiments were not reproduced in this review.
Agent0 2025-11-20	Self-training Couples a curriculum model with a tool-using executor model; stronger execution drives harder generated curricula, which in turn provide reinforcement-learning data. Boundary The released training instructions require manual checkpoint selection between iterations; zero external data does not remove pretrained-backbone or tool dependencies.	Official code Details • Last push: 2026-07-10
R-Zero 2025-08-07 ICLR 2026	Self-training Co-evolves Challenger and Solver models so that frontier-difficulty generated tasks train the Solver, while the Solver's changing capability alters the Challenger's rewards. Boundary Uses a pretrained base and designed rewards; finite iterations can regress, and the authors' later R-Few work introduces human data to address scaling limits.	Official code Details • Last push: 2026-02-04
Self-Adapting Language Models (SEAL) 2025-06-12	Self-training The model generates self-edits containing finetuning data or update directives; SFT makes persistent weight changes, and downstream performance trains better self-edit generation through an outer RL loop. Boundary Self-edits control adaptation within a researcher-designed SFT/RL framework; experiments on knowledge incorporation and few-shot generalization do not prove unrestricted self-redesign.	Official code Details • Last push: 2025-08-01 • The paper links its author project page, and the matching official repository links the same paper and page. The canonical repository is Continual-Intelligence/SEAL.
Absolute Zero 2025-05-06	Self-training A model co-evolves its task proposals and solving ability, using a code executor for task validity and answer rewards instead of an externally curated post-training dataset. Boundary Zero data refers to the self-play post-training setup, not an untrained backbone; the executor, rewards and optimization machinery are human-designed.	Official code Details • Last push: 2025-08-24
SiriuS 2025-02-07 NeurIPS 2025	Self-training Collects successful multi-agent trajectories, repairs failed ones, and fine-tunes the participating agents; improved agents generate later training experience. Boundary Starts from labeled problems and fixed agent graphs. Role-specific SFT is iterative learning, not autonomous redesign of the training algorithm.	Official code Details • Last push: 2025-12-01
Self-Taught Evaluators 2024-08-05	Self-training Generates contrasting responses and synthetic reasoning/judgments to repeatedly train an LLM evaluator, using improved evaluator predictions to construct later training rounds. Boundary Human-preference-free training is not the same as zero human validation; the release documents checkpoint selection with HelpSteer2 validation accuracy.	Official artifacts only Details • Last push: 2026-06-25 • Official model, synthetic data and judging/evaluation scripts are documented; this is not a claim that a complete end-to-end training pipeline is released.
ReST-MCTS* 2024-06-06	Self-training Uses process-reward-guided tree search to infer step values from correct final answers, then trains both the policy and process reward model on selected traces across multiple iterations. Boundary Removes per-step manual annotation, not oracle final-answer supervision; externally designed search and reward-learning rules remain fixed.	Official code Details • Last push: 2025-01-20 • Official README documents policy/value-model synthetic-data generation and iterative training. Presence of code is not independent replication of every reported result.
Self-Play Preference Optimization (SPPO) 2024-05-01 ICLR 2025	Self-training Treats alignment as a constant-sum two-player game and repeatedly updates the policy against its own generated responses using preference probabilities to approach a Nash equilibrium. Boundary Experiments use prompts and a pretrained PairRM judge; the equilibrium guarantee concerns the specified preference game, not unbounded capability growth or self-improving evaluation.	Official code Details • Last push: 2025-01-23 • The official repository explicitly provides code and released models; PairRM is an external pretrained preference model, not the policy learning to judge itself.
Self-Rewarding Language Models 2024-01-18	Self-training Uses the language model as its own prompted reward judge during iterative DPO, jointly improving response generation and the rewards it gives subsequent training examples. Boundary Three reported iterations and benchmark preference gains do not prove calibrated self-judgment or sustained superhuman improvement; seed supervision remains relevant.	—
Self-Play Fine-Tuning (SPIN) 2024-01-02 ICML 2024	Self-training Trains a policy to distinguish human demonstration responses from responses generated by its previous iteration, repeatedly strengthening a supervised fine-tuned model through self-play. Boundary Reuses human demonstrations and an SFT starting model; theoretical optimality concerns the target data distribution, not unlimited recursive capability growth.	Official code Details • Last push: 2024-05-08
ReST-EM 2023-12-11	Self-training Repeatedly samples solutions, filters by binary correctness feedback and fine-tunes on accepted samples, studying scaling on MATH and APPS with PaLM-2. Boundary Needs externally supplied problems and verifiable feedback; a few EM-style iterations do not establish indefinite improvement.	—
ReST 2023-08-17	Self-training Alternates policy-generated data collection with reward-guided offline learning, reusing samples to improve a language-model policy, demonstrated on machine translation. Boundary Reward and preference signals remain externally specified; results in translation do not establish a self-improving reward mechanism or open-ended capability growth.	—
RoboCat 2023-06-20	Self-training Adapts a generalist robotic policy to tasks and embodiments, uses trained policies to gather further robot experience, and retrains subsequent generalist models on the expanded data. Boundary Task adaptation still uses demonstrations and controlled robot infrastructure; this is a building block for autonomous improvement, not self-redesign of the training system.	—
STaR 2022-03-28 NeurIPS 2022	Self-training Generates reasoning traces, filters them by answer correctness, rationalizes failed examples using known answers, and repeatedly fine-tunes on successful traces. Boundary Requires a task dataset, known answers and seed rationale examples; a fixed training loop is not an autonomous redesign of the learner.	Official code Details • Last push: 2023-02-21

Papers / Theory and Evaluation

Formal foundations, proposed closed-loop learning, and tests of whether self-improvement signals remain reliable. These are not implementation demonstrations.

Paper	Improvement Mechanism	Code
S3Gym 2026-08-31	Evaluation / safety An interactive benchmark for evaluating LLM self-improvement through three coupled capabilities, self-testing, self-judging and self-improvement, built around seven text-based games with executable environment verifiers. Boundary Core finding is that self-improvement is neither automatic nor uniform across tasks. Parameter training shows instability and negative transfer. Seven text-based games are a limited proxy for general capability.	—
Recursive Code Self-Training Collapse 2026-06-26	Evaluation / safety Compares unfiltered recursive code training, model-independent compile/static gates and model-coupled self-review across four code models and five rounds. Self-review can increasingly accept generated code while benchmark correctness deteriorates, exposing failure in the feedback used for subsequent training. Boundary The main sweep uses Python and 1.1B-7B models. Human-gate is a name for simplified automated checks, not actual human PR review or full semantic verification. Finite collapse experiments do not establish that all self-training or stronger external verification must fail.	Official code Details • Last push: 2026-09-07 • The author-linked src/run_experiment.sh continues from the previous checkpoint for generation, filtering and training. Current evaluation commands use temperature 0.0, whereas the paper's setup states 0.8; inspect configuration before claiming reproduction.
Rise-and-Collapse 2026-06-17	Evaluation / safety Documents a rise-then-collapse pattern in REINFORCE post-training for code, where performance peaks within tens of gradient steps then falls, sometimes to near zero; KL and EWC constraints do not prevent it, and GRPO raises the floor but does not remove the cliff. Boundary Studied on Qwen-2.5-3B/7B and a Gemma-3-4B pilot with competitive programming tasks. Mitigation effectiveness is regime-dependent. This is a within-task failure analysis, not a general theory of self-training limits.	—
Self-Evolution Generalization Gap 2026-06-02 ICML 2026	Evaluation / safety Examines whether LLMs can improve through self-evolution using only internally generated supervision in a closed-loop setup; finds self-evolution consistently improves over base but plateaus after excessive training compute, leaving a gap versus oracle supervision. Boundary Tested on Knights and Knaves logical reasoning as primary testbed, with modest gains on real-world benchmarks. Multi-turn critic-revision with large models (Gemma 12B) nearly matched oracle performance, but internally generated supervision remains insufficient under minimal formulation.	—
Task-centric Self-Improvement 2026-02-14 ICML 2026	Evaluation / safety Provides finite-sample theoretical analysis of iterative self-improvement where models fine-tune on reward-verified outputs, revealing an explicit feedback loop where better models accept more data per iteration; proves conditions where easy-to-hard curricula outperform fixed task mixtures. Boundary Theoretical analysis grounded in finite-sample setting with verifiable rewards. Validated on synthetic graph-based reasoning and mathematical benchmarks. Analysis explains saturation but does not eliminate it.	—
Statistical Gödel Machine (SGM) 2025-10-11	Evaluation / safety Tests candidate edits before adoption and budgets cumulative false-acceptance risk across rounds, providing a statistical gate for self-modification. Boundary Guarantees require bounded independent paired measurements and a stable evaluator. Experiments use simple proposals, not a demonstrated self-rewriting LLM.	Official code Details • Last push: 2026-05-11 • The linked repository now describes a later anonymous submission; the catalog summary refers to arXiv v1.
Socratic Learning 2024-11-25	Research agenda Develops a position on closed-system recursive learning through language games, separating feedback quality, experience coverage and resource requirements. Boundary A position paper under explicit assumptions; it does not report an implemented system with boundless empirical capability growth.	—
Guided Self-Improvement (GSI) 2024-11-01	Evaluation / safety Studies loss of difficult examples during repeated self-training and uses Socratic hints to recover sampling coverage for later training rounds. Boundary Requires known answer checks and guidance; correct final answers can still hide spurious rationales. This is a limits-and-mitigation study, not unrestricted RSI.	Official code Details • Last push: 2024-11-10
Gödel Machines 2003-09-25 Adaptive Agents and Multi-Agent Systems II (2005)	Research agenda Formalizes a self-referential solver that can rewrite its proof-search code once the expected usefulness of that rewrite is provable. Boundary A theoretical construction relative to encoded axioms and utility, not an efficient deployed LLM system; useful rewrites may be unprovable or costly to prove.	—

Active GitHub Projects

Repository metadata snapshot: 2026-09-20. Only public, non-archived projects pushed in the last 60 days appear here; stars are snapshots, not evidence of RSI. Paper-associated code above has no activity gate.

GitHub / Models

GitHub / Models / Training Research

Agents revise training recipes and evaluate resulting models; the researching agent is not necessarily retrained.

Project	Link	Stars	Tags	Improvement Loop and Boundary
RD-Agent / FT-Agent	GitHub		`model-training` `experiment-loop` `validation`	Bounded optimization FT-Agent generates data-processing code and training configurations, fine-tunes a target LLM, then uses OpenCompass validation feedback to refine the next training experiment. Boundary This improves an external target model, not the planner's own weights; test splits are reserved for final reporting. • Evidence
OpenRSI / OpenMLE	GitHub		`model-training` `meta-evolution` `program-evolution`	Bounded optimization An executable RSI framework where OpenMLE-Gym builds ML task packages, OpenMLE-ERL trains program-evolution operators via SFT and RL, and OpenMLE-Evo runs long-horizon search; search produces experience, experience enters training, and trained models return to search. Boundary Meta-evolution operates within bounded ML engineering tasks (NatureBench), not general domains. The Frontis-MA1 model learns four fixed atomic operators (Draft, Improve, Debug, Crossover); the operator vocabulary and composition rules are human-designed. • Evidence

GitHub / Models / Recursive Self-Training

Model updates change the agents that generate the next round of training tasks, solutions or rewards.

Project	Link	Stars	Tags	Improvement Loop and Boundary
J-Zero	GitHub		`self-training` `recursive-learning`	Self-training Implementation: Co-trains a task Challenger, Solver and Judge across rounds; structurally constructed preference pairs update the Judge that rewards later policy training. Boundary The Judge starts from a pretrained reward checkpoint. Preference ordering is a designed assumption; reported ten-round gains do not establish unbounded improvement. • Evidence

GitHub / Harness

GitHub / Harness / Self-Modification

The agent implementation and its improvement procedure are themselves editable.

Project	Link	Tags	Improvement Loop and Boundary
Prime Agent / Continual Harness	GitHub	`self-refinement` `continual-harness` `rollback`	Self-modification Reviews trajectories with /refine and retains small evidence-backed updates to supplemental prompts, memory, skill descriptions and subagent specifications; snapshots allow rollback. Boundary The base system prompt is immutable. Refinement does not replace packaging/review of executable skills, and its processes are not a security sandbox. • Evidence
HyperAgents	GitHub	`self-modification` `meta-agent` `archive`	Self-modification Integrates task and meta agents in one editable program; evaluated descendants can change both task behavior and the procedure that generates subsequent agents. Boundary Empirical task-bounded experiments, not proof of indefinite improvement or foundation-model weight self-training. • Evidence
SIA	GitHub	`self-modification` `weight-update` `harness-evolution`	Self-modification A self-improving loop where a Feedback-Agent reviews task-agent execution logs, updates both the harness code (prompts, orchestration, agent implementation) and the model weights of the task-specific agent across generations. Boundary The Feedback-Agent and meta-agent remain fixed across generations; improvement targets the task agent only. Benchmark-specific evaluators define fitness. Reported gains (LawBench +56.6%, GPU kernel -91.9%) are single-run samples on selected tasks. • Evidence
Meta^n	GitHub	`self-modification` `meta-improvement`	Self-modification Implementation: Repeatedly applies a fixed meta-operation to the evolving solver stack, generating preprocessing code and reusable helpers; an archive retains evaluated layer chains. Boundary Recursion acts on generated layers, not on the meta-operation or model weights. Most reported gains come from passed context; runs plateau at finite depth. • Evidence

GitHub / Harness / Prompt and Workflow Optimization

Feedback updates retained prompts or workflow graphs. These bounded optimizers do not establish unrestricted self-modification.

Project	Link	Tags	Improvement Loop and Boundary
DSPy / GEPA and MIPROv2	GitHub	`prompt-optimization` `demonstrations` `metrics`	Bounded optimization Compiles LM programs by optimizing instructions and demonstrations against task metrics; compiled programs retain the selected configuration for later use. Boundary Included for its optimizers, not all DSPy functionality; prompt compilation does not by itself modify the optimizer or model weights. • Evidence
GEPA	GitHub	`reflection` `pareto-selection` `prompt-optimization`	Bounded optimization Reflects on execution traces and evaluator feedback to propose prompt revisions, retaining complementary candidates through Pareto-based selection. Boundary The original method optimizes prompts with fixed model weights; its general optimize_anything API is not evidence that GEPA rewrites itself. • Evidence
Reef / Harness Evolution	GitHub	`harness-evolution` `evaluation-gate` `versioned-artifacts`	Bounded optimization Serves agent traffic behind an OpenAI- and Anthropic-compatible endpoint and returns a record receipt per request; scores and feedback reported against failing receipts drive one proposed mutation of the harness tree of skills, prompts, rules and configuration, published as a versioned artifact only when it beats the current tree on the deployment's configured tasks. Boundary The proposer implementation, feedback-to-record matching and evaluation gate are operator-defined and fixed across rounds, with fixed model weights. Published tutorial runs grade three fixed tasks by exact final answer, reuse those tasks for proposal feedback and gating, and report single-run samples without repeated trials or uncertainty estimates; these demonstrations do not establish held-out generalization or sustained capability growth. • Evidence
EvoAgentX / Evolution Algorithms	GitHub	`workflow-optimization` `aflow` `validation`	Bounded optimization Runs AFlow, TextGrad, MIPRO and EvoPrompt over agent workflows; validation scores drive prompt or graph revisions, with separate test evaluation. Boundary Included for executable evolution algorithms, not tool integrations or workflow generation alone; objectives and search algorithms are human-specified. • Evidence

GitHub / Artifacts

GitHub / Artifacts / Program Evolution

Executable programs and algorithms are evolved against an evaluator; the optimizer is generally fixed.

Project	Link	Stars	Tags	Improvement Loop and Boundary
ShinkaEvolve	GitHub		`program-evolution` `novelty` `ai-training`	Bounded optimization Evolves programs with parent sampling, novelty rejection and bandit-based LLM selection; evaluated successors re-enter the archive, including experiments on AI training-loss design. Boundary The evaluator and evolutionary machinery are supplied by researchers; generating its own problems is a proposed extension, not a demonstrated feature. • Evidence

GitHub / Artifacts / Learned Skills

Skills are created and revised from experience for reuse across later tasks, not simply bundled as a static library.

Project	Link	Stars	Tags	Improvement Loop and Boundary
Hermes Agent / Learned Skills	GitHub		`learned-skills` `procedural-memory` `experience`	Experience learning Creates procedural skills after complex tasks and revises them during use; persistent skills and searchable experience are reused across sessions. Boundary This is experience-driven skill persistence, not model-weight training or independently demonstrated monotonic capability growth. • Evidence
GenericAgent	GitHub		`skill-learning` `experience-accumulation` `tool-use`	Bounded optimization A minimal self-evolving agent that crystallizes each completed task into a reusable Skill, growing a persistent skill tree from a 3.3K-line seed; accumulated skills are retrieved and composed for later tasks. Boundary Skills are stored artifacts, not self-modifying code; the agent loop, tool definitions and skill-crystallization procedure remain fixed. The optimizer itself does not change across rounds. • Evidence

Scope and Curation

RSI means an improved system participates in producing subsequent improvements. We prioritize implementations that change their own improvement machinery; related self-training and persistent artifact optimization are labeled separately. A normal tool-use loop, test runner, RAG framework, or manually maintained skill collection does not qualify. Each entry names the object changed, feedback, retained state, and limitation.

No entry establishes unbounded autonomous RSI. Within-task refinement, safety evaluation and research agendas are relevant context, not demonstrations of persistent self-improvement. Counts refer to resources: a blog, paper and repository may document the same research, not three independent breakthroughs.

Maintenance

All live entries are maintained in data/projects.yaml; both READMEs are generated. Source interpretation and translation require review; metadata and links cannot certify scientific claims.

python3 scripts/sync_github_metadata.py
python3 scripts/render_readme.py
python3 scripts/verify_catalog.py
python3 -m unittest discover -s tests -v

recursive-self-improvement

rsi

self-training

Contributors

Picrew

12 commits

BobbyZhouZijian

1 commits

Similar repos

lobehub/awesome-rsi

A curated research map of Recursive Self-Improvement (RSI): models, agents, harnesses, embodied…

313

pinkbubblebubble/awesome-rsi

A curated, evidence-aware collection of recursive self-improvement research, agents, harnesses,…

theseus-labs-rsi/awesome-rsi

A curated list of papers and resources on Recursive Self-Improvement (RSI) for AI.

110

XMUDeepLIT/Awesome-Self-Evolving-Agents

A Survey of Self-Evolving Agents | A curated list of resources (surveys, papers, benchmarks, and…

450

selfimproving-agent/Awesome-Self-Improving-Agents

A curated list for Self-Improvement in Foundation Model Based Agentic Systems.

504

sukoji/awesome-self-evolving-agents

Curated map of self-evolving LLM agents — 240+ papers across 18 topics incl. recursive…

leezythu/Awesome-Harness-Self-Improvement

A curated reading list on harness engineering for recursive self-improvement of LLM agents (EN/ZH).

wannabeyourfriend/awesome-harness-evolution

Awesome Research Paper List for AI agent harness evolution: Foundation, Benchmark, Recipe, Position

Languages

Python

100.0%

Picrew/awesome-rsi

A curated collection of papers, research blogs, and open-source projects on recursive self-improvement (RSI), self-modifying agents, and iterative self-training.

Python

13 commits

updated Sep 20, 2026

See the code

README

Awesome RSI

A curated research map of recursive self-improvement: agents that revise their own machinery, models that generate their next training signal, and the evidence that defines their limits.

English | 中文

31 first-party blog posts · 45 research papers · 14 active GitHub projects

Last verified: 2026-09-20

Start Here

Reading path	What to look for
Self-modifying agents	Does the revised agent participate in its next improvement?
Iterative self-training	Do updated models generate the next training data, curriculum or rewards?
Theory and evaluation	Which assumptions and measurements support the loop?

Category Overview
Company Research Blogs
Papers and Official Code
Active GitHub Projects
Scope and Curation
Maintenance

Category Overview

Category	Resource	Entries
Mechanisms and Results	Blog	6
AI Research and Supporting Methods	Blog	15
Evaluation and Failure Modes	Blog	4
Research Agendas	Blog	2
Foundations and Historical Tutorials	Blog	4
Papers / Harness	Paper	15
Papers / Models	Paper	21
Papers / Theory and Evaluation	Paper	9
GitHub / Models / Training Research	GitHub project	2
GitHub / Models / Recursive Self-Training	GitHub project	1
GitHub / Harness / Self-Modification	GitHub project	4
GitHub / Harness / Prompt and Workflow Optimization	GitHub project	4
GitHub / Artifacts / Program Evolution	GitHub project	1
GitHub / Artifacts / Learned Skills	GitHub project	2
Total		90