When LLM Agents Meet Reinforcement Learning

AgentsMeetRL is an awesome list that summarizes open-source repositories for training LLM Agents using reinforcement learning:

• 🤖 The criteria for identifying an agent project are that it must have at least one of the following: multi-turn interactions or tool use (so TIR projects, Tool-Integrated Reasoning, are considered in this repo).
• ⚠️ This project is based on code analysis from open-source repositories using LLM coding agents, which may contain unfaithful cases. Although manually reviewed, there may still be omissions. If you find any errors, please don't hesitate to let us know immediately through issues or PRs - we warmly welcome them!
• 🚀 We particularly focus on the reinforcement learning frameworks, RL algorithms, rewards, and environments that projects depend on, for everyone's reference on how these excellent open-source projects make their technical choices. See [Click to view technical details] under each table.
• 📅 Last updated: 2026-08-26
• 🤗 Feel free to submit your own projects anytime - we welcome contributions!
• 📚 If you find this repository helpful for your research, please cite it via the "Cite this repository" button on the right sidebar.

Taxonomy:

• Base Framework: General-purpose RL training frameworks for LLM agents (e.g., veRL, OpenRLHF, trl)
• General/MultiTask: Agent systems trained/evaluated across multiple tasks or environments
• Search & RAG: Search-augmented reasoning agents that use retrieval tools to enhance LLM reasoning
• Web & GUI: Agents that interact with web browsers, mobile/desktop GUIs, or operating systems
• Tool-Use: Agents trained to invoke external tools (APIs, code executors, MCP, etc.)
• Code & SWE: Software engineering and code generation agents
• Reasoning: Reasoning agents with tool-integrated or multi-turn reasoning (math, QA, visual)
• Multi-Agent RL: Multi-agent collaboration, negotiation, or credit assignment via RL
• Memory: Agents that learn to manage, retrieve, or evolve memory
• Embodied: Agents operating in embodied/physical simulation environments
• Domain-Specific: RL agents for specialized domains (medical, OS tuning, etc.)
• Reward & Training: Process/outcome reward models and training methodologies for agents
• Safety: RL for agent safety alignment, adversarial red-teaming, and jailbreak defense/attack
• VLM Agent: Vision-language model agents trained with RL for multimodal interaction
• Self-Evolution: Agents that self-evolve via RL feedback loops (⚠️ definition still evolving in the community)
• Environment: Benchmarks, gyms, and sandbox environments for agent training/evaluation

Some Enumeration:

• Enumeration for Reward Type:
  • External Verifier: e.g., a compiler or math solver
  • Rule-Based: e.g., a LaTeX parser with exact match scoring
  • Model-Based: e.g., a trained verifier LLM or reward LLM
  • Custom

---

Updates

• 📢 2026-08 Update: Added 23 new repositories across 9 categories (Environment +8 [Echoverse/PAST-Bench/PatientAgentBench/LegalWorld/Evo-Bench/DocOps/ScrambleToolBench/DigiWorld], Search & RAG +5 [EviSD/GTA-RAG/LAPO, plus catch-up of GrepSeek/PyRAG], Base Framework +3 [Molt, AReno, and catch-up of Microsoft Orchard], Self-Evolution +2 [AgentOPSD/BaT], Code & SWE +1 [Lego-RL, harness-native RL inside Claude Code/OpenHands/OpenCode], Reward & Training +1 [Agent-G²], VLM Agent +1 [InSight-doc], Memory +1 [MemPrism], Tool-Use +1 [MUA-RL, promoted from Under Review — its code had in fact been public since 2025.8]). Every repo was opened and confirmed to contain real RL-training (or executable-environment) code. Papers whose code is still unreleased were left out (Qwen-UI-Agent, Qwen-CUA, UI-Mate, SearchMaster, RoMeRL, Agon, SINKFLEX-RL, GRASP, MAVEN, EviBack, ChemWorld) — see Under Review. Notably no qualifying new Safety, Embodied, or Multi-Agent RL repos appeared this window: that crop was uniformly SFT-only, inference-only, or code-withheld.
• 📢 2026-07 Update: Added 13 new repositories from late-Jun–Jul 2026 across 8 categories (Self-Evolution +3 [SEED/OPID/UCOB, the on-policy-distillation-for-agentic-RL line], VLM Agent +2 [VTS/VSeek, long-video search agents], Tool-Use +2 [Tool-RL-Box; plus catch-up of AWorld-RL], Environment +2 [SETA terminal envs, OpenAgent tool-generalization sandbox], Web & GUI +1 [SCALE-CUA], Embodied +1 [REAL], Memory +1 [Supersede], Domain-Specific +1 [FaithMed]). Every entry was verified by opening the repo and confirming real RL-training (or environment) code — papers whose code is not yet released (EvoCUA-1.5, DeepSearch-World, CompactionRL, GUICrafter, VideoSearcher, Xiaomi-GUI-0) were deliberately left out.
• 📢 2026-06 Update: Added 43 new repositories across 11 categories (VLM Agent +8, Search & RAG +7, Environment +6, Reward & Training +4, Base Framework/Tool-Use/Self-Evolution/Embodied +3 each, Web & GUI/Code & SWE/Domain-Specific +2 each). New since the last update: Harness-1, FastContext, OpenWebRL, Polar, AgentJet, HarnessX, APPO, SPADER, DeepRubric, Embodied-R1.5, SIRI; plus catch-up of earlier-2026 misses (Vision-DeepResearch, ARM-Thinker, PyVision-RL, Gen-Searcher, DataMind, Tool-R0, Agent World Model, VisGym, Gym-Anything, ChemCraft, OpAgent, etc.).
• 📢 2026-05 Update: Added 17 new repositories from Apr–May 2026 across 11 categories (notably General/MultiTask +4 [SkillZero/T²PO/SDAR/StraTA, mostly ZJU-REAL & related agentic RL methods], VLM Agent +3 [MTA-Agent/ParaVT/OpenSearch-VL, multimodal deep search & video tool use], Web & GUI +2 [ClawGUI/ToolCUA]). Moved CoEvolve to "Under Review" (code not yet released).
• 📢 2026-04 Update: Added 67 new repositories covering Apr 2025 – Apr 2026 across nearly every category (notably VLM Agent +9, Search & RAG +10, Web & GUI +7, Tool-Use +7). Also reclassified SkyRL (→ General) and SPIRAL (→ Multi-Agent), and updated the VAGEN entry to its NeurIPS'25 upstream repo.
• 📢 2026-03 Update: Restructured taxonomy from 12 to 16 categories (added Multi-Agent RL, Reward & Training, Safety, VLM Agent, Self-Evolution, Domain-Specific; merged GUI into Web & GUI; retired TextGame/Biomedical). Added ~70 new repositories covering Sep 2025 – Mar 2026, growing the total from ~134 to 205.

🤖 Use as a Claude Code Skill

This list is also packaged as a Claude Code Skill — agents-meet-rl — that turns the corpus into an on-demand assistant for agentic-RL training, evaluation, and experiment design: reward not moving, KL / entropy / length blow-ups, GRPO / PPO / DAPO knobs, retokenization drift, tool-call parse failures, long-horizon credit assignment, LLM-judge inconsistency, benchmark contamination, and framework / benchmark / algorithm selection — each answer anchored to specific papers and repos from this list. Backed by a machine-readable corpus of 405 projects (snapshot 2026-08-26). Once installed, Claude Code auto-invokes it whenever your question matches.

Install as a plugin (recommended):

/plugin marketplace add thinkwee/claude-plugins
/plugin install agents-meet-rl@thinkwee

Or install manually:

git clone https://github.com/thinkwee/AgentsMeetRL
cp -r AgentsMeetRL/skills/agents-meet-rl ~/.claude/skills/

Then just ask, e.g. "my GRPO search agent's reward is flat but eval keeps dropping" or "which RL framework should I pick for a multi-turn tool-use agent?" — the skill routes your symptom to fixes grounded in this corpus.

🔧 Base Framework

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
Libra	Async GRPO	Single	Outcome	Multi	Agentic RL post-training with resource-aware training and rollout	All (Custom/External/Rule)	Yes (search, code execution, validation tools)
Orchard	Online RL (vendored slime)	Single	Both	Multi	Harness-native RL (browser/computer-use/SWE)	Model/Rule	Yes (live browser, containers, agent harnesses)
AReno	GSPO/GRPO (+SFT/DPO)	Single	Outcome	Multi	Single-node RL post-training (Math/SWE-style/Games)	Custom	Yes (inspect_tree/read_file/rg/apply_patch/run_command)
Molt	REINFORCE/RLOO/GRPO/Dr.GRPO/GAE + On-policy Distillation	Single	Outcome	Multi	Domain-agnostic agentic RL (Math/Geometry/Chat)	Custom	Yes (Python exec, HTTP, VLM tools)
AgentJet	GRPO/PPO (swarm, multi-dim reward)	Both	Both	Multi	Swarm agentic RL (heterogeneous multi-agent, multi-task)	All (Custom/External/Rule)	Yes (tool calls, agent frameworks)
HarnessX	GRPO/PPO (slime/verl recipes)	Single	Outcome	Multi	Composable agent-harness foundry (ALFWorld/GAIA/WebShop/SWE-bench)	External + Custom	Yes (harness orchestrates tools/memory)
Dressage	GRPO	Both	Outcome	Multi	Agentic RL for any agent and sandbox (SWE-Gym/ALFWorld/HotpotQA)	External/Rule	Yes (whitebox: code/shell/file/retrieval; blackbox: opencode/openclaw/claude_code/codex)
Polar	GRPO	Both	Outcome	Multi	Agentic RL on any harness (SWE-Bench/SWE-Gym)	External Verifier	Yes (real agent harnesses: shell/Codex/Claude Code)
uni-agent	GRPO/GSPO (partial rollout, fully-async)	Single	Outcome	Multi	SWE-Bench/Search/General Agent (1000+ concurrent)	All	Yes (unified model/tool/env abstractions)
VeRL-Omni	FlowGRPO/DanceGRPO/Diffusion DPO	Single	Outcome	Single	Multimodal generation RL (image/video/omni)	Model/External	No
OpenClaw-RL	GRPO/OPD	Both	Both	Multi	Terminal/GUI/SWE/Tool-call	Model/External	Yes
Claw-R1	Generic RL Framework	Multi	Both	Multi	General Agent	All	Yes (Framework-agnostic)
Open-AgentRL	GRPO-TCR	Single	Both	Multi	Reasoning/GUI/Coding	Model (PRM)	Yes (SandboxFusion)
NeMo-RL	GRPO/DAPO/GDPO/DPO	Single	Outcome	Multi	Math/Reasoning/Code	Rule/External	No
RLinf	PPO/GRPO/DAPO/SAC/REINFORCE++/CrossQ/RLPD	Both	Both	Multi	Robotics/Math/Code/QA/VQA	All (Rule/Model/External)	Yes
siiRL	PPO/GRPO/CPGD/MARFT	Multi	Both	Multi	LLM/VLM/LLM-MAS PostTraining	Model/Rule	Planned
slime	GRPO/GSPO/REINFORCE++	Single	Both	Both	Math/Code	External Verifier	Yes
agent-lightning	PPO/Custom/Automatic Prompt Optimization	Multi	Outcome	Multi	Calculator/SQL	Model/External/Rule	Yes
AReaL	PPO	Both	Outcome	Both	Math/Code	External	Yes
ROLL	PPO/GRPO/Reinforce++/TOPR/RAFT++	Multi	Both	Multi	Math/QA/Code/Alignment	All	Yes
MARTI	PPO/GRPO/REINFORCE++/TTRL	Multi	Both	Multi	Math	All	Yes
Tunix	PPO/GRPO/GSPO-Token/DAPO/Dr.GRPO	Single	Outcome	Multi	Math/Code/Game	Rule/External	Yes
RL2	Dr. GRPO/PPO/DPO	Single	Both	Both	QA/Dialogue	Rule/Model/External	Yes
verifiers	GRPO	Multi	Outcome	Both	Reasoning/Math/Code	All	Code
prime-rl	GRPO/PPO	Multi	Outcome	Multi	Math/Code/Search	Model/External	Yes
oat	PPO/GRPO	Single	Outcome	Multi	Math/Alignment	External	No
veRL	PPO/GRPO	Single	Outcome	Both	Math/QA/Reasoning/Search	All	Yes
OpenRLHF	PPO/REINFORCE++/GRPO/DPO/IPO/KTO/RLOO	Multi	Both	Both	Dialogue/Chat/Completion	Rule/Model/External	Yes
trl	PPO/GRPO/DPO	Single	Both	Single	QA	Custom	No

💪 General/MultiTask

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
T2PO	T²PO (token+turn uncertainty-guided)	Single	Both	Multi	WebShop/ALFWorld/SearchQA/Embody/Game	Rule	Yes (search, web, embodied)
StraTA	Hierarchical GRPO + Strategic Trajectory Abstraction	Single	Outcome	Multi	ALFWorld (93.1%)/WebShop (84.2%)/SciWorld (63.5%)	Rule + Model (self-judge)	Yes (interactive long-horizon envs)
SDAR	Self-Distilled Agentic RL (GRPO + gated OPSD)	Single	Outcome	Multi	ALFWorld/WebShop/Search-QA	Rule	Yes (interactive envs)
SkillZero	In-Context Agentic RL (GRPO + skill-context curriculum withdrawal)	Single	Outcome	Multi	ALFWorld/WebShop/Search-QA	Rule	Yes (interactive envs + skill library)
MetaClaw	GRPO (LoRA)	Single	Process	Multi	General Agentic	Model (PRM)	Yes (Skill-augmented)
SkillRL	GRPO	Single	Outcome	Multi	ALFWorld/WebShop/Search	Rule	Yes (Web search, actions)
LLM-in-Sandbox	GRPO++	Single	Outcome	Multi	Math/Physics/Chemistry/Biomedicine/Long-context/IF/SWE	Rule	Yes (Code Sandbox w/ Terminal, File, Internet)
youtu-agent	Training-Free GRPO	Single	Outcome	Multi	Deep Research/Data Analysis/Tool-use	Model/External	Yes (Web search, code, file)
DEPO	KTO + Efficiency Loss	Single	Both	Multi	Agent (BabyAI/WebShop)	Rule	Yes
SPEAR	GRPO/GiGPO + SIL	Single	Both	Multi	Math/Agent	Rule/External	Yes (Search, Sandbox, Browser)
DeepAgent	ToolPO	Single	Outcome	Multi	ToolBench/ALFWorld/WebShop/GAIA/HLE	Model	Yes (16,000+ RapidAPIs)
AgentRL	GRPO/REINFORCE++/RLOO/ReMax/GAE	Single	Outcome	Multi	Agent Tasks	External	Yes
AgentGym-RL	PPO/GRPO/RLOO/REINFORCE++	Single	Outcome	Multi	Web/Search/Game/Embodied/Science	Rule/Model/External	Yes (Web, Search, Env APIs)
Agent_Foundation_Models	DAPO/PPO	Single	Outcome	Single	QA/Code/Math	Rule/External	Yes
Trinity-RFT	PPO/GRPO	Single	Outcome	Both	Math/TextGame/Web	All	Yes
SPA-RL-Agent	PPO	Single	Process	Multi	Navigation/Web/TextGame	Model	No
verl-agent	PPO/GRPO/GiGPO/DAPO/RLOO/REINFORCE++	Multi	Both	Multi	Phone Use/Math/Code/Web/TextGame	All	Yes
SkyRL	GRPO/PPO	Single	Both	Multi	Long-horizon Agents (SWE-Bench/Search/Math/SQL)	Rule/External/Custom	Yes
VAGEN	PPO/GRPO (World Modeling RL)	Single	Both	Multi	Navigation/TextGame/Multimodal	All	Yes
ART	GRPO	Multi	Both	Multi	TextGame	All	Yes
OpenManus-RL	PPO/DPO/GRPO	Multi	Outcome	Multi	TextGame	All	Yes

🔍 Search & RAG Agent

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
PyRAG	Curriculum shared-parameter GRPO (LoRA)	Multi (Decompose/Plan/Answer)	Both	Multi	Multi-hop RAG via executable Python synthesis	Rule-Based + Execution	Yes (Python exec + E5 retriever)
GrepSeek	SFT cold-start + GRPO	Single	Outcome	Multi	Direct corpus interaction (shell/grep, no index)	Rule-Based (token-F1 x format gate)	Yes (shell pipelines over raw corpus)
LAPO	GRPO + Leave-One-Turn Attribution	Single	Both	Multi	Multi-turn search QA (NQ/TriviaQA/HotpotQA/2Wiki)	Rule-Based (self-generated)	Yes (retrieval)
GTA-RAG	3-stage GRPO	Single	Both	Multi	Multi-hop QA over entity-document graph	Rule-Based (support-doc + EM)	Yes (graph + dense retrieval)
EviSD	GRPO + Evidence-Conditioned Self-Distillation	Single	Outcome	Multi	Search-augmented multi-hop QA	Rule-Based	Yes (search/retrieval)
Harness-1	GRPO	Single	Outcome	Multi	Long-horizon search (web/finance/patents) w/ state-externalizing harness	External + Rule	Yes (search/retrieval/rerank)
SlimSearcher	GRPO + Adaptive Reward Gating	Single	Outcome	Multi	Efficiency-aware deep research (GAIA/BrowseComp/xBench)	Custom + Rule	Yes (web search, browse)
DeepRubric	GRPO + rubric rewards	Single	Process	Multi	Deep research report synthesis (evidence-tree rubric)	Model + Rule (rubric)	Yes (search/browse/scholar)
SAAS	RL w/ boundary-aware reward (2-stage curriculum)	Single	Outcome	Multi	Self-aware agentic search (over-search mitigation, 7 QA sets)	Rule-Based	Yes (search)
CuSearch	GRPO + Search-Depth curriculum rollout	Single	Outcome	Multi	Agentic RAG multi-hop QA	Rule-Based (EM)	Yes (retrieval/search)
ORBIT	GRPO	Single	Outcome	Multi	Verifiable data-gen + RL for web search (Qwen3-4B)	External + Rule	Yes (web search)
LiteResearcher	Scalable Agentic RL (curriculum w/ lite virtual world)	Single	Outcome	Multi	Deep Research (GAIA 71.3% / Xbench-DS 78.0%, 4B SOTA)	Rule/External	Yes (local search/browse env, Milvus+PostgreSQL)
DR-Venus	GRPO + IGPO (info-gain turn-level) w/ agentic SFT	Single	Both	Multi	Edge-scale Deep Research (4B)	Intrinsic (info-gain) + Rule (format)	Yes (Search/Browse)
MR-Search	In-context Meta-RL (multi-episode credit)	Single	Outcome	Multi	Agentic search w/ self-reflection	Rule-Based	Yes (search)
ProRAG	GRPO + DGA (dual-granularity advantage)	Single	Both	Multi	Multi-hop RAG	Model (PRM via MCTS)	Yes (Retrieval)
O-Researcher	GRPO + RLAIF	Multi	Process	Multi	Deep Research (Zhihu-KOL/WideSearch/ELI5)	Model (LLM-as-Judge)	Yes (Search/Crawl)
Agentic-RAG-R1	GRPO	Single	Outcome	Multi	Knowledge-intensive QA	Rule/Model	Yes (Wiki/Doc search)
MemSearcher	Multi-context GRPO	Single	Outcome	Multi	Search/QA + Memory	Rule/Model	Yes (Web search + Memory)
DR Tulu	GRPO + evolving rubrics	Single	Outcome	Multi	Long-form Deep Research	Model (rubrics)	Yes (Search/MCP)
IGPO	GRPO + IGPO (Information Gain turn-level reward)	Single	Both	Multi	Multi-turn Search Agent (BrowseComp/-ZH)	Intrinsic (belief Δ) + Outcome	Yes (Search)
ReSeek	GRPO/PPO	Single	Both	Multi	QA/Search	Rule	Search/JUDGE
AutoGraph-R1	GRPO (via VeRL)	Single	Outcome	Multi	KG Construction for QA	Rule	Yes (Graph retrieval)
WebSeer	GRPO-style	Single	Outcome	Multi	Web Search QA (w/ self-reflection)	Rule/Model	Yes (Search)
HiPRAG	PPO	Single	Process	Multi	Efficient Agentic RAG	Model/Rule	Yes (Retrieval)
Tree-GRPO	GRPO/Tree-GRPO	Single	Outcome	Multi	Search	Rule	Search
DeepResearch	RL-based	Single	Outcome	Multi	Deep Research	Model	Yes (Search, Browse)
DeepDive	GRPO	Single	Outcome	Multi	KG-augmented Search	Rule	Yes (KG + Search)
ASearcher	PPO/GRPO + Decoupled PPO	Single	Outcome	Multi	Math/Code/SearchQA	External/Rule	Yes
SSRL	GRPO	Single	Outcome	Multi	Self-Search	Rule	Yes (Self-search)
Research-Venus	GRPO	Single	Both	Multi	Deep Research	Model (atomic thought)	Yes (Search)
Graph-R1	GRPO/REINFORCE++/PPO	Single	Outcome	Multi	KGQA	Rule (EM/F1)	Yes (Graph retrieval)
Kimi-Researcher	REINFORCE	Single	Outcome	Multi	Research	Outcome	Search, Browse, Coding
R-Search	PPO/GRPO	Single	Both	Multi	QA/Search	All	Yes
R1-Searcher-plus	Custom	Single	Outcome	Multi	Search	Model	Search
StepSearch	PPO	Single	Process	Multi	QA	Model	Search
AutoRefine	PPO/GRPO	Multi	Both	Multi	RAG QA	Rule	Search
ZeroSearch	PPO/GRPO/REINFORCE	Single	Outcome	Multi	QA/Search	Rule	Yes
ReasonRAG	DPO + MCTS-based PRM	Single	Process	Multi	Multi-hop QA	Model (PRM)	Yes (Wikipedia search)
VRAG	GRPO	Single	Both	Multi	Visually-rich RAG	Rule/Model	Yes (Visual retrieval)
MaskSearch	DAPO	Single	Outcome	Multi	RAMP Pretraining + QA	Rule/Model	Yes (Search)
R3-RAG	PPO	Single	Both	Multi	Multi-hop QA	Rule	Yes (Retrieval)
O2-Searcher	GRPO	Single	Outcome	Multi	Open-ended QA	Rule/Model	Yes (Search)
s3	GRPO	Single	Outcome	Multi	RAG / Medical QA	Model (Gain-Beyond-RAG)	Yes (Retrieval)
knowledge-r1	GRPO	Single	Outcome	Multi	Knowledge-intensive QA (KB-aware)	Rule	Yes (Retrieval)
WebThinker	DPO	Single	Outcome	Multi	Reasoning/QA/Research	Model/External	Web Browsing
DeepResearcher	PPO/GRPO	Multi	Outcome	Multi	Research	All	Yes
Search-R1	PPO/GRPO	Single	Outcome	Multi	Search	All	Search
R1-Searcher	PPO/DPO	Single	Both	Multi	Search	All	Yes
C-3PO	PPO	Multi	Outcome	Multi	Search	Model	Yes
DeepRetrieval	GRPO	Single	Outcome	Multi	Query Generation/IR	Rule	Yes (Search)

🌐 Web & GUI Agent

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
SCALE-CUA	GRPO (fully async, Frontier Sampling + visual context segmentation)	Single	Outcome	Multi	Computer Use (OSWorld, ScienceBoard; 24K+ synthesized verifiable tasks)	External Verifier (executable judge functions)	Yes (GUI actions in Docker desktop)
OpenWebRL	GRPO (online multi-turn)	Single	Both	Multi	Visual web browsing on live sites (WebVoyager/Online-Mind2Web)	Rule + Model (format + LLM-judge)	Yes (Playwright browser)
ToolCUA	Tool-Bootstrapped GUI RFT + Online Agentic RL (Tool-Efficient Path Reward)	Single	Both	Multi	Computer Use (OSWorld-MCP, hybrid GUI+tool)	Rule (path-efficiency)	Yes (GUI actions + structured tool calls)
ClawGUI	GiGPO + Process Reward Model	Single	Both	Multi	Mobile GUI (Android/HarmonyOS/iOS, MobileWorld)	Rule + Model (PRM)	Yes (GUI + hybrid CLI-GUI + persistent memory)
OpAgent	Online agentic RL (GRPO/PPO)	Multi	Both	Multi	Web navigation (WebArena 71.6% pass@5)	Rule + Model (RDTree + WebJudge)	Yes (Playwright browser)
GUI-Libra	KL-regularized GRPO (Partially Verifiable RL)	Single	Outcome	Multi	GUI (AndroidWorld/WebArena/Online-Mind2Web)	Rule	Yes
MobileAgent	semi-online RL	Single	Both	Multi	MobileGUI/Automation	Rule	Yes
UI-TARS	Multi-turn RL	Single	Both	Multi	GUI (Cross-platform)	Model	Yes (GUI actions)
MobileRL	AdaGRPO (Difficulty-Adaptive)	Single	Outcome	Multi	Mobile GUI (AndroidWorld/AndroidLab)	Rule	Yes (Android)
DART-GUI	Decoupled GRPO	Single	Outcome	Multi	GUI (OSWorld)	Rule	Yes
Mano-P	Three-stage SFT→Offline RL→Online RL	Single	Both	Multi	GUI (OSWorld)	Rule	Yes
InfiGUI-G1	AEPO	Single	Outcome	Single	GUI/Grounding	Rule	No
gui-rcpo	RCPO	Single	Outcome	Single	GUI Grounding	Rule (self-supervised)	No
UI-AGILE	GRPO	Single	Outcome	Single	GUI Grounding	Rule (continuous)	No
GUI-G2	GRPO (Gaussian Reward)	Single	Outcome	Single	GUI Grounding	Rule (continuous)	No
MagicGUI	Reinforcement Fine-Tuning (RFT)	Single	Outcome	Multi	Mobile GUI	Model/Rule	Yes
Grounding-R1	GRPO	Single	Outcome	Multi	GUI Grounding	Model	Yes
AgentCPM-GUI	GRPO	Single	Outcome	Multi	Mobile GUI	Model	Yes
TTI	REINFORCE/BC	Single	Outcome	Multi	Web	External	Web Browsing
GTA1	GRPO-style (click-success reward)	Single	Outcome	Multi	GUI Grounding (OSWorld/ScreenSpot-Pro)	Rule	Yes
SE-GUI	GRPO	Single	Both	Single	GUI Grounding	Rule	Yes
ARPO	GRPO	Single	Outcome	Multi	GUI	External	Computer Use
GUI-G1	GRPO	Single	Outcome	Single	GUI	Rule/External	No
WebAgent-R1	M-GRPO	Single	Outcome	Multi	Web Navigation (WebArena-Lite)	Rule (task success)	Yes (Web browsing)
ZeroGUI	Online RL	Single	Outcome	Multi	GUI Agent	Rule	Yes (GUI actions)
GUI-R1	GRPO	Single	Outcome	Multi	GUI	Rule	No
InfiGUI-R1	RL + sub-goal guidance	Single	Both	Multi	GUI Reasoning	Rule	Yes
UI-R1	GRPO	Single	Process	Both	GUI	Rule	Computer/Phone Use
CollabUIAgents	DPO (credit re-assignment)	Multi	Process	Multi	GUI (Mobile + Web)	Model (LLM)	Yes (GUI interaction)
DigiQ	Value-based offline RL	Single	Outcome	Multi	Android Device Control	Model (Q-function)	Yes
GUI-Agent-RL	Value-based RL (VEM)	Single	Outcome	Multi	GUI (Web Shopping)	Model	Yes
WebAgent	DAPO	Multi	Process	Multi	Web	Model	Yes

🔨 Tool-Use Agent

📋 Click to view technical details

| Github Repo | RL Algorithm | Single/Multi Agent | Outcome/Process Reward | Single/Multi Turn | Task | Reward Type | Tool usage |

MUA-RL	GRPO	Single	Outcome	Multi	Multi-turn user-interacting agentic tool use (tau-bench/tau2-bench/ACEBench)	Rule-Based (task completion)	Yes (simulated user + tool APIs)
Tool-RL-Box	GRPO + supervisory signals (anti format-collapse)	Single	Process	Multi	Multi-step function calling (FCL / ToolACE, pluggable tool servers)	Model (LLM-judge error taxonomy) + Rule	Yes (function-calling tools)
SPADER	GRPO + Step-wise Peer Advantage (SPA)	Single	Both	Multi	Long-horizon tool-augmented multi-answer QA (QAMPARI)	Rule-Based (entity-match + diversity)	Yes (search)
APPO	APPO (procedure-aware branching; extends ARPO/GRPO)	Single	Process	Multi	Multi-turn TIR (reasoning+search+code, 13 benchmarks)	Rule-Based	Yes (search + code)
AgenticQwen	Multi-round RL (Reasoning RL + Agentic RL w/ dual data flywheels)	Single	Outcome	Multi	Industrial Tool Use (search, data analysis, tau-bench airline/retail/telecom)	Rule + Model (rubric)	Yes (Python interpreter, web search, mock tools)
Agent-STAR	GRPO + dense/curriculum reward (STAR recipe)	Single	Both	Multi	Long-horizon tool-using agents (TravelPlanner, ReAct up to 60 turns)	Rule + External	Yes (planning APIs)
ToolOrchestra	End-to-end RL (outcome+efficiency+preference)	Single	Both	Multi	Tool orchestration / agentic workflows	All	Yes (Search/Code/LLMs)
ToolMaster	SFT + GRPO (trial-then-execute)	Single	Outcome	Multi	Tool trialing + execution (ToolHop/TMDB/StableToolBench)	Rule/External	Yes (Simulated tools)
MATPO	GRPO (multi-agent)	Multi	Outcome	Multi	Tool-use/Search	Rule	Yes (MCP: Serper, Web scraping)
AWorld-RL	Collection: RODS / HardGen / FunReason-MT / Environment Tuning / V2P / RAG-R1	Both	Both	Multi	Multi-turn function calling + GUI grounding + deep search (BFCL etc.)	Rule + Model (progress reward)	Yes (function calls, GUI, search)
CodeGym	GRPO-family	Single	Outcome	Multi	Synthetic Multi-turn Tool-Use	Rule (verifiable)	Yes (Synthesized tools)
UserRL	GRPO (multi-turn credit)	Single	Both	Multi	User-centric (Function/Persuade/Search/Tau Gyms)	Model/External	Yes
ToolBrain	GRPO/DPO	Single	Outcome	Multi	Agentic tool training	Rule/Model	Yes (User-defined tools)
Tool-R1	Policy optimization (PPO-style)	Single	Outcome	Multi	Agentic Tool Use (GAIA)	Model + External	Yes (Python exec)
MiroRL	GRPO	Single	Both	Multi	Reasoning/Planning/ToolUse	Rule-based	MCP
verl-tool	PPO/GRPO	Single	Both	Both	Math/Code	Rule/External	Yes
Multi-Turn-RL-Agent	GRPO	Single	Both	Multi	Tool-use/Math	Rule/External	Yes
Tool-N1	PPO	Single	Outcome	Multi	Math/Dialogue	All	Yes
Tool-Star	PPO/DPO/ORPO/SimPO/KTO	Single	Outcome	Multi	Multi-modal/Tool Use/Dialogue	Model/External	Yes
RL-Factory	GRPO	Multi	Both	Multi	Tool-use/NL2SQL	All	MCP
calculator_agent_rl	GRPO	Single	Outcome	Multi	Calculator Tool Use	Model (Claude-judge)	Yes
ReTool	PPO	Single	Outcome	Multi	Math	External	Code
ToolRL	GRPO/PPO	Single	Outcome	Multi	Tool Learning	Rule/External	Yes
AWorld	GRPO	Both	Outcome	Multi	Search/Web/Code	External/Rule	Yes
Agent-R1	PPO/GRPO	Single	Both	Multi	Tool-use/QA	Model	Yes
ReCall	PPO/GRPO/RLOO/REINFORCE++/ReMax	Single	Outcome	Multi	Tool-use/Math/QA	All	Yes

💻 Code & SWE Agent

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
Lego-RL	PPO/GRPO/GSPO (sync & async)	Single	Outcome	Multi	SWE-bench Verified inside real coding harnesses	External Verifier (Harbor sandbox)	Yes (native harness bash/file-edit/test)
FastContext	Task-grounded RL	Single	Outcome	Multi	Repo-explorer subagent (context gathering + citations)	Rule-Based	Yes (Read/Glob/Grep, parallel)
SWE-Edit	GRPO (adaptive mode selection)	Multi (Viewer + Editor subagents)	Outcome	Multi	SWE-bench Verified (find-replace vs whole-file rewrite)	Rule/External (test-based)	Yes (bash, file ops, viewer subagent)
CodeScout	GSPO	Single	Outcome	Multi	Repo-level code search/localization (terminal)	Rule-Based (F1)	Yes (terminal: rg/sed/cat)
CUDA-Agent	Agentic RL (staged)	Single	Outcome	Multi	CUDA Kernel Generation	Rule (correctness + performance)	Yes (compile/verify/profile)
SWE-World	RL with learned world model (SWT + SWR)	Single	Both	Multi	Docker-free SWE (SWE-Bench Verified)	Model (surrogate) + Rule	Yes
LLM-in-Sandbox	GRPO++	Single	Outcome	Multi	Code/SWE + General (Math/Sci/Bio)	Rule	Yes (Code Sandbox w/ Terminal, File, Internet)
CUDA-L2	Contrastive RL	Single	Outcome	Single	HGEMM / CUDA Matmul	Rule (TFLOPs)	Yes (compile/benchmark)
PPP-Agent	PPP-RL	Single	Both	Multi	SWE/Research	Rule+Model	Search, Ask, Browse
DeepAnalyze	Curriculum RL	Single	Outcome	Multi	Data Science	Rule/External	Yes (Code exec)
RepoDeepSearch	GRPO	Single	Both	Multi	Search/Repair	Rule/External	Yes
CUDA-L1	Contrastive RL	Single	Outcome	Single	CUDA Optimization	Rule (performance)	No
SWE-Swiss	Two-stage RL curriculum	Single	Outcome	Multi	SWE (Localization/Repair/Unit-Test)	Rule (test-based)	Yes
MedAgentGym	SFT/DPO/PPO/GRPO	Single	Outcome	Multi	Medical/Code	External	Yes
CURE	PPO	Single	Outcome	Single	Code	External	No
Time-R1	PPO/GRPO/DPO	Multi	Outcome	Multi	Temporal	All	Code
ML-Agent	Custom	Single	Process	Multi	Code	All	Yes
R1-Code-Interpreter	GRPO	Single	Outcome	Multi	Code Interpretation	Rule/External	Yes (Code exec)
digitalhuman	PPO/GRPO/ReMax/RLOO	Multi	Outcome	Multi	Empathy/Math/Code/MultimodalQA	Rule/Model/External	Yes
Skywork-OR1	Large-scale rule-based RL (GRPO variant)	Single	Outcome	Single	Math + Code (AIME/LiveCodeBench)	Rule (verifiable)	No
sweet_rl	DPO	Multi	Process	Multi	Design/Code	Model	Web Browsing
swe-rl	RL-based	Single	Outcome	Single	SWE (SWE-bench)	Rule (similarity)	No
CTRL	RL (critique-revision)	Single	Process	Multi	Code Refinement	Model	Yes (Code exec)
AceCoder	GRPO	Single	Outcome	Single	Code Generation	External (test cases)	Yes
rllm	PPO/GRPO	Single	Outcome	Multi	Code Edit	External	Yes
open-r1	GRPO	Single	Outcome	Single	Math/Code	All	Yes

🤔 Reasoning Agent

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
Agent0	ADPO	Multi	Process	Multi	Math/Visual	Model/Verifier	Yes
KG-R1	GRPO/PPO	Single	Both	Multi	KGQA	Rule/Model	KG Retrieval
AgentFlow	Flow-GRPO	Single	Outcome	Multi	Search/Math/QA	Model/External	Yes
THOR	Hierarchical GRPO (trajectory+step)	Single	Both	Multi	Math (MATH500/AIME/Olympiad)	External (SandboxFusion)	Yes (Python)
Tool-Light	Self-Evolved DPO	Single	Outcome	Multi	Tool-Integrated Reasoning	Model (preference)	Yes (FlashRAG/Python)
ARPO	GRPO	Single	Outcome	Multi	Math/Coding	Model/Rule	Yes
terminal-bench-rl	GRPO	Single	Outcome	Multi	Coding/Terminal	Model+External Verifier	Yes
AutoTIR	PPO	Single	Outcome	Multi	Autonomous Tool Selection (QA/Math/IF)	Rule	Yes (Search/Python)
MOTIF	GRPO	Single	Outcome	Multi	QA	Rule	No
cmriat/l0	PPO	Multi	Process	Multi	QA	All	Yes
agent-distillation	PPO	Single	Process	Multi	QA/Math	External	Yes
EasyR1	GRPO	Single	Process	Multi	Vision-Language	Model	Yes
AutoCoA	GRPO	Multi	Outcome	Multi	Reasoning/Math/QA	All	Yes
ToRL	GRPO	Single	Outcome	Single	Math	Rule/External	Yes
ReMA	PPO	Multi	Outcome	Multi	Math	Rule	No
Agentic-Reasoning	Custom	Single	Process	Multi	QA/Math	External	Web Browsing
SimpleTIR	PPO/GRPO (with extensions)	Single	Outcome	Multi	Math, Coding	All	Yes
openrlhf_async_pipline	PPO/REINFORCE++/DPO/RLOO	Single	Outcome	Multi	Dialogue/Reasoning/QA	All	No

👥 Multi-Agent RL

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
Maestro	Outcome RL (lightweight orchestrator over frozen expert ensembles)	Multi (orchestrator + frozen experts)	Outcome	Multi	10 multimodal benchmarks (math/chart/HR/domain — 70.1% avg, beats GPT-5 & Gemini-2.5-Pro)	External	Yes (expert models + 2-tier skill library: OCR/detection/visual)
DrMAS	GRPO (agent-wise)	Multi	Outcome	Multi	Multi-agent LLM Systems	Rule	No
MarsRL	RLVR (agent-specific rewards)	Multi	Both	Multi	Math Reasoning (AIME/BeyondAIME)	Rule (verifiable)	No
PettingLLMs	AT-GRPO	Multi	Both	Multi	Game/Code/Math/Planning	Rule (verifiable)	No
MASPRM	PRM (trained from MCTS rollouts)	Multi	Process	Multi	Reasoning (GSM8K/MATH/MMLU)	Learned PRM	No
MrlX	M-GRPO (hierarchical)	Multi	Outcome	Multi	Deep Research (GAIA/XBench)	Rule + Model	Yes (Search)
CoMAS	RL w/ LLM-Judge intrinsic reward	Multi	Process	Multi	Co-evolving Reasoning	Model	No
MAPoRL	PPO	Multi	Outcome	Multi	Collaborative LLM Tasks	Rule	No
CoMLRL	MAGRPO / MAREINFORCE / MARLOO	Multi	Outcome	Multi	Writing / Code / Minecraft	Custom	Minimal
ARIA	REINFORCE	Both	Process	Multi	Negotiation/Bargaining	Other	No
SPIRAL	Role-conditioned Advantage Estimation (RAE)	Multi	Outcome	Multi	Zero-sum Games (TicTacToe/Kuhn/Negotiation)	Rule	No
AMPO	BC/AMPO(GRPO improvement)	Multi	Outcome	Multi	Social Interaction	Model-based	No
FlowReasoner	GRPO	Multi	Outcome	Multi	Multi-agent Workflow Design	Rule	Yes
MARFT	MARFT paradigm (action+token level)	Multi	Both	Multi	Research / Math	Rule	Yes

🧠 Memory

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
MemPrism	GRPO/GiGPO	Single	Both	Multi	Memory-view selection for ALFWorld/ALFRED + Mind2Web	Rule-Based	Yes (memory-view action + env actions)
Supersede	GRPO (+ LoRA)	Single	Outcome	Multi	Memory-update gap: keeping notes current across sessions (LongMemEval knowledge-update)	Rule-Based (answered_current / stale_penalty)	Yes (capped notes memory as action space)
AgeMem	Step-wise GRPO (3-stage progressive RL)	Single	Process	Multi	Unified LTM/STM management (memory ops as tools)	Rule (task accuracy + memory quality)	Yes (store/retrieve/update/summarize/discard memory tools)
Mem-alpha	GRPO	Single	Outcome	Multi	Long-context QA + Memory Construction	Rule (downstream QA)	Yes (memory tools)
MEM1	PPO/GRPO	Single	Outcome	Multi	WebShop/GSM8K/QA	Rule/Model	Yes
M3-Agent	RL-based	Single	Outcome	Multi	Long-video QA (M3-Bench)	Rule/Model	Yes (multimodal memory graph)
Memento	soft Q-Learning	Single	Outcome	Multi	Research/QA/Code/Web	External/Rule	Yes
MemAgent	PPO, GRPO, DPO	Multi	Outcome	Multi	Long-context QA	Rule/Model/External	Yes

🦾 Embodied

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
REAL	GRPO/GSPO (online RL over an MCP tool interface)	Single	Outcome	Multi	Open-world mobile manipulation in Isaac Sim (REAL-Bench, 241 tasks)	External Verifier (target world-state check)	Yes (8 MCP tools: navigate_to/pick/place/ask/...)
Embodied-R1.5	RFT (GRPO-family multimodal)	Single	Outcome	Multi	Embodied foundation model w/ Planner-Grounder-Corrector closed-loop	Rule-Based	No (closed-loop PGC)
AVA-VLA	PPO (latent reasoning as sequential decision)	Single	Both	Multi	VLA manipulation (LIBERO/ALOHA), latent CoT w/ early-exit	External (task success) + Custom	No (closed-loop manipulation)
WorldVLN	Action-aware GRPO	Single	Both	Multi	Aerial (UAV) vision-language navigation (closed-loop)	Rule + Model	No (closed-loop UAV control)
Embodied-R1	GRPO	Single	Outcome	Single	Grounding/Waypoint	Rule	No
VIKI-R	GRPO (RFT after SFT)	Multi	Outcome	Multi	Embodied Multi-Robot Cooperation (VIKI-Bench)	Rule + Model	No
STeCa	DPO (RFT)	Single	Both	Multi	Embodied/Household	Rule/MC	Environment Actions

🏷️ Domain-Specific

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
FaithMed	SFT (LLaMA-Factory) + agentic RL w/ process reward	Single	Both	Multi	Faithful evidence-based medical QA (MedQA/MedMCQA/MedXpertQA/...)	Rule + Model (step-level faithfulness)	Yes (medcorp evidence search)
Gene-Disease-Curation	Process-supervised Multi-Agent RL	Multi	Both	Multi	Clinical gene-disease validity curation (ClinGen)	Model (process) + Rule (outcome)	Yes (agent-as-tool, evidence synthesis)
MedSAM-Agent	GRPO (via veRL)	Single	Both	Multi	Medical Image Segmentation	Model (clinical fidelity)	Yes (SAM/MedSAM2)
ChemCraft	SMILES-GRPO	Single	Both	Multi	Chemical LM orchestrating chemistry tools (molecular design/synthesis)	External (dense chemical) + Rule	Yes (chemical agent sandbox)
Doctor-R1	Experiential Agentic RL	Multi	Both	Multi	Clinical inquiry & diagnosis	Model + Rule + safety veto	No
Alpha-R1	GRPO	Single	Outcome	Multi	Alpha factor screening (with real-time news)	External (portfolio returns) + Model	Yes
OS-R1	GRPO (via veRL)	Single	Outcome	Multi	Linux Kernel Tuning	Rule	Yes (LightRAG, kernel config)
MMedAgent-RL	Unknown	Multi	Unknown	Unknown	Unknown	Unknown	Unknown
MedResearcher-R1	GRPO-based (SFT + Online RL)	Single	Outcome	Multi	Medical Deep Research (MedBrowseComp)	Rule + Model	Yes (Search/KG)
LegalDelta	GRPO (CoT-guided info-gain)	Single	Process	Multi	Legal Reasoning	Model + Rule	No
DoctorAgent-RL	GRPO	Multi	Both	Multi	Consultation/Diagnosis	Model/Rule	No
Biomni	TBD	Single	TBD	Single	scRNAseq/CRISPR/ADMET/Knowledge	TBD	Yes

🎯 Reward & Training Methodology

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
DRACO	GRPO w/ rubric-attributed step-level advantages	Single	Process	Multi	AppWorld / τ-Bench (tool use)	Model (LLM judge on dynamic rubrics)	Yes
Agent-G²	GRPO + Gaussian-guided adaptive-depth exploration	Single	Outcome	Multi	ALFWorld / WebShop	Rule-Based/External	Yes (interactive env actions)
AgentV-RL	RL (verl) training an agentic verifier	Single	Process	Multi	Tool-augmented deliberative verifier (reward model)	Model-Based	Yes (verifier invokes tools, e.g. code)
DataMind	RL w/ generative PRM (DataPRM)	Single	Process	Multi	Agentic data analysis (Python/SQL; ScienceAgentBench/DABench)	Model (PRM) + External (execution)	Yes (code-based multi-turn)
ARLArena	SAMPO (Stable Agentic Policy Optimization)	Single	Outcome	Multi	Stable agentic RL across web/embodied/math/game/search	External + Rule	Yes (code/web/search/embodied)
Agent-RRM	Agentic RL w/ trained reasoning RM	Single	Process	Multi	Reward model for agents (web nav, multi-hop QA)	Model-Based (RRM)	Yes (agentic trajectories)
ToolPRMBench	N/A (Benchmark)	Single	Process	Multi	Tool-Use	Rule/Model	Yes
RLVR-World	RLVR	Single	Outcome	Multi	World Modeling (Language/Video)	Model (verifiable)	No
AgentProg	Online RL w/ progress reward	Single	Process	Multi	GUI Agent Training	Model (ProgRM)	Yes
AgentPRM	PPO/DPO + PRM	Single	Process	Multi	ALFWorld/General	Model (PRM)	Yes
Agentic-Reward-Modeling	DPO/Best-of-N	Single	Outcome	Single	General Instruction	Model (Reward Agent)	Yes (Verification)
AgentRM	MCTS/RM-guided	Single	Outcome	Multi	9 Agent Tasks	Model (regression PRM)	Yes

🛡️ Safety

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
ToolSafe	Multi-task GRPO	Single	Process	Multi	Tool-Invocation Safety Guardrail	Rule + Model	Yes (tool monitoring)
TROJail	Multi-turn GRPO variant	Single	Both	Multi	Multi-turn Jailbreak Attack	Model (harmfulness judge) + Rule	Yes (target LLM)
SafeSearch	PPO (GAE/GRPO)	Single	Both	Multi	Safe QA/Search	Rule + Model	Search
Jailbreak-R1	GRPO (3-stage: imitation→warm-up→progressive)	Single	Both	Multi	Red-teaming Prompt Generation	Model (judge)	Yes (target LLM)
GuardReasoner-VL	Online RL w/ rejection sampling	Single	Both	Multi	VLM Safety Guard (multimodal)	Rule + Model	No
xJailbreak	RL	Single	Outcome	Multi	Jailbreaking	Model (embedding)	Yes (iterative)
Auto-RT	PPO	Single	Outcome	Multi	Red Teaming	Model	Yes (strategy exploration)
RLbreaker	Custom PPO	Single	Outcome	Multi	Jailbreaking	Model	Yes (mutator selection)
curiosity_redteam	RL + Curiosity	Single	Outcome	Multi	Red Teaming	Model	Yes (iterative query)

👁️ VLM Agent

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
InSight-doc	SFT cold-start + GRPO	Single	Both	Multi	Long/multi-page document VQA (DUDE/MP-DocVQA)	Rule-Based	Yes (zoom-in tool)
VTS	GRPO (multi-turn tree rollout w/ backtracking)	Single	Both	Multi	Grounded long-video QA (search video as an adaptive temporal tree)	Rule-Based (answer + format + evidence IoU)	Yes (zoom_in/zoom_out/shift/answer)
VSeek	GRPO	Single	Both	Multi	Long-video QA as multi-turn evidence retrieval (LongVideoBench/LVBench/Video-MME/MLVU)	Rule-Based (neuro-symbolic temporal-logic grounding)	Yes (ViCLIP+FAISS retrieval server)
HyperEyes	Dual-grained efficiency-aware RL	Single	Both	Multi	Parallel multimodal search (unified grounded search)	Custom + Rule	Yes (visual grounding + retrieval)
ODE	GRPO (async) + SFT cold-start	Single	Both	Multi	Visual-native multimodal deep search (9-tool harness)	External + Rule	Yes (web/image/visual search, code)
ParaVT	PARA-GRPO (Parseability-Anchored, Ratio-gAted)	Multi (main + parallel sub-agents w/ shared weights)	Both (outcome + targeted format)	Single-turn parallel	Long-video understanding (VideoMME/LongVideoBench/LVBench/MLVU/MMVU/Charades-STA)	Rule + Model	Yes (parallel video-window crop tools)
OpenSearch-VL	Multi-turn fatal-aware GRPO	Single	Outcome	Multi	Multimodal Deep Search (Qwen3-VL base)	Rule + Model (LLM judge)	Yes (text/image search, OCR, crop, sharpen, SR, perspective)
MTA-Agent	DAPO (w/ cached tool interactions)	Single	Outcome	Multi	Multimodal Deep Search (21K MTA-Vision-DeepSearch; 32B beats GPT-5 54.63%)	Rule/External	Yes (web search, web read, Google Lens, image search)
Gen-Searcher	GRPO (after SFT)	Single	Both	Multi	Search-augmented image-generation deep research	Model (dual text+image)	Yes (search, image search, browse, image-gen)
MM-DeepResearch	Multi-turn agentic GRPO	Single	Both	Multi	Multimodal agentic search baseline	Model (judge) + Rule	Yes (image/text search engines)
PyVision-RL	GRPO	Single	Both	Multi	Agentic image+video understanding w/ dynamic Python tooling	External + Custom	Yes (Python-as-tool, frame sampling)
Vision-DeepResearch	GRPO (after cold-start SFT)	Single	Both	Multi	Multimodal deep-research MLLM (dozens of turns)	External + Rule	Yes (visual+textual search, browse)
ARM-Thinker	GRPO (two-stage)	Single	Both	Multi	Agentic multimodal reward modeling (Think-Act-Verify)	External + Rule	Yes (zoom/crop, doc retrieval, validators)
CodeDance	GRPO/DAPO (agent-loop)	Single	Both	Multi	Executable visual reasoning (visual search/math/chart via code)	External + Custom	Yes (Python sandbox: crop/draw/plot)
DeepEyesV2	Outcome RL	Single	Outcome	Multi	Multimodal Reasoning	Rule	Yes (Code exec, Web search)
Mini-o3	GRPO	Single	Outcome	Multi	Visual Search (V*/HR-Bench)	Rule	Yes (image crop)
VisionThink	GRPO w/ LLM-as-Judge	Single	Outcome	Multi	Efficient VQA	Model (LLM-Judge)	Yes (hi-res request)
multimodal-search-r1	GRPO	Single	Outcome	Multi	Multimodal Search	Rule	Yes (Search)
AutoVLA	GRPO (RFT after SFT)	Single	Outcome	Multi	Autonomous Driving (nuScenes/nuPlan/Waymo)	Rule (PDMS)	No
VDeepEyes	PPO/GRPO	Multi	Process	Multi	VQA	All	Yes
CoSo	Soft RL (counterfactual)	Single	Outcome	Multi	Android/Card/Embodied	Rule	Yes
Pixel-Reasoner	Curiosity-driven GRPO	Single	Both	Multi	Visual Reasoning (V*/TallyQA/Info-VQA)	Rule + Model	Yes (zoom/select-frame)
Visual-ARFT	GRPO (agentic RFT)	Single	Outcome	Multi	Multimodal Agentic Tool Use (MAT-Search/Coding)	Rule	Yes (Search/Python)
VTool-R1	RFT (GRPO-based)	Single	Outcome	Multi	Chart/Table VQA	Rule	Yes (Python visual tools)
OpenThinkIMG	V-ToolRL (GRPO)	Single	Outcome	Multi	Chart Reasoning	Rule	Yes (GroundingDINO/SAM/OCR/crop)
Chain-of-Focus	AGAR (GRPO)	Single	Outcome	Multi	Visual Reasoning (V*)	Rule (outcome+format)	Yes (zoom-in)
GRIT	GRPO-GR (Grounded Reasoning)	Single	Outcome	Single	Visual Reasoning (bbox)	Rule	Yes (bbox)
AlphaDrive	GRPO	Single	Outcome	Multi	Autonomous Driving	Rule (4 planning rewards)	No
VSC-RL	Variational RL	Single	Outcome	Multi	Mobile Device Control	Rule	Yes
RL4VLM	PPO	Single	Outcome	Multi	GymCards/ALFWorld	Rule	Yes

🔄 Self-Evolution

⚠️ Note: The definition of "Self-Evolution" in the context of RL for LLM agents is still evolving and not yet well-established. This category currently collects works whose paper titles explicitly contain "self-evolving" or "self-evolution", where the agent improves itself through RL-driven feedback loops.

📋 Click to view technical details

Github Repo	RL Algorithm	Single/Multi Agent	Outcome/Process Reward	Single/Multi Turn	Task	Reward Type	Tool usage
BaT	SFT init + GRPO + BiCuRL (bilevel curriculum)	Single	Both	Multi	Medical research agent (AutoMedBench)	Model-Based (stage rubrics) + Rule	Yes (stage skills + fixed exec env)
AgentOPSD	GRPO + critic-free recursive turn-level self-distillation	Single	Both	Multi	ALFWorld / WebShop / Search-QA	Rule-Based + External	Yes (env actions, search)
SEED	GRPO + self-evolving on-policy distillation (hindsight-skill SFT → OPD during RL)	Single	Outcome	Multi	ALFWorld / WebShop / Search-QA / EZPoints / Sokoban	Rule + External	Yes (interactive env actions, search)
OPID	On-policy skill distillation (hierarchical hindsight skills → token-level dense supervision)	Single	Both	Multi	ALFWorld / WebShop / Search-QA	Rule + External	Yes (interactive env actions, search)
UCOB	Credit-aware on-policy bidirectional self-distillation (skill-conditioned vs skill-free branches)	Single	Both	Multi	ALFWorld / WebShop / Search-QA	Rule (return-based)	Yes (interactive env actions, search)
SIRI	GiGPO + self-skill mining/distillation	Single	Both	Multi	Self-internalizing intrinsic skills (ALFWorld/WebShop)	External + Custom	Yes (interactive actions)
world-knowledge	Outcome-based RL (reward-free self-evolution)	Single	Outcome	Multi	Web Agents (WebVoyager/WebWalker; +20% on Qwen3-30B & Seed-OSS-36B)	Model (intrinsic; world-knowledge gain)	Yes (web pipeline for env-specific knowledge construction)
ARISE	Hierarchical RL (options + intra-option) w/ skill evolution	Single	Both	Multi	Reasoning w/ intrinsic skill library (7 Olympiad benchmarks)	External + Custom (skill-quality)	No (skill reuse over multi-step reasoning)
Tool-R0	Self-play RL (generator+solver co-evolution)	Multi	Both	Multi	Self-evolving tool-learning from zero data	External + Custom	Yes (real tool/function calls)
MemSkill	PPO	Single	Process	Multi	QA/ALFWorld	Model (learned skills)	Yes
MemRL	RL-based (Q-value)	Single	Process	Multi	HLE/BigCodeBench/ALFWorld	Model (retrieval)	Yes
AgentEvolver	ADCA-GRPO	Single	Outcome	Multi	Social Game/Tool-use	Rule	Yes
EvolveR	GRPO (closed-loop online+offline)	Single	Outcome	Multi	Multi-hop QA (NQ/HotpotQA)	Rule	Yes (experience retrieval)
SEAgent	GRPO	Single	Outcome	Multi	Computer Use (OSWorld)	Model	Yes (Screenshot-based)
R-Zero	GRPO (Challenger + Solver co-evolution)	Multi	Outcome	Multi	Math/SuperGPQA/MMLU-Pro/BBEH	Rule (majority voting)	No
Absolute-Zero-Reasoner	TRR++ (Task-Relative REINFORCE++)	Single	Outcome	Single	Code/Math Reasoning (HumanEval/MBPP/LiveCodeBench)	Rule + learnability	Yes (Python exec)
RAGEN	PPO/GRPO (StarPO)	Single	Both	Multi	TextGame	All	Yes
WebRL	Actor-Critic RL + ORM	Single	Outcome	Multi	Web Navigation (WebArena)	Model (ORM)	Yes (Web browsing)

⛰️ Environment

Under Review/Waiting for Open Source

• Qwen-UI-Agent: Towards Next-Generation Real-World Centric Foundation GUI Agent (Tongyi-MAI — repo released, but no RL training code)
• Qwen-CUA (xlang-ai — technical report + demo only, weights hosted separately)
• UI-Mate (Tencent — inference/eval code only)
• SearchMaster (training code marked "planned for release")
• RoMeRL (training code "being organized"; eval checkpoints only)
• GroundCUA (ServiceNow — training/ marked "Coming Soon")
• Agon
• SINKFLEX-RL
• GRASP
• MAVEN: Evidence-State Rewards
• EviBack
• ChemWorld
• Test-Time Self-Evolving GUI Visual Grounding
• CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution (ACL 2026, AMAP/Alibaba — repo created but code not yet released)
• JoyAgents-R1: Joint Evolution Dynamics for Versatile Multi-LLM Agents with Reinforcement Learning
• Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learning
• Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning
• Acting Less is Reasoning More! Teaching Model to Act Efficiently
• Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning
• ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
• Understanding Tool-Integrated Reasoning
• Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning
• Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planning
• SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents
• WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents
• EnvX: Agentize Everything with Agentic AI
• UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning
• UI-Venus Technical Report: Building High-performance UI Agents with RFT
• Agent2 : An Agent-Generates-Agent Framework for Reinforcement Learning Automation
• Adversarial Reinforcement Learning for Large Language Model Agent Safety
• Learning to Refine: An Agentic RL Approach for Iterative SPARQL Query Construction
• InfoFlow: Reinforcing Search Agent Via Reward Density Optimization

Star History

---