Awesome-LLM-Research-Collections

Language: English | 中文

A Tool Recommend for read paper faster: 幻觉翻译

Contents

• Attention
  • Attention Architecture
• LLMs
  • Foundation Models
  • Inference
  • Detection
• Multimodal LLMs
  • Vision-Language
  • Multimodal Reasoning
  • VLA
• Embeddings
• SFT
  • SFT Methods
• Training
  • Data Preparation
  • Optimization
  • Distillation
• Reinforcement Learning
  • Policy Optimization
  • OPD
  • Reward Modeling
  • Video Generation RL
  • Multimodal RL
  • Reasoning RL
  • Agentic RL
  • VLA RL
• Agents Application
  • Computer Use
  • Tool Use
  • Web Agents
  • Data Agents
  • AI Research
  • Agent Skills
  • Agent Development
  • Agent Evaluation
  • Memory
• Vision
  • Object Detection
  • Semantic Correspondence
• Auto-Prompt
  • Prompt Optimization
  • Judge Prompting
• Notes
  • Paper Readings
  • Technical Reflections
• Blogs

Attention

Attention Architecture

• FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention (2026.06)
  Description: This paper proposes Lookahead Sparse Attention, which uses a separately trained neural memory indexer to predict future context needs and retain only query-critical KV chunks on GPU. FlashMemory reduces the physical KV cache footprint to 13.5% of full-context attention on average while preserving or slightly improving long-context accuracy.
  

• MiniMax Sparse Attention (2026.06)
  Description: This paper introduces MiniMax Sparse Attention, a blockwise sparse attention mechanism built on GQA that uses a lightweight Index Branch to select group-specific Top-k KV blocks before exact sparse attention. Co-designed GPU kernels turn the sparsity into large practical speedups at million-token context while maintaining performance close to dense GQA.
  

• GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding (2026.05)
  Description: This paper proposes Group-Query Latent Attention, a minimal MLA modification that exposes both MQA-absorb and GQA decoding paths from the same trained weights. The runtime can select the path that matches target hardware without retraining or custom kernels, enabling H100-style compressed decoding, H20-oriented GQA plus MTP, and up to 8-way zero-redundancy tensor parallelism.
  

• Attention Residuals (2026.03)
  Description: This work replaces fixed residual accumulation with attention over previous layer outputs, enabling input-dependent depth-wise aggregation and reducing PreNorm-induced representation dilution. It also introduces Block AttnRes for scalable training with lower memory and communication overhead.
  

• Kimi Linear: An Expressive, Efficient Attention Architecture (2025.10)
  Description: This paper introduces Kimi Linear, a hybrid architecture whose Kimi Delta Attention module combines fine-grained gating with an efficient chunkwise algorithm based on specialized diagonal-plus-low-rank transitions. A 48B-parameter model with 3B activated parameters outperforms full MLA under the same training recipe while reducing KV cache usage by up to 75% and delivering up to 6× decoding throughput at a 1M-token context.
  

• RoFormer: Enhanced Transformer with Rotary Position Embedding (2021.04)
  Description: This paper introduces Rotary Position Embedding (RoPE), which encodes absolute positions through rotations while making self-attention explicitly depend on relative positions. RoPE supports flexible sequence lengths, distance-aware dependency decay, and relative position encoding for linear self-attention.
  

• Fast Transformer Decoding: One Write-Head is All You Need (2019.11)
  Description: This paper introduces multi-query attention, sharing keys and values across attention heads to reduce the memory-bandwidth cost of incremental Transformer decoding. The variant speeds up decoding substantially while incurring only minor quality degradation relative to multi-head attention baselines.
  

LLMs

Foundation Models

• BDH-CQ: In-Context Learning with Recurrent Latent Reasoning (2026.08)
  Description: BDH-CQ combines in-context learning through continuously updated recurrent memory with iterative reasoning in a high-dimensional latent space, without verbalizing intermediate reasoning. On ARC-AGI-1, its 150M-parameter configuration attains 29.5% pass@2 at a computed cost of $0.0007 per task, improving the reported cost-accuracy frontier.
  

• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability (2026.08)
  Description: Qwen3.8-Flash-Next combines hybrid Gated DeltaNet and sparse attention, gated residual branches, and host-memory n-gram embeddings in a 125B-parameter MoE backbone with 6B parameters activated per token. Joint architecture and Muon optimizer ablations examine downstream quality, training and inference costs, and stability, yielding competitive pretraining performance at roughly one-ninth the predecessor's training FLOPs.
  

• Kimi K3: Open Frontier Intelligence (2026.07)
  Description: Kimi K3 is a 2.8T-parameter MoE foundation model with 104B activated parameters, native vision, and a 1-million-token context window, built on Kimi Delta Attention, Attention Residuals, and Stable LatentMoE. These architectural and training advances improve overall scaling efficiency by about 2.5× over Kimi K2 and support frontier-level long-horizon coding, agentic, reasoning, and vision performance.
  

• The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence (2026.05)
  Description: This technical report presents the MiniMax-M2 series, MoE language models with a small active-parameter footprint designed for real-world agentic deployment. It combines agent-driven verifiable data pipelines, the Forge agent-native RL system, and early self-evolution in M2.7 to improve coding, deep-search, office-task, and reasoning performance.
  

• The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook (2026.04)
  Description: This survey argues that continuous latent space is becoming a native computational substrate for language-based models, addressing the inefficiencies of explicit token-level generation such as redundancy, discretization bottlenecks, and semantic loss. It further organizes the field through mechanism and ability perspectives, and outlines key open challenges for future research.
  

• DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence (2026.04)
  Description: DeepSeek-V4 introduces 1.6T-parameter Pro and 284B-parameter Flash MoE language models, both supporting one-million-token contexts. Its hybrid Compressed Sparse Attention and Heavily Compressed Attention, Manifold-Constrained Hyper-Connections, and Muon optimizer improve long-context efficiency, with Pro using 27% of DeepSeek-V3.2's single-token inference FLOPs and 10% of its KV cache in the one-million-token setting.
  

• GLM-5: from Vibe Coding to Agentic Engineering (2026.02)
  Description: GLM-5 is a next-generation foundation model targeting long-horizon agentic engineering, with reduced training and inference cost and preserved long-context capability. It introduces asynchronous RL infrastructure and agent RL algorithms to improve post-training efficiency and real-world coding performance.
  

• Kimi K2.5: Visual Agentic Intelligence (2026.02)
  Description: This paper introduces an open-source multimodal agentic model that jointly optimizes text and vision through unified pretraining, SFT, and reinforcement learning. It also proposes Agent Swarm, a parallel orchestration framework for decomposing and executing complex tasks with coordinated agents.
  

• MiMo-V2-Flash Technical Report (2026.01)
  Description: MiMo-V2-Flash is a 309B-parameter MoE foundation model with 15B active parameters, built for fast reasoning, coding, and agentic workloads through hybrid sliding-window/global attention, 27T-token pretraining, and long-context extension to 256k. It introduces Multi-Teacher On-Policy Distillation for scalable post-training and repurposes multi-token prediction as a draft model for speculative decoding speedups.
  

• Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models (2026.01)
  Description: This paper introduces conditional memory as a sparsity axis complementary to MoE, instantiated by Engram for constant-time lookup of static knowledge. A scaling law guides the allocation between neural computation and memory, enabling Engram models to improve knowledge, reasoning, code, math, and long-context retrieval at matched parameters and FLOPs.
  

• DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models (2025.12)
  Description: DeepSeek-V3.2 is an open large language model that combines efficient long-context computation with strong reasoning and agent performance. Its key ingredients include DeepSeek Sparse Attention, scalable RL post-training, and a large-scale agentic task synthesis pipeline for improving tool-use generalization and instruction-following robustness.
  

• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models (2025.08)
  Description: GLM-4.5 introduces an open-source MoE foundation model with hybrid reasoning modes (thinking/direct response) to better support agentic, reasoning, and coding tasks. It combines large-scale pretraining and RL-based post-training, and releases both full and compact variants with strong benchmark performance.
  

• Kimi K2: Open Agentic Intelligence (2025.07)
  Description: Kimi K2 presents a trillion-parameter MoE language model focused on strong agentic, reasoning, and coding capabilities with stable large-scale training. The work introduces MuonClip with QK-clip to improve optimization stability and token efficiency during pretraining.
  

• Qwen3 Technical Report (2025.05)
  Description: This report presents the Qwen3 family spanning dense and MoE models across a wide parameter range, emphasizing stronger multilingual performance and efficiency. It unifies deliberative thinking and fast response modes in one framework and scales post-training to improve reasoning, coding, and agentic behavior.
  

• MiniMax-01: Scaling Foundation Models with Lightning Attention (2025.01)
  Description: MiniMax-01 introduces a long-context model family built around Lightning Attention and MoE to improve scaling efficiency and practical throughput. It combines optimized parallelization and communication-computation overlap to train large models with stronger long-context performance.
  

• DeepSeek-V3 Technical Report (2024.12)
  Description: DeepSeek-V3 is a 671B-parameter MoE language model with 37B activated parameters per token, built for efficient inference and cost-effective large-scale training. It extends MLA and DeepSeekMoE with auxiliary-loss-free load balancing and a multi-token prediction objective, achieving strong open-model performance with stable 14.8T-token pretraining and SFT/RL post-training.
  

• Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement (2024.09)
  Description: This paper presents Qwen2.5-Math, a family of math-specialized language models that applies self-improvement throughout pre-training, post-training, and inference. The approach strengthens mathematical reasoning and tool-augmented problem solving across multiple model sizes.
  

• Qwen2 Technical Report (2024.07)
  Description: This report introduces the Qwen2 series of dense and mixture-of-experts language models, covering base and instruction-tuned variants across a broad parameter range. It emphasizes stronger multilingual, coding, math, and reasoning performance while remaining competitive with proprietary systems.
  

• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model (2024.05)
  Description: DeepSeek-V2 is a 236B-parameter MoE language model with 21B activated parameters per token and 128K context length, designed for economical training and efficient inference. It combines Multi-head Latent Attention for KV-cache compression with DeepSeekMoE sparse computation, reducing training cost and KV cache while improving throughput and open-model performance.
  

Inference

• LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers (2026.08)
  Description: LLMRouter formulates LLM routing as a sequential decision process spanning single-turn, multi-turn, and personalized settings, and provides modular infrastructure with an automated pipeline for joint quality-cost evaluation. Its xRouteBench benchmark covers five routing scenarios, while experiments across more than 16 representative routers show that learned routers achieve a 14.6% relative improvement over the strongest fixed-model baseline.
  

• AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses (2026.08)
  Description: This paper introduces strong-to-weak scaffolding, in which a stronger builder model constructs an inference-time harness that improves a weaker target model without updating its parameters. Across four Theory-of-Mind benchmarks, the harness nearly doubles average performance by offloading unstable reasoning to deterministic code, task-specific routing, and strict answer-format enforcement.
  

• Large Language Models Explore by Latent Distilling (2026.04)
  Description: This paper proposes Exploratory Sampling (ESamp), a decoding method that addresses the shallow lexical variation of standard stochastic sampling by encouraging semantic exploration. It trains a lightweight Distiller at test time to predict deep-layer representations from shallow ones, then uses prediction error as a novelty signal to reweight candidate tokens and improve Pass@k efficiency.
  

• Caterpillar of Thoughts: The Optimal Test-Time Algorithm for Large Language Models (2026.03)
  Description: This paper presents a theoretical framework for optimal test-time computation in LLMs, proving that the optimal algorithm always generates a caterpillar tree structure, and introduces CaT which achieves better success rate than Tree-of-Thoughts with fewer token generations.
  

• FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning (2025.06)
  Description: FinChain introduces a contamination-free benchmark for machine-verifiable multi-step financial reasoning, covering 58 topics across 12 domains through parameterized templates with executable Python traces. Its ChainEval metric jointly evaluates final-answer correctness and step-level reasoning consistency, revealing persistent weaknesses across 26 leading LLMs.
  

Detection

• Base Models Look Human To AI Detectors (2026.05)
  Description: This paper finds that commercial AI-text detectors often label base-model outputs as more human than outputs from instruction-tuned counterparts, suggesting they track tuning artifacts and local context rather than invariant machine-text signals. It proposes Humanization by Iterative Paraphrasing (HIP), a detector-agnostic fine-tuning and iterative paraphrasing pipeline that improves semantic preservation while evading detectors.
  

Multimodal LLMs

Vision-Language

• Intern-S2-Preview: Scientific Agentic Foundation Model (2026.08)
  Description: Intern-S2-Preview is a series of scientific agentic foundation models that combines multimodal scientific pre-training with unified supervised fine-tuning, multi-task and agentic reinforcement learning, and on-policy distillation for scientific understanding, reasoning, generation, and long-horizon tasks. Its 397B model adds time-series forecasting and a separate Memory Decoder path for rapid scientific specialization, achieving competitive or leading results across scientific, multimodal, agentic, and general-purpose benchmarks.
  

• VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding (2026.07)
  Description: VideoChat3 is a fully open 4B video-centric MLLM that combines I3D-ViT with adaptive frame resolution for efficient spatiotemporal and streaming perception. Its scalable data synthesis pipeline curates datasets for general, long-form, and streaming video, improving cross-domain generalization while reducing computation.
  

• Lance: Unified Multimodal Modeling by Multi-Task Synergy (2026.05)
  Description: Lance introduces a lightweight native unified multimodal model for image and video understanding, generation, and editing without relying mainly on capacity scaling. It combines shared interleaved context modeling, decoupled capability pathways, dual-stream MoE, modality-aware rotary positional encoding, and staged multi-task training to improve both generation and understanding.
  

• Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding (2026.04)
  Description: This paper introduces Video-MME-v2, an improved video understanding benchmark addressing the saturation issue in existing benchmarks where inflated leaderboard scores fail to reflect real-world model capabilities.
  

• V-Reflection: Transforming MLLMs from Passive Observers to Active Interrogators (2026.04)
  Description: V-Reflection converts MLLMs from passive visual consumers to active interrogators through a think-then-look reflection mechanism that grounds each reasoning step in visual evidence. A two-stage distillation design improves fine-grained perception while keeping inference fully autoregressive and efficient.
  

• ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model (2026.03)
  Description: This paper proposes a VLM-guided JEPA-style latent world modeling framework that combines dense-frame dynamics prediction with long-horizon semantic guidance through a dual-temporal design. It further introduces a hierarchical pyramid representation extraction module to transfer multi-layer VLM reasoning features into latent forecasting for more robust hand-manipulation trajectory prediction.
  

• Qwen3-VL Technical Report (2025.11)
  Description: We introduce Qwen3-VL, the most capable vision-language model in the Qwen series to date, achieving superior performance across a broad range of multimodal benchmarks. It natively supports interleaved contexts of up to 256K tokens, seamlessly integrating text, images, and video.
  

• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency (2025.08)
  Description: This paper introduces InternVL 3.5, a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency, featuring the Cascade Reinforcement Learning framework.
  

• Qwen2.5-VL Technical Report (2025.02)
  Description: This technical report introduces Qwen2.5-VL, a flagship vision-language model with stronger visual recognition, precise localization, robust document parsing, and long-video understanding. It also improves agentic interaction with visual environments through better grounding and structured perception capabilities.
  

• InternVL 2.5: Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (2024.12)
  Description: This paper introduces InternVL 2.5, an advanced multimodal LLM series that was the first open-source MLLMs to surpass 70% on the MMMU benchmark, achieving a 3.7-point improvement through Chain-of-Thought reasoning.
  

• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution (2024.09)
  Description: This paper introduces Qwen2-VL, a vision-language model series that uses Naive Dynamic Resolution to process images at arbitrary resolutions and M-RoPE to fuse text, image, and video positional information. Scaling the model to 2B, 8B, and 72B parameters with larger multimodal data yields competitive image, video, multilingual OCR, document understanding, and agentic visual interaction performance.
  

• LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models (2024.07)
  Description: This paper introduces LLaVA-NeXT-Interleave, which simultaneously tackles Multi-image, Multi-frame (video), Multi-view (3D), and Multi-patch scenarios in large multimodal models, extending visual instruction tuning to multi-modal scenarios.
  

• How Good is my Video LMM? Complex Video Reasoning and Robustness Evaluation Suite for Video-LMMs (2024.05)
  Description: This paper introduces CVRR-ES, a benchmark that comprehensively assesses Video-LMMs across 11 diverse real-world video dimensions, evaluating 9 recent models and finding that most open-source Video-LMMs struggle with robustness and reasoning on complex videos.
  

• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond (2023.08)
  Description: This paper introduces Qwen-VL, a vision-language model series built on Qwen-LM with a visual receptor, multimodal interface, three-stage training pipeline, and multilingual multimodal corpus. By aligning image-caption-box tuples, Qwen-VL supports visual understanding, grounding, and text reading while achieving strong results across visual-centric benchmarks.
  

• LLaVA: Visual Instruction Tuning (2023.04)
  Description: This paper presents LLaVA, a large multimodal model trained end-to-end on machine-generated instruction tuning data, showing impressive multimodal chat abilities and achieving state-of-the-art results on Science QA.
  

Multimodal Reasoning

• Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking (2026.09)
  Description: This paper characterizes rare-entity failures in multilingual multimodal entity linking using 15 metrics spanning popularity, documentation, knowledge-graph structure, and cross-lingual coverage. It introduces a training-free vision-language framework that iteratively retrieves and reasons over Wikipedia evidence, improving accuracy overall and especially on rare-entity slices, and releases the MERLIN-Rare benchmark.
  

• Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey (2025.03)
  Description: This survey addresses the lack of an up-to-date review of multimodal Chain-of-Thought reasoning in MLLMs across image, video, speech, audio, 3D, and structured data. It introduces foundational definitions, a comprehensive taxonomy, methodological analysis across applications, and open challenges for future multimodal reasoning research.
  

VLA

• Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving (2026.08)
  Description: Qwen-Drive-1.0 integrates 3D perception, visual question answering, and motion planning around an unchanged pretrained vision-language model, using an external BEV perception head and a Planning Expert for future ego trajectories. A staged recipe combines driving supervision with general vision-language data, preserving broad visual understanding while achieving strong 3D scene and closed-loop planning performance.
  

• LA4VLA: Learning to Act without Seeing via Language-Action Pretraining (2026.06)
  Description: This paper proposes LA4VLA, a language-action pretraining framework that teaches VLA policies language-conditioned action priors without visual observations by decomposing demonstrations into atomic action segments with low-level descriptions. It builds the LA-33K dataset and shows that combining language-action and VLA supervision improves manipulation success in both simulation and real-world tasks.
  

• Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation (2026.04)
  Description: OneVL addresses real-time trajectory planning in VLA-based autonomous driving by compressing Chain-of-Thought reasoning into compact latent tokens supervised by both language reconstruction and future-frame prediction. Its three-stage training pipeline yields latent reasoning that surpasses explicit CoT while keeping answer-only inference latency.
  

• CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videos (2026.01)
  Description: CLAP learns an executable latent action vocabulary from robot trajectories and contrastively aligns human video transitions with it, enabling VLA pretraining on abundant unlabeled human videos. It combines an autoregressive VLA with a rectified-flow action head and knowledge-matching regularization for efficient control and target-domain adaptation.
  

Embeddings

• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025.06)
  Description: This paper introduces Qwen3 Embedding, a family of text embedding and reranking models built on Qwen3 foundation models with a multi-stage training pipeline, model merging, and LLM-synthesized multilingual data. The series covers 0.6B, 4B, and 8B sizes and achieves state-of-the-art results across multilingual embedding, retrieval, reranking, code retrieval, and cross-lingual benchmarks.
  

• CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval (2025.05)
  Description: This paper introduces CoQuIR, the first large-scale multilingual benchmark for quality-aware code retrieval, with 42,725 queries and 134,907 snippets annotated for correctness, efficiency, security, and maintainability. Its two quality-centric metrics and evaluation of 23 retrievers expose major quality-awareness gaps, while contrastive training improves quality-aware retrieval without sacrificing semantic relevance.
  

SFT

SFT Methods

• Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning (2026.05)
  Description: This paper systematically studies difficulty-based data selection for supervised fine-tuning and shows that no single difficulty level is universally optimal. It explains the data-size-dependent optimum through a tradeoff between in-distribution generalization and extrapolation, with the best difficulty shifting toward harder examples as the data budget grows.
  

• Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability (2026.04)
  Description: This paper challenges the common claim that supervised fine-tuning (SFT) only memorizes while RL generalizes, finding cross-domain generalization from reasoning SFT with long chain-of-thought supervision depends jointly on optimization dynamics, training data, and base model capability.
  

• ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection (2026.01)
  Description: This paper presents ProFit, a supervised fine-tuning method that mitigates single-reference overfitting by using token probability as a proxy for semantic importance and masking low-probability tokens. The approach focuses learning on core logical content and improves reasoning and math performance over standard SFT baselines.
  

• BEFT: Bias-Efficient Fine-Tuning of Language Models in Low-Data Regimes (2025.09)
  Description: This paper studies which attention-projection bias terms to fine-tune in low-data settings and finds that directly tuning the value bias generally outperforms tuning query or key biases. The result holds across encoder-only and decoder-only language models up to 6.7B parameters, including bias-free models augmented with a value bias.
  

Training

Data Preparation

• BigBang: Pursuing Open-Ended Intelligence through Self-Evolving Synthesis of Verifiable Frontier Tasks (2026.08)
  Description: BigBang is a general-purpose 35B-A3B model post-trained on verifiable frontier tasks generated by an adversarial, self-evolving generator-critic framework. Calibrated with held-out research tasks, the pipeline iteratively improves task difficulty and evaluation quality, yielding broad gains in scientific research, reasoning, coding, and tool use.
  

• DataPrep-Bench: Benchmarking LLMs as Training Data Preparators (2026.05)
  Description: DataPrep-Bench is the first unified, downstream-grounded benchmark for evaluating how LLMs, agents, and data-centric workflows construct supervised training data and predict candidate datasets' downstream utility. It also introduces a skill-guided construction agent and the Distributional Alignment Score, which outperforms existing quality, diversity, and heuristic evaluators across most tested domains.
  

Optimization

• PowLU: An Activation Function for Stable Pre-Training of LLMs (2026.05)
  Description: This paper identifies SwiGLU's near-quadratic amplification on large positive inputs as a source of outliers and numerical instability in low-precision large-scale LLM pre-training. It proposes Power Linear Unit (PowLU), a rational-power activation that preserves adaptive nonlinearity while stabilizing spike regions, with scaling-law and Ling-model experiments showing competitive performance and improved training scalability.
  

• Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning (2026.03)
  Description: This paper shows that Warmup-Stable-Only pre-training, which keeps the learning rate constant after warmup, consistently improves downstream SFT performance over decay-based schedules across 1B and 8B models. Its loss-landscape analysis attributes the gain to flatter minima that preserve model adaptability.
  

Distillation

• Rethinking On-Policy Distillation of Large Language Models II: One Training Example (2026.09)
  Description: This paper studies on-policy distillation at the data-minimal limit and finds that training on a single query recovers most of the gain from full-data OPD across domains and model families. It explains this through state coverage and student-teacher alignment, showing that 16 semantically distinct queries can match full-data training while OPD is data-overfed but algorithm-starved.
  

• Simple-OPD: Demystifying Warm-up for On-policy Distillation (2026.08)
  Description: This paper shows that effective warm-up for on-policy distillation depends on teacher-compatible chain-of-thought supervision rather than answer correctness alone. It proposes Simple-OPD, a plug-and-play initialization recipe that uses near-saturation LoRA warm-up on teacher-generated reasoning traces to balance in-domain adaptation and out-of-distribution generalization.
  

• Escaping the KL Agreement Trap in On-Policy Distillation (2026.06)
  Description: This paper identifies a low-KL agreement trap in on-policy distillation, where a teacher locally agrees with a student's corrupted rollout and provides little corrective supervision. It proposes KAT, an adaptive online termination rule that filters these uninformative suffixes, improving mathematical reasoning accuracy while reducing average rollout length by 59.73%.
  

• On-Policy Distillation with Best-of-N Teacher Rollout Selection (2026.05)
  Description: This paper proposes BRTS, which samples multiple teacher trajectories and selects supervision by prioritizing correctness and then alignment with the student's current behavior. An auxiliary teacher-context branch and ground-truth-conditioned recovery improve OPD on challenging mathematical reasoning benchmarks.
  

• Trust Region On-Policy Distillation (2026.05)
  Description: This paper proposes TrOPD to stabilize on-policy distillation under large teacher-student distribution mismatch by restricting reverse-KL supervision to reliable regions. It handles outliers through clipping, masking, or forward-KL estimation and uses teacher-prefix off-policy guidance to steer exploration back toward trustworthy regions.
  

• Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes (2026.03)
  Description: This paper identifies three failure modes in sampled-token OPD: imbalanced token supervision, unreliable teacher guidance on student-generated prefixes, and tokenizer or special-token mismatch. It combines teacher top-K local support matching with top-p rollout sampling and special-token masking, improving stability and outperforming standard OPD by 19.8%.
  

• Knowledge Distillation of Large Language Models (2023.06)
  Description: This paper studies white-box knowledge distillation for generative LLMs and proposes MiniLLM, replacing the standard forward KLD objective with reverse KLD to avoid overestimating low-probability teacher regions. The method derives an effective optimization procedure and improves instruction-following quality, calibration, exposure bias, and long-text generation across model families from 120M to 13B parameters.
  

Reinforcement Learning

Policy Optimization

• Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training (2026.07)
  Description: This paper introduces layer contribution to measure how much of full-parameter RL improvement can be recovered by training each transformer layer independently. Across multiple models, RL algorithms, and tasks, it finds that gains consistently concentrate in a few middle layers and that single-layer training can match or surpass full-parameter training.
  

• Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignment (2026.07)
  Description: This paper introduces Selective Importance Sampling (SIS), a plug-in correction for off-policy LLM reinforcement learning that uses token-level rejection tests to treat accepted tokens as on-policy while retaining standard importance sampling for rejected tokens. SIS reduces the gap between token- and sequence-level gradient estimators with negligible overhead, improving performance and robustness across dense and MoE models.
  

• SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning (2026.07)
  Description: This paper introduces SERPO, a test-time reinforcement learning framework for open-ended generation that co-evolves response evidence, query-specific rubrics, and policy parameters without labeled feedback, external reward models, or stronger judges. Probabilistic criterion scoring turns verdict-token likelihoods into rewards, enabling the actor and its self-generated evaluation criteria to improve in a closed loop.
  

• Rethinking the Divergence Regularization in LLM RL (2026.06)
  Description: This paper proposes Divergence Regularized Policy Optimization (DRPO), replacing DPPO's hard divergence mask with a smooth advantage-weighted quadratic regularizer that preserves its trust-region geometry. DRPO provides bounded continuous gradient weights and corrective signals beyond the trust-region boundary, improving LLM RL training stability and efficiency.
  

• Constraint-Infused Policy Optimization: Principles and Practices for Harnessing Advanced LLM Reasoning (2026.05)
  Description: This paper formulates LLM reinforcement learning as constrained policy optimization, unifying existing algorithms through different constraint choices and exposing the roles of clipping, KL regularization, and trust regions. It derives Constraint-Infused Policy Optimization (CIPO), which improves reasoning performance and training stability across diverse tasks and model families.
  

• BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning (2026.03)
  Description: BandPO projects trust regions defined by general divergence measures into dynamic, probability-aware clipping intervals, expanding the update margin for low-probability high-advantage actions. This principled replacement for fixed PPO-style clipping improves exploration and robustly mitigates entropy collapse across LLM reinforcement learning settings.
  

• Soft Adaptive Policy Optimization (2025.11)
  Description: This paper introduces Soft Adaptive Policy Optimization (SAPO), replacing hard clipping in group-based LLM reinforcement learning with a temperature-controlled soft gate that continuously attenuates off-policy token updates. Its sequence-coherent, token-adaptive objective improves training stability, sample efficiency, and reasoning performance across text and Qwen3-VL models.
  

• Group Sequence Policy Optimization (2025.07)
  Description: This paper introduces GSPO, a reinforcement learning algorithm for LLMs that replaces token-level importance ratios with sequence-level likelihood ratios and performs sequence-level clipping, rewarding, and optimization. GSPO improves training efficiency and performance over GRPO, stabilizes MoE RL training, and helps simplify large-scale RL infrastructure for Qwen3 models.
  

• DAPO: An Open-Source LLM Reinforcement Learning System at Scale (2025.03)
  Description: This paper introduces Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO), an open large-scale reinforcement learning system for eliciting LLM reasoning. It releases the training recipe, code, dataset, and model weights, reaching 50 points on AIME 2024 with Qwen2.5-32B and improving reproducibility for large-scale LLM RL.
  

• Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer (2024.05)
  Description: This paper identifies RLHF overoptimization as a consequence of distribution shift and uncertainty in learned preferences, then introduces Regularized Preference Optimization (RPO), which combines a preference optimization objective with an SFT loss that acts as an adversarial regularizer. RPO provides finite-sample guarantees and empirically improves alignment over DPO while reducing drift toward undesired responses.
  

• KTO: Model Alignment as Prospect Theoretic Optimization (2024.02)
  Description: This paper frames successful LLM alignment losses as human-aware losses that encode biases from prospect theory, then introduces KTO to optimize generation utility directly from binary desirable/undesirable feedback. KTO matches or exceeds preference-pair methods from 1B to 30B scales, highlighting how the best alignment loss depends on the setting's inductive biases.
  

• Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023.05)
  Description: This paper introduces Direct Preference Optimization (DPO), which reparameterizes the RLHF reward model so the optimal policy can be learned directly from preference data with a simple classification loss. DPO removes separate reward-model fitting and online reinforcement learning while matching or improving PPO-based RLHF with simpler, more stable training.
  

• Proximal Policy Optimization Algorithms (2017.07)
  Description: This paper introduces Proximal Policy Optimization (PPO), a family of policy-gradient methods that alternates environment sampling with multiple minibatch epochs on a surrogate objective. PPO retains key trust-region benefits while being simpler to implement and empirically balancing sample efficiency, performance, and wall-clock time.
  

OPD

• Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement (2026.08)
  Description: This paper finds that noisy teacher supervision is not the main source of on-policy distillation gains: suppressing student-sampled low-probability tokens with a fixed negative advantage achieves comparable performance. It introduces On-Policy Self-Adaptation (OPSA), a teacher- and reward-free method that scales negative advantages by token entropy to improve reasoning and generalization.
  

• PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation (2026.06)
  Description: This paper identifies the unbounded log-ratio reward as the source of high-variance gradients and unstable training in sampled-token on-policy distillation. PowerOPD replaces it with a bounded, sign-consistent reward family derived from the Box-Cox power transformation, improving mathematical reasoning accuracy while reducing time and memory relative to full-vocabulary OPD.
  

• Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information (2026.05)
  Description: This paper analyzes why on-policy self-distillation can hurt math reasoning, showing through pointwise mutual information that privileged context overemphasizes solution-implied tokens while suppressing deliberation tokens needed for search. It proposes AntiSD, which ascends rather than descends the self-distillation divergence with an entropy gate, reaching GRPO-level accuracy in 2 to 10x fewer steps and improving final accuracy by up to 11.5 points.
  

• Draft-OPD: Adapting Speculative Draft Models from LLMs via On-Policy Distillation (2026.05)
  Description: This paper proposes Draft-OPD, which adapts speculative draft models from RL-trained LLM traces through on-policy distillation without requiring expensive online generation for the draft model. It proves an equivalence between RL training and OPD-style distillation, reuses collected RL experience, and improves speculative decoding speed by up to 2.14x while preserving task performance.
  

• OmniOPD: Logit-Free On-Policy Distillation via Speculative Verification (2026.05)
  Description: This paper introduces OmniOPD, a logit-free on-policy distillation framework that replaces brittle token-level logit matching with Monte Carlo chunk rollouts scored by semantic similarity, enabling black-box teachers. A peak-entropy scheduler focuses verification on uncertain reasoning forks, while Bayesian smoothing and a base-model KL anchor stabilize training; it outperforms standard OPD by up to 28.64% on math.
  

• Self-Distilled RLVR (2026.04)
  Description: This paper studies on-policy self-distillation for RLVR and shows that relying only on a privileged self-teacher can cause information leakage and unstable long-term training. It proposes RLSD, which uses self-distillation to estimate token-level update magnitudes while keeping RLVR's environment feedback as the reliable update direction.
  

• Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation (2026.02)
  Description: This paper shows that on-policy distillation is a special case of dense KL-constrained RL, then proposes G-OPD with a flexible reference model and reward scaling factor. Its reward extrapolation variant, ExOPD, improves over standard OPD and can let students surpass domain teachers when merging RL-trained experts.
  

• Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models (2026.01)
  Description: This paper introduces On-Policy Self-Distillation (OPSD), where a single LLM acts as teacher and student under different contexts: the teacher sees privileged verified reasoning traces while the student samples from the question-only policy. By matching per-token distributions over the student's on-policy rollouts, OPSD provides dense supervision without an external teacher and achieves stronger token efficiency than GRPO and off-policy distillation on math reasoning.
  

• Self-Distillation Enables Continual Learning (2026.01)
  Description: This paper introduces Self-Distillation Fine-Tuning (SDFT), an on-policy self-distillation method that learns from expert demonstrations by using a demonstration-conditioned model as its own teacher. It improves new-task acquisition while reducing catastrophic forgetting, enabling sequential accumulation of skills and knowledge without explicit reward functions.
  

Reward Modeling

• The Verification Horizon: No Silver Bullet for Coding Agent Rewards (2026.06)
  Description: This paper frames reliable verification as the emerging bottleneck for coding agents and evaluates reward signals by scalability, faithfulness, and robustness. Through test, rubric, user-feedback, and automated-agent verifiers, it shows that targeted designs can curb reward hacking and argues that verification must co-evolve with increasingly capable generators.
  

• Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning (2026.04)
  Description: This paper revisits value modeling for LLM reinforcement learning and argues that one-shot discriminative critics are limited by their expressiveness. It introduces Generative Actor-Critic (GenAC), whose critic reasons before estimating value and uses in-context conditioning to track the current actor, improving value approximation, ranking reliability, out-of-distribution generalization, and downstream RL performance.
  

• Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty (2026.04)
  Description: This paper introduces E-GRM, which estimates uncertainty from the convergence of parallel model generations and triggers chain-of-thought reasoning only when needed. A lightweight discriminative scorer trained with a hybrid regression-ranking objective provides fine-grained reasoning-path rewards, reducing inference cost while improving accuracy.
  

• MemReward: Graph-Based Experience Memory for LLM Reward Prediction with Limited Labels (2026.03)
  Description: This paper introduces MemReward, a graph-based experience memory framework that achieves 97.3% of Oracle performance on 3B and 96.6% on 1.5B models for reward prediction with limited labels, surpassing Oracle in out-of-domain tasks.
  

• Scaling Reward Modeling without Human Supervision (2026.03)
  Description: This paper studies unsupervised reward model scaling by learning preferences over web-corpus document prefixes and suffixes without human annotations. It reports consistent RewardBench gains across model backbones and shows downstream improvements in best-of-N selection and policy optimization.
  

• Reward Modeling from Natural Language Human Feedback (2026.01)
  Description: This paper introduces RLVR on preference data for training Generative Reward Models, demonstrating that binary classification tasks make GRMs susceptible to guessing correct outcomes without sound critiques, and proposes a method to address this limitation.
  

• A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Models (2025.10)
  Description: This survey reviews Process Reward Models for evaluating and guiding LLM reasoning at the step or trajectory level rather than only judging final answers. It organizes the full loop of process data generation, PRM construction, and PRM use in test-time scaling and reinforcement learning across math, code, multimodal reasoning, robotics, and agents.
  

Video Generation RL

• KVPO: ODE-Native GRPO for Autoregressive Video Alignment via KV Semantic Exploration (2026.05)
  Description: KVPO aligns streaming autoregressive video generators with human preferences using an ODE-native online GRPO framework. It replaces noise-based exploration with causal-semantic routing of historical KV cache entries and optimizes a velocity-field surrogate policy based on Trajectory Velocity Energy.
  

Multimodal RL

• CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning (2025.09)
  Description: This paper introduces CapRL, the first RLVR framework for open-ended image captioning, which rewards captions by whether a vision-free language model can answer image questions using only the generated description. The resulting CapRL-3B model produces more informative and diverse captions, while its generated caption data improves large vision-language model pretraining across 12 benchmarks.
  

• Visual Planning: Let's Think Only with Images (2025.05)
  Description: This paper introduces Visual Planning, a paradigm that performs step-by-step planning through sequences of images rather than text for vision-first spatial tasks. Its Visual Planning via Reinforcement Learning (VPRL) framework uses GRPO to post-train large vision models and improves planning on FrozenLake, Maze, and MiniBehavior.
  

Reasoning RL

• Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning (2026.07)
  Description: This paper scales reinforcement learning with verifiable rewards from a base model to one trillion parameters through a stable pipeline combining clipped importance sampling, training-inference ratio correction, and mixed-precision control. The resulting model improves sample efficiency and reasoning quality while spontaneously developing structured, self-verifying, and adaptive reasoning behaviors.
  

• All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models (2026.04)
  Description: This paper presents MUPO, a reinforcement learning approach that addresses diversity collapse in GRPO-trained VLMs by incentivizing divergent thinking across multiple solutions, enabling deeper yet broader reasoning patterns.
  

• VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning (2026.04)
  Description: This paper introduces VL-Calibration, a reinforcement learning framework that separates visual and reasoning confidence in large vision-language models to address confidently incorrect predictions. It estimates visual certainty from image-perturbation grounding and token entropy, then applies token-level advantage reweighting to improve calibration and visual reasoning accuracy.
  

• The Art of Efficient Reasoning: Data, Reward, and Optimization (2026.03)
  Description: This paper studies efficient reasoning in LLMs, using RL to incentivize short accurate trajectories, with findings on training stages, rewards, and generalization across models from 0.6B to 30B parameters.
  

• FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization (2026.03)
  Description: This paper presents FIPO, a reinforcement learning algorithm that overcomes reasoning bottlenecks in LLMs by addressing coarse-grained credit assignment in GRPO-style training, where outcome-based rewards fail to distinguish critical logical pivots from trivial tokens.
  

• Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesis (2026.02)
  Description: This paper introduces Agentic Proposing, a framework that uses a specialized agent with Multi-Granularity Policy Optimization (MGPO) to dynamically select and compose modular reasoning skills for synthesizing high-precision training trajectories.
  

• The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models (2025.05)
  Description: This paper studies policy entropy collapse as a bottleneck in RL for reasoning language models, showing an empirical relationship between entropy and downstream performance that makes the performance ceiling predictable. It derives entropy dynamics from the covariance between action probability and logit updates, then proposes Clip-Cov and KL-Cov to preserve exploration and improve downstream performance.
  

• Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (2025.05)
  Description: This paper introduces LatentSeek, a test-time instance-level adaptation framework that uses policy gradients and self-generated rewards to iteratively optimize an LLM's latent representations without updating model parameters. It improves reasoning across GSM8K, MATH-500, and AIME2024 while typically converging within only a few iterations.
  

• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (2025.01)
  Description: This paper shows that pure reinforcement learning can directly elicit advanced reasoning behaviors in LLMs without human-labeled reasoning traces. The proposed framework induces self-reflection, verification, and adaptive strategy use, leading to strong gains on math, coding, and STEM reasoning tasks.
  

• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2024.02)
  Description: This paper introduces DeepSeekMath 7B, combining a carefully engineered web-scale math data selection pipeline with Group Relative Policy Optimization (GRPO), a PPO variant. The approach improves mathematical reasoning while reducing PPO's memory usage, reaching strong competition-level MATH performance without external tools or voting.
  

Agentic RL

• SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment (2026.09)
  Description: SafeEvolve uses safety evidence from on-policy agent trajectories to co-evolve runtime safety prompts, hierarchical skills, and the policy. It combines bounded, auditable harness updates with harness-use SFT and harness-augmented RL using verifier-decomposed rewards, improving the balance between safety and task utility during multi-step interactions.
  

• DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents (2026.08)
  Description: This paper proposes DART-SD, a topology-aware self-distillation framework that represents multi-turn tool execution as an Interaction-State Transition Graph (ISTG), preserving the diamond-shaped alternatives induced by order-independent sub-goals. It identifies the Critical Topological Breakpoint (CTB) in failed student rollouts and applies supervision only to retrieved recovery steps, protecting valid prefixes while progressively improving tool-use behavior.
  

• ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL (2026.08)
  Description: ContextPilot equips long-horizon agents with planning, long-term memory, and soft context-offloading tools to keep working context compact. Its context-aware partial rollout and action-level credit assignment focus exploration and policy updates on consequential context-editing decisions, improving long-context QA and deep-search performance.
  

• EnvHarness: Awakening Static Worlds for Agent Learning (2026.08)
  Description: EnvHarness wraps static agent environments with composable plug-in components that reshape their behavior while preserving the original logic and verifiers. Its automated EnvRigger diagnoses policy weaknesses from trajectories and synthesizes targeted environments, providing stronger learning signals for reinforcement learning and continuous policy-environment co-evolution.
  

• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills (2026.07)
  Description: This paper introduces Skill Self-Play, a reinforcement learning framework in which a proposer, solver, and dynamic skill controller co-evolve through skill-conditioned task generation, frontier exploration, and feedback-driven skill-library updates. It combines reliable skill-specific verification with open-ended task diversity to improve tool use and reasoning across diverse LLM backbones.
  

• OpenForgeRL: Train Harness-native Agents in Any Environment (2026.07)
  Description: OpenForgeRL is an open-source framework for end-to-end reinforcement learning of agents inside the stateful, multi-process inference harnesses and environments used at deployment. A lightweight recording proxy converts harness model calls into data for standard RL stacks, while a Kubernetes orchestrator isolates and scales rollouts across tool-use and multimodal GUI environments.
  

• OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning (2026.06)
  Description: This paper proposes OPID, which extracts hierarchical episode- and step-level skills from completed on-policy trajectories and converts their effect into token-level self-distillation advantages alongside the outcome advantage. Its critical-first routing supplies dense, distribution-matched hindsight supervision, improving agent performance, sample efficiency, and robustness on embodied, web-shopping, and search-based tasks.
  

• Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning (2026.06)
  Description: This paper proposes Group-Graph Policy Optimization (G2PO), which converts sampled interaction trajectories into a global state-transition graph to reduce variance in state-value estimation for long-horizon agentic reinforcement learning. Its group-aggregation values and edge-centric advantages provide fine-grained credit assignment under sparse, delayed rewards, improving success rates by up to 22.2% over GRPO.
  

• Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning (2026.05)
  Description: Spreadsheet-RL is an RL fine-tuning framework for training specialized spreadsheet agents in a realistic Microsoft Excel environment, addressing complex multi-step workflows that prompting-based agents struggle with. It adds automated start-goal spreadsheet data collection, a multi-turn Spreadsheet Gym with sandboxed Excel tools, and a Domain-Spreadsheet benchmark to improve real-world spreadsheet automation.
  

• Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration (2026.02)
  Description: This paper introduces Actor-Refiner collaboration to address the multi-scale credit assignment problem in search-integrated reasoning RL, where sparse trajectory-level rewards fail to distinguish high-quality reasoning from fortuitous guesses, reducing redundant or misleading search behaviors.
  

• SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning (2026.02)
  Description: SkillRL distills raw agent trajectories into a hierarchical SkillBank, retrieves general and task-specific heuristics adaptively, and recursively evolves the skill library alongside the policy during reinforcement learning. This reduces token overhead while improving generalization and performance across embodied, web-shopping, and search-augmented tasks.
  

• Arena-RL: Training LLMs as Game Players with Vision-Language Action Models (2026.01)
  Description: This paper introduces Arena-RL, a reinforcement learning framework that trains LLM-driven agents to play visual games via vision-language action models, focusing on policy improvement from interactive game feedback. It demonstrates that reward-driven optimization over game trajectories can significantly improve strategic decision-making and generalization across game environments.
  

• Agentic Reinforced Policy Optimization (2025.07)
  Description: This paper introduces ARPO, an agentic RL algorithm for training multi-turn LLM agents that must balance long-horizon reasoning with stepwise tool interactions. It uses entropy-adaptive rollout sampling and advantage attribution over tool-use steps, improving performance across computational reasoning, knowledge reasoning, and deep search benchmarks with roughly half the tool budget of prior trajectory-level RL methods.
  

• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning (2025.03)
  Description: This paper introduces Search-R1, an RL framework where LLMs learn to autonomously generate search queries during step-by-step reasoning with real-time retrieval, improving their ability to acquire external knowledge and up-to-date information.
  

• Search-o1: Agentic Search-Enhanced Large Reasoning Models (2025.01)
  Description: This paper introduces Search-o1, a framework that enhances large reasoning models with an agentic retrieval-augmented generation mechanism and a Reason-in-Documents module for refining retrieved documents, addressing knowledge insufficiency in extended reasoning processes.
  

VLA RL

• SRPO: Self-Referential Policy Optimization for Vision-Language-Action Models (2025.11)
  Description: This paper proposes SRPO, a reinforcement learning framework for vision-language-action models that replaces sparse binary rewards with progress-wise rewards derived from the model's own successful trajectories. It uses latent world-model representations to measure behavioral progress robustly and achieves state-of-the-art manipulation success on LIBERO with far fewer RL steps.
  

Agents Application

Computer Use

• Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents (2026.06)
  Description: This paper proposes a failure-driven inference-time self-improvement loop for computer-use agents that turns failed trajectories into LLM-diagnosed strategy and code patches, rather than discarding them. On OSWorld, it upgrades OpenCUA-72B from 42.3% to 48.9% without additional training and only modest inference overhead.
  

Tool Use

• Is Grep All You Need? How Agent Harnesses Reshape Agentic Search (2026.05)
  Description: This paper empirically studies how retrieval strategy, agent harness design, and tool-result delivery interact in agentic search. Across LongMemEval experiments with Chronos and provider CLI harnesses, grep often outperforms vector retrieval, while overall performance remains strongly shaped by the harness and tool-calling style.
  

• Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction (2026.05)
  Description: This paper introduces Direct Corpus Interaction (DCI), a retrieval paradigm in which agents search raw corpora with general-purpose terminal tools such as grep, file reads, and lightweight scripts instead of fixed top-k retrievers. DCI requires no embedding model, vector index, or retrieval API, and substantially outperforms sparse, dense, and reranking baselines on IR benchmarks and end-to-end agentic search tasks.
  

• Thinking with Programming Vision: Towards a Unified View for Thinking with Images (2025.12)
  Description: This paper identifies brittleness in current multimodal tool-using reasoning under simple image orientation changes and corruptions, and proposes CodeVision, a code-as-tool framework that lets models invoke arbitrary image operations through generated code. It combines SFT and RL with dense process rewards to improve multi-tool reasoning, execution efficiency, and error recovery on thinking-with-images tasks.
  

Web Agents

• Tongyi DeepResearch Technical Report (2025.10)
  Description: Tongyi DeepResearch is a 30.5B-parameter agentic language model with 3.3B parameters activated per token, built for long-horizon information-seeking tasks through agentic mid-training and post-training. A fully automatic data synthesis pipeline and stage-specific environments enable scalable, stable interactions across training stages.
  

Data Agents

• Data Agents: Levels, State of the Art, and Open Problems (2026.02)
  Description: Data agents use LLMs and tools to automate data management, preparation, and analysis, but inconsistent definitions obscure capability and accountability boundaries. This tutorial introduces an L0-L5 autonomy taxonomy and lifecycle-driven survey, mapping current systems and outlining a roadmap toward proactive and generative data agents.
  

• Clio: Privacy-Preserving Insights into Real-World AI Use (2024.12)
  Description: Clio is a privacy-preserving platform that uses AI assistants to extract, cluster, and summarize aggregate patterns from millions of conversations without requiring human reviewers to inspect raw data. Its evaluations and deployment on one million Claude.ai conversations show how large-scale usage analysis can reveal real-world applications and emerging safety risks while protecting user privacy.
  

AI Research

• Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills (2026.09)
  Description: This paper identifies operational knowledge as the missing layer in autonomous ML research agents and presents DisCo, a skill-powered agent that distills repository knowledge into compact, verified, reusable skills. Its AREX-Skill Library contains over 5,000 skills distilled from 1,000 repositories, and the skill-equipped agent substantially improves performance across four autonomous research benchmarks under a fixed backbone, harness, and execution budget.
  

• Agentic Artifact Creation: Systems, Evaluation, Principles, and Opportunities (2026.08)
  Description: This survey defines agentic artifact creation as stateful construction in which an AI system materially builds or revises a deliverable, with intermediate observations redirecting later work. It reviews 230 systems and 29 benchmarks across six artifact families, analyzes construction and evaluation challenges, and proposes principles for explicit responsibility, targeted repair, and post-change revalidation.
  

• AutoResearch: Insight In, Hallucination Out (2026.08)
  Description: AutoResearch is a two-stage multi-agent system that links research-idea generation with evidence-grounded execution, using cross-model review, experiment decomposition, and independent audits. Across cross-modal retrieval, systems optimization, and benchmark-driven machine learning, it turns candidate ideas into measurable progress while detecting and correcting unreliable experimental results.
  

• Spark-to-Paper: End-to-End Research Paper Generation as a Composable Skill (2026.08)
  Description: Spark-to-Paper implements end-to-end research paper generation as thirteen composable skills inside an existing coding assistant, covering literature retrieval, experiment planning and execution, evidence-guided revision, and editable figure production. It combines deterministic integrity checks with self-critique and bounded recovery from self-refutation loops to keep long-horizon research workflows grounded in measured evidence.
  

• AI for Auto-Research: Roadmap & User Guide (2026.05)
  Description: This survey analyzes AI-assisted research across creation, writing, validation, and dissemination, showing where automation is reliable and where autonomy still fails on novelty, experiments, and scientific judgment. It provides a lifecycle taxonomy, benchmark suite, tool inventory, design principles, and practitioner playbook for human-governed AI research workflows.
  

• Crafter: A Multi-Agent Harness for Editable Scientific Figure Generation from Diverse Inputs (2026.05)
  Description: This paper proposes Crafter, a multi-agent harness for generating publication-style scientific figures across multiple figure types and input conditions, and CraftEditor for converting raster outputs into editable SVGs. It also introduces CraftBench, a human-annotated benchmark for scientific figure generation, and shows gains over standalone generators and agentic baselines.
  

• AIRA_2: Overcoming Bottlenecks in AI Research Agents (2026.03)
  Description: This paper introduces AIRA_2, an AI research agent architecture that addresses limited experiment throughput, noisy validation-based selection, and static single-turn operators. It combines asynchronous multi-GPU workers, Hidden Consistent Evaluation, and interactive ReAct agents to improve long-horizon research task performance.
  

Agent Skills

• SkillEvo: Self-Renewing Evolution Gradients from Multi-Turn Interaction Feedback (2026.08)
  Description: SkillEvo evolves agent skills using multi-turn user simulation to continuously generate feedback that reveals layered interaction failures. An independent governance layer repairs factual degradation and structural bloat, outperforming self-reflection and single-turn-QA-driven evolution across production cloud-service skills.
  

• SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution (2026.05)
  Description: SkillsVote addresses noisy and hard-to-govern agent trajectories by treating Agent Skills as reusable experience artifacts with collection, recommendation, attribution, and evolution controls. It profiles large-scale open-source skill corpora, recommends structured skill context before execution, and admits only evidence-gated successful discoveries to improve frozen agents without model updates.
  

• From Context to Skills: Can Language Models Learn from Context Skillfully? (2026.04)
  Description: Ctx2Skill addresses context learning for long, dense contexts where manual skill annotation is costly and automated skill construction lacks external feedback. It uses a multi-agent self-play loop with Cross-time Replay to autonomously discover, refine, and select reusable natural-language skills that improve solving rates across language models.
  

• SkillReducer: Optimizing LLM Agent Skills for Token Efficiency (2026.03)
  Description: This paper presents SkillReducer, a two-stage optimization framework that compresses LLM agent skills (pre-packaged instruction sets) by 48% for descriptions and 39% for body while improving functional quality by 2.8%, reducing token costs and attention dilution in agent contexts.
  

Agent Development

• LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks (2026.08)
  Description: LongHorizon-Harness reframes long-horizon execution as explicit task-state management, updating external state only with facts independently verified from the environment. Its Manage-Execute-Audit loop separates planning, fresh-context execution, and read-only verification, improving reliability across computer-use and terminal benchmarks.
  

• OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents (2026.08)
  Description: OneDayAgent turns open-ended long-horizon requests into a managed execution process that combines bounded subtask decomposition, execution memory under context pressure, and global verification with targeted repair across heterogeneous tools. On 104 AgentIF-OneDay tasks, it scores 0.821 with GLM-5.2 and runs across five backend LLMs from three model families without backend-specific tuning.
  

• Self-Evolving Coding Agents (2026.08)
  Description: This survey defines self-evolving coding agents and organizes the field through an object-centered taxonomy of evolving frameworks, memory, skills, tools, models, and collaboration structures, complemented by when evolution occurs and which software-specific evidence drives it. It also identifies executable feedback, repository context, and coding trajectories as distinctive foundations for evolution while outlining challenges in reliability, safety, cost, maintainability, and generalization.
  

• Prime Agent: A Self-Improving RLM Harness (2026.08)
  Description: Prime Agent is an open-source harness for long-horizon evaluation and coding workflows that combines a persistent IPython REPL, durable histories and reusable agent assets, and recursively coordinated subagents. It standardizes execution, recovery, verification, and resource accounting while letting models construct their own strategies, substantially improving performance across reasoning, coding, and autonomous-task benchmarks.
  

• Apodex 1.1: Scaling Agentic Intelligence for Complex Work (2026.08)
  Description: Apodex 1.1 develops sustained, verifiable progress on complex real-world tasks through Environment Scaling and Agentic Coordination Scaling, combining executable file, search, and code environments with stateful multi-agent decomposition, asynchronous coordination, and replanning. Its shared execution harness and AgentOS turn trajectories and coordination traces into reliable behavior, while the 35B-parameter Apodex 1.1 Mini provides a locally deployable form of this working capability.
  

• Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable (2026.07)
  Description: This paper introduces Harness Handbook, a behavior-centric representation that combines static analysis with LLM-assisted structuring to map distributed agent-harness behaviors to their source code. Its Behavior-Guided Progressive Disclosure method improves behavior localization and edit planning while reducing planner token usage.
  

• LLM-as-a-Verifier: A General-Purpose Verification Framework (2026.07)
  Description: This paper introduces LLM-as-a-Verifier, a training-free framework that derives continuous verification scores from scoring-token probability distributions and scales verification through finer score granularity, repeated evaluation, and criteria decomposition. It provides accurate, fine-grained feedback for selecting and tracking agent solutions across coding, robotics, and medicine.
  

• Towards Long-Horizon Agents: A Survey (2026.07)
  Description: This survey formalizes long-horizon agency as a harness-coupled decision process and presents a unified taxonomy spanning externalized harness engineering and internalized model optimization. It organizes the field's evolution, applications, benchmarks, and open problems through six connected perspectives.
  

• Nurture-First Agent Development: Building Domain-Expert AI Agents Through Conversational Knowledge Crystallization (2026.03)
  Description: This paper proposes Nurture-First Development, a paradigm for growing domain-expert agents through structured conversational interaction rather than fixed code-first or prompt-first construction. It formalizes a Knowledge Crystallization Cycle, Three-Layer Cognitive Architecture, Dual-Workspace Pattern, and Spiral Development Model for continuously converting tacit practitioner knowledge into reusable agent assets.
  

• Controlled Self-Evolution for Algorithmic Code Optimization (2026.01)
  Description: This paper proposes EvoControl, a controlled self-evolution framework for algorithmic code optimization that balances correctness with exploration across generate-verify-refine cycles. It uses staged self-evolution, genetic-style population search, and evolutionary memory to improve code quality on challenging algorithmic benchmarks.
  

• Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents (2025.05)
  Description: Darwin Gödel Machine (DGM) is a self-improving coding-agent system that repeatedly rewrites its own code, evaluates each change on coding benchmarks, and explores an expanding archive of agent variants. It raises performance from 20.0% to 50.0% on SWE-bench and from 14.2% to 30.7% on Polyglot, outperforming ablations without self-improvement or open-ended exploration.
  

Agent Evaluation

• SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring (2026.08)
  Description: SWE-Bench ProMax is an expert-curated benchmark of 170 large-scale code-refactoring tasks drawn from real commits across seven programming languages, with rewritten specifications and manually reviewed tests. Its cross-file tasks average 11.4 modified files and 261.6 changed lines, while the best evaluated frontier model resolves only 41.2%.
  

• HarnessEval-W: Agentifying the Evaluation of Visual Worlds (2026.08)
  Description: HarnessEval-W is an agentified pipeline for evaluating world-model rollouts: a parent agent decomposes each case into measurable subproblems, delegates them to tool-equipped specialists, validates their evidence, and produces an auditable evidence tree instead of only a scalar score. Across 330 cases and 18 world models, its judgments closely align with human preferences while providing fine-grained diagnoses.
  

• LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering (2026.08)
  Description: LoopArena benchmarks how well a Controller model guides a separate, fixed coding-agent Worker through long-running software tasks using structured evidence and loop contracts. Its three evaluation settings range from next-step contract selection to full-task control; the best observed full-task strict success rate is 24.69%, while the lower-cost condensed setting reduces estimated inference cost by 64.4% on average.
  

• AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks? (2026.06)
  Description: AutoLab introduces a benchmark for ultra-long-horizon closed-loop optimization across system optimization, puzzles, model development, and CUDA kernels, with 36 expert-curated tasks that require agents to iteratively edit, run, measure, and improve real artifacts under strict budgets. Evaluations of 17 frontier models show that persistence and empirical feedback, rather than initial attempt quality alone, dominate success, motivating more time-aware and iterative autonomous-agent research.
  

• From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape (2026.06)
  Description: This survey presents rubrics as explicit, structured criteria that turn holistic quality judgments into verifiable and actionable standards for LLM evaluation and training. It organizes rubric construction, optimization, reliability, and benchmarks, tracing their roles from evaluation through dense process-level feedback to self-improvement.
  

Memory

• MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use (2026.08)
  Description: MemTrapBench evaluates memory-induced cognitive traps in which faithful, relevant retrieved memories distort LLM reasoning or beliefs, covering Reasoning Fixation and Belief Distortion. Across memory frameworks, it finds performance below a no-memory baseline and introduces AdaptiveMem, an inference-time prompting method that mitigates these failures while preserving standard memory-benchmark performance.
  

• Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory (2026.08)
  Description: Agent Memory Distillation is a training-free framework that transfers successful teacher-agent experience to 4B-8B student agents through complementary Workflow, Subtask, and Function memories, using proactive injection for planning and reactive retrieval for tool errors. Across four student models, it improves average accuracy over zero-shot by 27.2, 11.2, and 3.4 percentage points on AppWorld, BFCL V3, and ToolSandbox, respectively.
  

• ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory (2026.07)
  Description: This paper presents ABot-AgentOS, a deliberative runtime layer above robot controllers that coordinates planning, isolated skill execution, verification, edge-cloud collaboration, and persistent multimodal graph memory. It also introduces EmbodiedWorldBench and a leakage-resistant self-evolution loop that turns diagnosed memory failures into gated runtime improvements for long-horizon embodied tasks.
  

• Trajectory-Informed Memory Generation for Self-Improving Agent Systems (2026.03)
  Description: This paper presents a framework that extracts actionable learnings from LLM agent execution trajectories and retrieves them as contextual memory for future tasks. It combines trajectory intelligence extraction, decision attribution, contextual learning generation, and adaptive memory retrieval to improve AppWorld task completion, especially on complex scenarios.
  

• Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs (2025.10)
  Description: This paper introduces BEAM, a benchmark of long, coherent conversations and probing questions for evaluating long-term memory in LLMs, and proposes LIGHT, a memory framework with episodic memory, working memory, and a scratchpad. Together, they expose long-context memory limitations and improve performance on long-horizon conversational reasoning tasks.
  

• MLP Memory: A Retriever-Pretrained Memory for Large Language Models (2025.08)
  Description: This paper introduces MLP Memory, a lightweight parametric module that learns to internalize retrieval patterns by pretraining an MLP to imitate a kNN retriever's behavior, bridging the gap between RAG and fine-tuning approaches.
  

• MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent (2025.07)
  Description: This paper introduces MemAgent, a multi-conversation RL-based memory agent that addresses the challenge of handling infinitely long documents with linear complexity without performance degradation during extrapolation.
  

Vision

Object Detection

• DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022.03)
  Description: DINO improves DETR-like object detectors by introducing contrastive denoising training, mixed query selection for anchor initialization, and a look-forward-twice box prediction scheme, achieving state-of-the-art results on COCO with significantly reduced model and data requirements.
  

Semantic Correspondence

• SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models (2026.05)
  Description: SOCO introduces a taxonomy-driven benchmark with consistent, functionally meaningful keypoint annotations across 100 categories and more than one million correspondence pairs. Its evaluation reveals gaps in cross-category transfer and object-part geometry understanding while showing that correspondence performance strongly predicts dense downstream task performance.
  

Auto-Prompt

Prompt Optimization

• GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning (2025.07)
  Description: GEPA introduces a prompt optimizer that uses natural language reflection to learn high-level rules from trial and error, outperforming GRPO by 6% on average with up to 35x fewer rollouts. It also beats MIPROv2 by over 10% and shows promising results as an inference-time search strategy for code optimization.
  

Judge Prompting

• Becoming Experienced Judges: Selective Test-Time Learning for Evaluators (2025.12)
  Description: This paper introduces Learning While Evaluating (LWE), enabling LLM-as-a-judge systems to improve sequentially at inference time by updating an evolving meta-prompt with self-generated feedback. It further proposes Selective LWE, which updates only on self-inconsistent cases to improve evaluation quality with better cost efficiency.
  

• Auto-Prompt Ensemble for LLM Judge (2025.10)
  Description: APE improves LLM-as-a-judge reliability by automatically discovering auxiliary evaluation dimensions from failure cases and ensembling them with confidence-aware selection. It boosts agreement with human-aligned benchmarks by using test-time computation more effectively.
  

Notes

Paper Readings

• DeepSeek-V4 Hybrid Attention: From Compressed Memory to Million-Token Context (Created: 2026-09-18; Updated: 2026-09-18)
  Description: A derivation-first reading of DeepSeek-V4's CSA, Lightning Indexer, HCA, and mHC residual paths, grounded in the technical report and a pinned official inference implementation.
  [Note] [中文]

• The DINO Series: Architecture, Training, and Task Adaptation (Created: 2026-09-12; Updated: 2026-09-12)
  Description: A practical account of DINO, DINOv2, and DINOv3: self-distillation, local features, training steps, and losses, with a DINOv2 + SimCLR retrieval case study.
  [Note] [中文]

• Self-Evolving Coding Agents: Taxonomy, Feedback Loops, and Trustworthy Evolution (Created: 2026-08-17; Updated: 2026-08-22)
  Description: A structured reading of how coding agents evolve their frameworks, memory, skills and tools, models, and collaboration structures, organized by evolution object, timing, and software evidence, with an extension on validation, versioning, and rollback for trustworthy evolution.
  [Note] [中文]

• SkillRL: How a Failure Becomes a Reusable Skill (Created: 2026-08-10; Updated: 2026-08-11)
  Description: A reading of SkillRL through ALFWorld's Skipping State Changes failure, tracing candidate-skill extraction, SkillBank organization, skill-aware policy training, and the boundary between paper evidence, repository behavior, and production admission controls.
  [Note] [中文]

• Kimi Linear and KDA: From Channel-Wise Forgetting to Hardware-Efficient Linear Attention (Created: 2026-08-03; Updated: 2026-08-22)
  Description: A technical reading of Kimi Linear through its recurrence, WY/UT parallel algorithm, 3:1 KDA/MLA hybrid, experiments, and public implementations.
  [Note] [中文]

• Kimi K3: Architecture, Training, and Million-Token Agentic RL (Created: 2026-08-01; Updated: 2026-08-22)
  Description: A plain-language explanation of Kimi K3's KDA, Block AttnRes, LatentMoE, 1M context, and nine-policy RL, with clear boundaries between paper claims and external tests.
  [Note] [中文]

• Data Agents: Levels, State of the Art, and Open Problems (Created: 2026-07-29; Updated: 2026-08-15)
  Description: A lifecycle-based reading of the L0–L5 autonomy framework, tracing data agents from responsive assistants and procedural executors to supervised orchestrators while examining gaps in responsibility, governance, and evaluation.
  [Note] [中文]

• From Qwen to Qwen3.6: The Architecture and Training Evolution of Seven Generations (Created: 2026-07-28; Updated: 2026-08-22)
  Description: A seven-generation account of how Qwen evolved across attention, MoE, data, post-training, native multimodality, and agent environments—and why Qwen3-Next is an architectural bridge rather than a separate generation.
  [Note] [中文]

• SAPO: Replacing Hard Ratio Clipping with a Continuous Soft Gate (Created: 2026-06-30; Updated: 2026-07-01)
  Description: An analysis of SAPO through its surrogate objective, gradient weights, asymmetric temperatures, and Qwen3-VL experiments, with attention to its token-level behavior and sequence-level approximation.
  [Note] [中文]

• Entropy Collapse: Policy Entropy Consumption in LLM Reinforcement Learning (Created: 2026-06-18; Updated: 2026-06-19)
  Description: A note on entropy collapse in LLM reinforcement learning, covering policy entropy, the difference between SFT and RL, DAPO's Clip-Higher strategy, and covariance regularization.
  [Note] [中文]

• From Qwen-VL to Qwen3-VL: Four Generations of Architecture and Training (Created: 2026-06-15; Updated: 2026-06-15)
  Description: A technical review of how four Qwen-VL generations evolved across vision-language alignment, dynamic resolution, spatiotemporal position encoding, video modeling, and deep visual fusion.
  [Note] [中文]

• CapRL: Stimulating Vision-Language Captioning Capabilities with Reinforcement Learning (Created: 2026-06-15; Updated: 2026-06-15)
  Description: CapRL evaluates captions through the MCQ accuracy of a vision-free LLM, turning subjective caption-quality scoring into a verifiable reward for training image-captioning models.
  [Note] [中文]

Technical Reflections

• Jev in One Reading: From a System One Decision Model to Interactive Agents (Created: 2026-09-22; Updated: 2026-09-23)
  Description: A source-bounded explanation of why Jev gives up text generation, how its probabilistic interface works, and where it fits in agents, video editing, and interactive narratives.
  [Note] [中文]

• KDA and Gated DeltaNet: How Linear Attention Reads and Writes Finite State (Created: 2026-09-20; Updated: 2026-09-20)
  Description: A concise comparison of Kimi KDA and Qwen Gated DeltaNet through the delta rule, decay granularity, hybrid stacks, and public code.
  [Note] [中文]

• A Comprehensive Guide to RSI (Self-Evolution, Self-Evolving Agents) (Created: 2026-09-14; Updated: 2026-09-14)
  Description: A taxonomy of Recursive Self-Improvement (RSI) across improvement targets, evolution structures, update roles, timing, and feedback.
  [Note] [中文]

• Query Understanding and Rewriting: Training, Retrieval, and Agent Architecture Boundaries (Created: 2026-09-07; Updated: 2026-09-08)
  Description: Intent recognition, clarification, behavior-driven rewriting, distillation, and retrieval evaluation, with evidence-dependent multi-hop training data and a discussion of Agent state management versus AIGC recaptioning.
  [Note] [中文]

• From Policy Lag to Advantage Collapse: Effective Update Signals in LLM Reinforcement Learning (Created: 2026-09-01; Updated: 2026-09-01)
  Description: An analysis of ineffective LLM RL updates through rollout policies, PPO clipping, homogeneous GRPO rewards, and interventions across data, sampling, supervision, and optimization.
  [Note] [中文]

• Building a Cold-Start Evaluation Dataset for AI Agents Without Production Data (Created: 2026-08-28; Updated: 2026-08-28)
  Description: A practical method for building auditable pre-launch agent eval datasets from capability maps, human-authored gold seeds, controlled mutations, state oracles, resettable environments, and release gates.
  [Note] [中文]

• KL Divergence: From Mathematical Derivations to LLM Reinforcement Learning (Created: 2026-08-27; Updated: 2026-08-27)
  Description: Understand the mathematical structure of KL divergence and what it controls in PPO, DPO, GRPO, and On-Policy Distillation.
  [Note] [中文]

• Agentic RL Trajectory Data Contracts: From Harness State to Trainable Samples (Created: 2026-08-26; Updated: 2026-08-26)
  Description: A practical account of how Agentic RL preserves auditable token-level training data from model calls, context reconstruction, rollout grouping, and credit assignment.
  [Note] [中文]

• Local Credit Assignment and the Boundary of Global Correctness in OPD: Starting from a Distilled-RL Intuition (Created: 2026-08-25; Updated: 2026-09-01)
  Description: Starting from a teacher-defined dense reward intuition, this note derives OPD's reverse-KL signal, examines local agreement after a corrupted prefix, and separates the roles of KAT, BRTS, TrOPD, local support matching, and verifiers.
  [Note] [中文]

• Recursive Self-Improvement: From Artifacts and Harnesses to Model–Scaffold Co-evolution (Created: 2026-08-21; Updated: 2026-08-22)
  Description: A two-axis review of 2026 recursive self-improvement work, with an auditable standard for admitting self-generated updates.
  [Note] [中文]

• What RL Actually Changes: Reasoning Patterns, Distillation, and Controllable Post-Training (Created: 2026-08-21; Updated: 2026-08-24)
  Description: A technical reflection on SFT, RLVR, distillation, and second-stage RL through the existence, selection, transfer, and control of reasoning patterns.
  [Note] [中文]

• How to Evaluate AI Agents Scientifically: Metrics, Environments, Judges, and the Regression Loop (Created: 2026-08-19; Updated: 2026-08-28)
  Description: A practical guide to AI agent evaluation covering outcome and process metrics, environment and task design, LLM-as-a-Judge, failure attribution, and dual regression.
  [Note] [中文]

• From Clio to an Executable Skill: Engineering LLM-Assisted Embedding Clustering (Created: 2026-08-12; Updated: 2026-08-22)
  Description: An engineering reconstruction of Clio's semantic clustering subsystem, evaluated on 20 Newsgroups to expose the gains, costs, reproducibility limits, and missing privacy controls of LLM-assisted hierarchy building.
  [Note] [中文]

• The Benchmark Behind the Benchmark: Judge Noise and Trustworthy Agent Comparisons (Created: 2026-08-10; Updated: 2026-08-22)
  Description: An analysis of agent noise, judge variance, evidence contracts, continuous scoring, ensembles, and model comparison through a 106-task multi-judge experiment, with a clear distinction between consistency and correctness.
  [Note] [中文]

• Treating Agents as Algorithms: Engineering Skills, CLIs, and Workflows (Created: 2026-08-05; Updated: 2026-08-06)
  Description: An engineering reflection on constraining execution variability and making multi-step agent tasks recoverable through responsibility boundaries, progressive disclosure, Gate validation, persistent state, and Workflow orchestration.
  [Note] [中文]

• Long Trajectories, Learned Values, and Adaptive Verification: The Changing Constraints of Agentic RL (Created: 2026-07-06; Updated: 2026-07-06)
  Description: A synthesis of GLM-5.2, Qwen, GenAC, OPID, and two related studies on how long-horizon agents reshape trajectory sampling, credit assignment, reward verification, and dense supervision.
  [Note] [中文]

• Keeping an SFT Gradient in DPO: From Relative Preference to a Chosen-Likelihood Anchor (Created: 2026-07-02; Updated: 2026-08-22)
  Description: A code-centered analysis of how chosen-response SFT changes DPO gradients, data and memory constraints, and the experiments needed to retain or reject the hypothesis.
  [Note] [中文]

• Agent Experience Learning: From Textual Reflection and Programmatic Skills to Policy Internalization (Created: 2026-06-30; Updated: 2026-06-30)
  Description: A technical synthesis of how LLM agents turn task trajectories into retrievable experience, executable skills, and parametric behavior, with an emphasis on library maintenance, evaluation, and engineering boundaries.
  [Note] [中文]

• Dissecting GRPO Loss in Code: Components, Negative Values, and Objective Ascent (Created: 2026-06-30; Updated: 2026-07-02)
  Description: A line-by-line analysis of importance ratios, clipping, KL penalties, and aggregation in a minimal GRPO implementation, explaining negative loss values, objective ascent, and near-zero experiments.
  [Note] [中文]

• From 1D-RoPE to Qwen's MRoPE: Frequency Allocation in Rotary Position Embeddings (Created: 2026-06-29; Updated: 2026-06-29)
  Description: Starting from a six-dimensional vector, this note derives the relative-position property of 1D-RoPE and compares blockwise MRoPE in Qwen2.5-VL with Interleaved-MRoPE in Qwen3-VL.
  [Note] [中文]

• PPO, DPO, and GRPO: Objectives and Training Loops for LLM Alignment (Created: 2026-06-16; Updated: 2026-06-16)
  Description: A comparison of PPO, DPO, and GRPO through their objectives, advantage estimators, training loops, engineering tradeoffs, and practical boundaries.
  [Note] [中文]

• OPD: Capability Integration Interface in Post-training (Created: 2026-05-28; Updated: 2026-08-25)
  Description: A technical reflection on how OPD becomes a capability integration interface in post-training through Qwen3, GLM-5, MiMo-V2, and DeepSeek-V4.
  [Note] [中文]

Blogs

• When AI builds itself (2026-06-04)
  Description: Anthropic Institute uses public benchmarks and internal Anthropic data to argue that AI is already accelerating AI development, then discusses recursive self-improvement, future scenarios, and the need for stronger oversight and coordination.
  [Blog]