A list of Offline to Online RL papers (continually updated)
103
33 commits
updated Jul 22, 2026
This is a collection of research and review papers for Offline to Online Reinforcement Learning (RL) (or Offline Online RL). If you find this repository helpful, please consider citing our survey and giving a ⭐!
Maintainers:
Please feel free to pull request with the instructions provided in Contributing.
Format:
- [title](paper linnk) [links]
- author 1, author 2, et al. arXiv/conferences/journals, month/year.
For any questions, feel free to contact: lephamvanlinh@gmail.com
Credit:
@article{le2026offline2onlinerl,
title={Awesome Offline to Online RL Papers},
author={Linh Le},
journal={https://github.com/linhlpv/awesome-offline-to-online-RL-papers},
year={2026}
}
WOMBET: World Model-based Experience Transfer for Robust and Sample-efficient Reinforcement Learning
Offline-to-Online Reinforcement Learning with Prioritized Experience Selection
Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixing
Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Control
MOORe: Model-based Offline-to-Online Reinforcement Learning
Launchpad: Learning to Schedule Using Offline and Online RL Methods
Improving Offline-to-Online Reinforcement Learning with Q Conditioned State Entropy Exploration
Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration
Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness
Sample Efficient Reward Augmentation in offline-to-online Reinforcement Learning
AWAC: Accelerating Online Reinforcement Learning with Offline Datasets page
Addressing Distribution Shift in Online Reinforcement Learning with Offline Datasets
Launchpad: Learning to Schedule Using Offline and Online RL Methods
Guiding Online Reinforcement Learning with Action-Free Offline Pretraining
Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions
PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learning
Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic
A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning
Sample Efficient Offline-to-Online Reinforcement Learning
Flow Matching with Injected Noise for Offline-to-Online Reinforcement Learning
EXPO: Stable Reinforcement Learning with Expressive Policies
Action-Free Offline-to-Online RL via Discretised State Policies
On-policy Reinforcement Fine-tuning with Offline reward for Multi-step Embodied Planning
The Three Regimes of Offline-to-Online Reinforcement Learning
Towards Optimism-Pessimism Trade-off in Model-based Offline-to-Online Reinforcement Learning
Action-Free Offline-To-Online RL via Discretised State Policies
Exploration for Deployment-Efficient Reinforcement Learning Agents
EXPO: Stable Reinforcement Learning with Expressive Policies
Trajectory Generation for Offline-to-Online Reinforcement Learning via Entropy Perspective
SAMG: Offline-to-Online Reinforcement Learning via State-Action-Conditional Offline Model Guidance
Explicitly Bounding Q‑Function Estimates for Offline-to-Online Reinforcement Learning
Flow Matching with Injected Noise for Offline-to-Online Reinforcement Learning
Efficient Zero-Shot Coordination via Offline Policy Diversity and Online Belief Reasoning
Enhancing Offline-to-Online Reinforcement Learning by Adaptive Experience Aligned Diffusion Sampling
Online Finetuning Decision Transformers with Policy Gradients
Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Data
Offline-to-Online Reinforcement Learning with Classifier-Free Diffusion Generation
Online Pre-Training for Offline-to-Online Reinforcement Learning
Leveraging Offline Data in Linear Latent Contextual Bandits
FOSP: Fine-tuning Offline Safe Policy through World Models
Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data
Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
Leveraging (Biased) Information: Multi-armed Bandits with Offline Data
Hybrid Reinforcement Learning from Offline Observation Alone
Bayesian Design Principles for Offline-to-Online Reinforcement Learning
Energy-Guided Diffusion Sampling for Offline-to-Online Reinforcement Learning
OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning
Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL
SUF: Stabilized Unconstrained Fine-Tuning for Offline-to-Online Reinforcement Learning
A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning
Efficient and Stable Offline-to-online Reinforcement Learning via Continual Policy Revitalization
ENOTO: Improving Offline-to-Online Reinforcement Learning with Q-Ensembles
Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees
Adaptive Offline Data Replay in Offline-to-Online Reinforcement Learning
Bayesian Offline-to-Online Reinforcement Learning : A Realist Approach
SERA: Sample Efficient Reward Augmentation in offline-to-online Reinforcement Learning
Planning to Go Out-of-Distribution in Offline-to-Online Reinforcement Learning
Offline RL for Online RL: Decoupled Policy Learning for Mitigating Exploration Bias
A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning
Guided Decoupled Exploration for Offline Reinforcement Learning Fine-tuning
Collaborative World Models: An Online-Offline Transfer RL Approach
Jump-Start Reinforcement Learning
Efficient Online Reinforcement Learning with Offline Data
Leveraging Offline Data in Online Reinforcement Learning
Actor-Critic Alignment for Offline-to-Online Reinforcement Learning
Warm-Start Actor-Critic: From Approximation Error to Sub-optimality Gap
Semi-Offline Reinforcement Learning for Optimized Text Generation
MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learning
Finetuning Offline World Models in the Real World
Offline Meta-Reinforcement Learning with Online Self-Supervision
Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble
Don’t Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning page
Online Tuning for Offline Decentralized Multi-Agent Reinforcement Learning
M3: Modularization for Multi-task and Multi-agent Offline Pre-training
Offline-to-online reinforcement learning with policy ensemble and policy-extended value
A list of Offline to Online RL papers (continually updated)
103
33 commits
updated Jul 22, 2026
This is a collection of research and review papers for Offline to Online Reinforcement Learning (RL) (or Offline Online RL). If you find this repository helpful, please consider citing our survey and giving a ⭐!
Maintainers:
Please feel free to pull request with the instructions provided in Contributing.
Format:
- [title](paper linnk) [links]
- author 1, author 2, et al. arXiv/conferences/journals, month/year.
For any questions, feel free to contact: lephamvanlinh@gmail.com
Credit:
@article{le2026offline2onlinerl,
title={Awesome Offline to Online RL Papers},
author={Linh Le},
journal={https://github.com/linhlpv/awesome-offline-to-online-RL-papers},
year={2026}
}
WOMBET: World Model-based Experience Transfer for Robust and Sample-efficient Reinforcement Learning
Offline-to-Online Reinforcement Learning with Prioritized Experience Selection
Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixing
Adversarial Fine-tuning in Offline-to-Online Reinforcement Learning for Robust Robot Control
MOORe: Model-based Offline-to-Online Reinforcement Learning
Launchpad: Learning to Schedule Using Offline and Online RL Methods
Improving Offline-to-Online Reinforcement Learning with Q Conditioned State Entropy Exploration
Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration
Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness
Sample Efficient Reward Augmentation in offline-to-online Reinforcement Learning
AWAC: Accelerating Online Reinforcement Learning with Offline Datasets page
Addressing Distribution Shift in Online Reinforcement Learning with Offline Datasets
Launchpad: Learning to Schedule Using Offline and Online RL Methods
Guiding Online Reinforcement Learning with Action-Free Offline Pretraining
Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions
PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learning
Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic
A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning
Sample Efficient Offline-to-Online Reinforcement Learning
Flow Matching with Injected Noise for Offline-to-Online Reinforcement Learning
EXPO: Stable Reinforcement Learning with Expressive Policies
Action-Free Offline-to-Online RL via Discretised State Policies
On-policy Reinforcement Fine-tuning with Offline reward for Multi-step Embodied Planning
The Three Regimes of Offline-to-Online Reinforcement Learning
Towards Optimism-Pessimism Trade-off in Model-based Offline-to-Online Reinforcement Learning
Action-Free Offline-To-Online RL via Discretised State Policies
Exploration for Deployment-Efficient Reinforcement Learning Agents
EXPO: Stable Reinforcement Learning with Expressive Policies
Trajectory Generation for Offline-to-Online Reinforcement Learning via Entropy Perspective
SAMG: Offline-to-Online Reinforcement Learning via State-Action-Conditional Offline Model Guidance
Explicitly Bounding Q‑Function Estimates for Offline-to-Online Reinforcement Learning
Flow Matching with Injected Noise for Offline-to-Online Reinforcement Learning
Efficient Zero-Shot Coordination via Offline Policy Diversity and Online Belief Reasoning
Enhancing Offline-to-Online Reinforcement Learning by Adaptive Experience Aligned Diffusion Sampling
Online Finetuning Decision Transformers with Policy Gradients
Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Data
Offline-to-Online Reinforcement Learning with Classifier-Free Diffusion Generation
Online Pre-Training for Offline-to-Online Reinforcement Learning
Leveraging Offline Data in Linear Latent Contextual Bandits
FOSP: Fine-tuning Offline Safe Policy through World Models
Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data
Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
Leveraging (Biased) Information: Multi-armed Bandits with Offline Data
Hybrid Reinforcement Learning from Offline Observation Alone
Bayesian Design Principles for Offline-to-Online Reinforcement Learning
Energy-Guided Diffusion Sampling for Offline-to-Online Reinforcement Learning
OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning
Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL
SUF: Stabilized Unconstrained Fine-Tuning for Offline-to-Online Reinforcement Learning
A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning
Efficient and Stable Offline-to-online Reinforcement Learning via Continual Policy Revitalization
ENOTO: Improving Offline-to-Online Reinforcement Learning with Q-Ensembles
Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees
Adaptive Offline Data Replay in Offline-to-Online Reinforcement Learning
Bayesian Offline-to-Online Reinforcement Learning : A Realist Approach
SERA: Sample Efficient Reward Augmentation in offline-to-online Reinforcement Learning
Planning to Go Out-of-Distribution in Offline-to-Online Reinforcement Learning
Offline RL for Online RL: Decoupled Policy Learning for Mitigating Exploration Bias
A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning
Guided Decoupled Exploration for Offline Reinforcement Learning Fine-tuning
Collaborative World Models: An Online-Offline Transfer RL Approach
Jump-Start Reinforcement Learning
Efficient Online Reinforcement Learning with Offline Data
Leveraging Offline Data in Online Reinforcement Learning
Actor-Critic Alignment for Offline-to-Online Reinforcement Learning
Warm-Start Actor-Critic: From Approximation Error to Sub-optimality Gap
Semi-Offline Reinforcement Learning for Optimized Text Generation
MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learning
Finetuning Offline World Models in the Real World
Offline Meta-Reinforcement Learning with Online Self-Supervision
Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble
Don’t Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning page
Online Tuning for Offline Decentralized Multi-Agent Reinforcement Learning
M3: Modularization for Multi-task and Multi-agent Offline Pre-training
Offline-to-online reinforcement learning with policy ensemble and policy-extended value