Robo-ValueRL Dataset

[Project Page] [GitHub] [Model] [Paper]

This repository contains the dataset for Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learning.

The Robo-ValueRL dataset provides heterogeneous real-robot experience for studying reliable value estimation, value-guided offline policy pretraining, and online residual adaptation.

Dataset Description

The Robo-ValueRL dataset contains real-robot trajectories collected on two long-horizon manipulation tasks:

• Chip Insertion: millimeter-level precision manipulation requiring PCB grasping, pose adjustment, chip grasping, and insertion.
• Block Disassembly: generalizable object disassembly requiring robust grasping, separation, and classification behaviors.

The dataset includes:

• 240 hours of offline demonstrations
• 3,000+ online rollout trajectories
• Multi-view robot observations
• Language task instructions
• Robot states and action chunks
• Mixed-quality trajectories, including successful demonstrations, corrections, suboptimal behaviors, and failure cases
• Value-derived action-quality labels / conditions for policy learning
• Online rollout segments for residual adaptation

Associated Model

The dataset is released together with the Robo-ValueRL model suite:

[Robo-ValueRL Model]

The associated models include a history-conditioned value estimator, a quality-conditioned VLA policy, and an online residual adaptation module.

Data Usage

The dataset is designed for:

1. Training and evaluating history-conditioned value estimators.
2. Studying value reliability under heterogeneous robotic data.
3. Deriving action-quality conditions from value differences.
4. Training quality-conditioned VLA policies.
5. Evaluating value-guided offline-to-online reinforcement learning.
6. Training online residual adapters from value-filtered rollout segments.

Task Setup

Chip Insertion

A precision manipulation task where the robot must grasp a PCB, adjust it to a feasible insertion pose, grasp a chip, and insert it into millimeter-scale clearance.

Block Disassembly

A generalizable manipulation task where the robot must grasp, separate, and classify block components under varied configurations.

Key Features

• Heterogeneous Robot Experience: Includes successful, suboptimal, corrective, and failed trajectories.
• Offline and Online Data: Supports both offline pretraining and online improvement studies.
• Value-Oriented Labels: Provides value-derived action-quality conditions for policy learning.
• Real-Robot Evaluation: Collected from physical robot manipulation tasks rather than simulation-only benchmarks.
• Offline-to-Online Pipeline Support: Designed to connect value estimation, policy pretraining, and residual adaptation.

Highlights

• 240h offline demonstrations
• 3,000+ online rollout trajectories
• Two real-robot manipulation tasks
• Multi-view visual observations
• Language-conditioned task instructions
• Action-quality labels / conditions derived from reliable value estimation

Recommended Use

This dataset can be used to reproduce the Robo-ValueRL pipeline or to study new methods for:

• value estimation in robotic manipulation
• data filtering from mixed-quality demonstrations
• quality-conditioned VLA policy learning
• offline-to-online reinforcement learning
• stable online adaptation from real-world rollouts

Please refer to the GitHub repository for data loading, preprocessing, and training scripts.

Citation

If you use the Robo-ValueRL dataset in your research, please cite our work. Citation will be updated after the arXiv release.

License

Please refer to the license file in the GitHub repository.

Contact

For questions, please open an issue on our GitHub repository.