VLAct QwenOFT Qwen3-VL-4B for RoboDojo (100K Fine-tuned)
1
5 commits
2 linked in READMEs
updated Sep 4, 2026
Paper: Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
Project page: https://starvla.github.io/VLAct
Code: https://github.com/starVLA/starVLA
This repository contains a RoboDojo-fine-tuned StarVLA QwenOFT policy.
It was fine-tuned for 100,000 steps on the 35-task RoboDojo LeRobot v2.1
mixture using 2 nodes x 8 GPUs. The uploaded checkpoint is the exact
steps_100000_pytorch_model.pt used for the evaluation reported below. It is
a complete framework state dict, not a LoRA or adapter-only release.
VLAct pretrained describes only the initialization source. RoboDojo
fine-tuning started from the 100K-step 0706 multi-embodiment, three-head VLAct
pretraining run. The checkpoint released here is the resulting RoboDojo
fine-tune; it is not the raw VLAct pretraining checkpoint.
| Item | Value |
|---|---|
| Framework | StarVLA QwenOFT |
| VLM backbone | StarVLA/Qwen3-VL-4B-Instruct-Action |
| Source initializer | 0706_agilex_franka_5data_manualvel_balance33_66_50_vlm3_qwen3OFT_GR00T_PI_xrobot_3head_agilex_franka, step 100K |
| Initialization setting | reload_modules: qwen_vl_interface; training starts from step 0 (is_resume: false) |
| Fine-tuning dataset | robodojo_v21_all_h50_q99, 35 training tasks |
| Observation | Three RGB cameras at 224 x 224 plus 14D proprioceptive state |
| Action | 14D absolute joint position (abs_qpos) |
| Action horizon | 50 |
| Action model | QwenOFT diffusion model, 16 layers |
| Repeated diffusion steps | 8 |
| Inference diffusion steps | 4 |
| Evaluation execution horizon | 16 |
| Fine-tuning checkpoint | 100,000 steps |
| Fine-tuning hardware | 2 nodes x 8 GPUs |
| Batch size | 16 per GPU, global batch 256 |
The original run used the action-extended Qwen3-VL snapshot at revision
8c99a9a3db659019e6e785add928fcea8e14a18a. The packaged config.yaml
uses the public Hub model ID so that the framework can reconstruct the model.
training_config.original.yaml preserves the exact training configuration and
local provenance paths used by the run.
| Setting | Value |
|---|---|
| Training seed | 42 |
| Frozen modules | None |
| VLA / VLM loss scale | 1.0 / 0.0 |
| Gradient accumulation | 1 |
| Gradient checkpointing | Enabled |
| Optimizer | AdamW, betas (0.9, 0.95), epsilon 1e-8 |
| VLM learning rate | 1e-5 |
| Qwen-VL interface learning rate | 1e-5 |
| Action-model learning rate | 1e-4 |
| Schedule | Cosine, 5,000 warmup steps, minimum LR 5e-7 |
Values are shown as success rate (SR, %) and score; higher is better. Group values are unweighted means over their tasks, and Average is the unweighted mean over all 42 tasks.
| Evaluation group | SR (%) | Score |
|---|---|---|
| Average | 8.17 | 11.28 |
| Generalization | 4.72 | 8.18 |
| Precision | 18.33 | 23.44 |
| Long-Horizon | 15.42 | 21.29 |
| Memory | 0.56 | 0.67 |
| Open | 1.67 | 1.71 |
include_state=true), arx_x5 normalization, three
camera inputs, and executes 16 actions before requesting the next chunk.This is a scaled local reproduction protocol. It is not the official 50-episode-per-task / 2,100-episode protocol, and the values should not be presented as a verified official leaderboard submission.
| Group / task | SR (%) | Score |
|---|---|---|
| Generalization | 4.72 | 8.18 |
| stack_bowls | 26.67 | 31.67 |
| push_T | 0.00 | 0.00 |
| pack_objects_into_box | 0.00 | 5.33 |
| fold_clothes | 0.00 | 4.67 |
| hang_mugs | 3.33 | 7.83 |
| sweep_blocks | 0.00 | 0.00 |
| pour_liquid_into_cup | 6.67 | 6.67 |
| make_toast | 0.00 | 5.83 |
| arrange_largest_number | 0.00 | 2.17 |
| sort_nesting_dolls_by_size | 3.33 | 3.33 |
| store_laptop_and_headphones | 3.33 | 11.33 |
| stack_blocks | 13.33 | 19.33 |
| Precision | 18.33 | 23.44 |
| fasten_screws | 0.00 | 5.67 |
| plug_in_charger | 10.00 | 10.00 |
| insert_tubes | 56.67 | 66.00 |
| pour_balls_into_vase | 16.67 | 16.67 |
| play_Xylophone | 0.00 | 0.00 |
| deposit_coin | 10.00 | 14.67 |
| insert_key | 0.00 | 14.50 |
| build_tower | 53.33 | 60.00 |
| Long-Horizon | 15.42 | 21.29 |
| put_bottles_into_dustbin | 63.33 | 71.33 |
| fill_pen_holder | 0.00 | 12.00 |
| classify_objects | 0.00 | 4.83 |
| play_tic_tac_toe | 0.00 | 4.33 |
| fill_egg_holder | 0.00 | 2.00 |
| organize_table | 3.33 | 19.17 |
| make_kong | 56.67 | 56.67 |
| play_stacking_toy | 0.00 | 0.00 |
| Memory | 0.56 | 0.67 |
| cover_blocks | 0.00 | 0.00 |
| match_and_pick_from_conveyor | 3.33 | 3.33 |
| swap_blocks | 0.00 | 0.00 |
| swap_T | 0.00 | 0.00 |
| press_by_number | 0.00 | 0.00 |
| imitate_sorting_sequence | 0.00 | 0.67 |
| Open | 1.67 | 1.71 |
| align_blocks | 0.00 | 0.00 |
| general_pickup | 13.33 | 13.33 |
| stack_blocks_by_language | 0.00 | 0.00 |
| solve_equation | 0.00 | 0.00 |
| classify_objects_by_language | 0.00 | 0.33 |
| pick_from_conveyor_by_image | 0.00 | 0.00 |
| store_tools_in_toolbox | 0.00 | 0.00 |
| pour_by_language | 0.00 | 0.00 |
Machine-readable values and protocol metadata are in
evaluation_results.json.
README.md
config.yaml
dataset_statistics.json
evaluation_results.json
summary.jsonl
training_config.original.yaml
checkpoints/
└── steps_100000_pytorch_model.pt
config.yaml and dataset_statistics.json must remain two directory levels
above the checkpoint file; StarVLA resolves both relative to the checkpoint.
Download the repository, then start the StarVLA model server:
huggingface-cli download StarVLA/VLAct-Qwen3VL4B-OFT-RoboDojo \
--local-dir VLAct-Qwen3vl4b-OFT-RoboDojo
CKPT=VLAct-Qwen3vl4b-OFT-RoboDojo/checkpoints/steps_100000_pytorch_model.pt
python deployment/model_server/server_policy.py \
--ckpt_path "$CKPT" \
--port 57700 \
--use_bf16
RoboDojo evaluation must match the training-time input contract:
export STARVLA_CKPT_PATH="$CKPT"
export STARVLA_INCLUDE_STATE=True
export STARVLA_UNNORM_KEY=arx_x5
export STARVLA_EXECUTE_HORIZON=16
Changing state usage, camera ordering, normalization statistics, action order, or execution horizon can materially change the result.
This checkpoint is intended for research on RoboDojo with the ARX X5 dual-arm embodiment. Performance under different camera calibration, state/action ordering, normalization, robot embodiments, or real hardware has not been established.
@article{yang2026beyond,
title={Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models},
author={Yang, Senqiao and Wang, Chengyao and Chen, Yuxin and Wang, Zixuan and Tang, Longxiang and Gui, Haokun and Ye, Jinhui and Lu, Changsheng and Wu, Xiaoyang and Zhu, Mingkang and others},
journal={arXiv preprint arXiv:2608.27550},
year={2026}
}
VLAct QwenOFT Qwen3-VL-4B for RoboDojo (100K Fine-tuned)
1
5 commits
2 linked in READMEs
updated Sep 4, 2026
Paper: Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
Project page: https://starvla.github.io/VLAct
Code: https://github.com/starVLA/starVLA
This repository contains a RoboDojo-fine-tuned StarVLA QwenOFT policy.
It was fine-tuned for 100,000 steps on the 35-task RoboDojo LeRobot v2.1
mixture using 2 nodes x 8 GPUs. The uploaded checkpoint is the exact
steps_100000_pytorch_model.pt used for the evaluation reported below. It is
a complete framework state dict, not a LoRA or adapter-only release.
VLAct pretrained describes only the initialization source. RoboDojo
fine-tuning started from the 100K-step 0706 multi-embodiment, three-head VLAct
pretraining run. The checkpoint released here is the resulting RoboDojo
fine-tune; it is not the raw VLAct pretraining checkpoint.
| Item | Value |
|---|---|
| Framework | StarVLA QwenOFT |
| VLM backbone | StarVLA/Qwen3-VL-4B-Instruct-Action |
| Source initializer | 0706_agilex_franka_5data_manualvel_balance33_66_50_vlm3_qwen3OFT_GR00T_PI_xrobot_3head_agilex_franka, step 100K |
| Initialization setting | reload_modules: qwen_vl_interface; training starts from step 0 (is_resume: false) |
| Fine-tuning dataset | robodojo_v21_all_h50_q99, 35 training tasks |
| Observation | Three RGB cameras at 224 x 224 plus 14D proprioceptive state |
| Action | 14D absolute joint position (abs_qpos) |
| Action horizon | 50 |
| Action model | QwenOFT diffusion model, 16 layers |
| Repeated diffusion steps | 8 |
| Inference diffusion steps | 4 |
| Evaluation execution horizon | 16 |
| Fine-tuning checkpoint | 100,000 steps |
| Fine-tuning hardware | 2 nodes x 8 GPUs |
| Batch size | 16 per GPU, global batch 256 |
The original run used the action-extended Qwen3-VL snapshot at revision
8c99a9a3db659019e6e785add928fcea8e14a18a. The packaged config.yaml
uses the public Hub model ID so that the framework can reconstruct the model.
training_config.original.yaml preserves the exact training configuration and
local provenance paths used by the run.
| Setting | Value |
|---|---|
| Training seed | 42 |
| Frozen modules | None |
| VLA / VLM loss scale | 1.0 / 0.0 |
| Gradient accumulation | 1 |
| Gradient checkpointing | Enabled |
| Optimizer | AdamW, betas (0.9, 0.95), epsilon 1e-8 |
| VLM learning rate | 1e-5 |
| Qwen-VL interface learning rate | 1e-5 |
| Action-model learning rate | 1e-4 |
| Schedule | Cosine, 5,000 warmup steps, minimum LR 5e-7 |
Values are shown as success rate (SR, %) and score; higher is better. Group values are unweighted means over their tasks, and Average is the unweighted mean over all 42 tasks.
| Evaluation group | SR (%) | Score |
|---|---|---|
| Average | 8.17 | 11.28 |
| Generalization | 4.72 | 8.18 |
| Precision | 18.33 | 23.44 |
| Long-Horizon | 15.42 | 21.29 |
| Memory | 0.56 | 0.67 |
| Open | 1.67 | 1.71 |
include_state=true), arx_x5 normalization, three
camera inputs, and executes 16 actions before requesting the next chunk.This is a scaled local reproduction protocol. It is not the official 50-episode-per-task / 2,100-episode protocol, and the values should not be presented as a verified official leaderboard submission.
| Group / task | SR (%) | Score |
|---|---|---|
| Generalization | 4.72 | 8.18 |
| stack_bowls | 26.67 | 31.67 |
| push_T | 0.00 | 0.00 |
| pack_objects_into_box | 0.00 | 5.33 |
| fold_clothes | 0.00 | 4.67 |
| hang_mugs | 3.33 | 7.83 |
| sweep_blocks | 0.00 | 0.00 |
| pour_liquid_into_cup | 6.67 | 6.67 |
| make_toast | 0.00 | 5.83 |
| arrange_largest_number | 0.00 | 2.17 |
| sort_nesting_dolls_by_size | 3.33 | 3.33 |
| store_laptop_and_headphones | 3.33 | 11.33 |
| stack_blocks | 13.33 | 19.33 |
| Precision | 18.33 | 23.44 |
| fasten_screws | 0.00 | 5.67 |
| plug_in_charger | 10.00 | 10.00 |
| insert_tubes | 56.67 | 66.00 |
| pour_balls_into_vase | 16.67 | 16.67 |
| play_Xylophone | 0.00 | 0.00 |
| deposit_coin | 10.00 | 14.67 |
| insert_key | 0.00 | 14.50 |
| build_tower | 53.33 | 60.00 |
| Long-Horizon | 15.42 | 21.29 |
| put_bottles_into_dustbin | 63.33 | 71.33 |
| fill_pen_holder | 0.00 | 12.00 |
| classify_objects | 0.00 | 4.83 |
| play_tic_tac_toe | 0.00 | 4.33 |
| fill_egg_holder | 0.00 | 2.00 |
| organize_table | 3.33 | 19.17 |
| make_kong | 56.67 | 56.67 |
| play_stacking_toy | 0.00 | 0.00 |
| Memory | 0.56 | 0.67 |
| cover_blocks | 0.00 | 0.00 |
| match_and_pick_from_conveyor | 3.33 | 3.33 |
| swap_blocks | 0.00 | 0.00 |
| swap_T | 0.00 | 0.00 |
| press_by_number | 0.00 | 0.00 |
| imitate_sorting_sequence | 0.00 | 0.67 |
| Open | 1.67 | 1.71 |
| align_blocks | 0.00 | 0.00 |
| general_pickup | 13.33 | 13.33 |
| stack_blocks_by_language | 0.00 | 0.00 |
| solve_equation | 0.00 | 0.00 |
| classify_objects_by_language | 0.00 | 0.33 |
| pick_from_conveyor_by_image | 0.00 | 0.00 |
| store_tools_in_toolbox | 0.00 | 0.00 |
| pour_by_language | 0.00 | 0.00 |
Machine-readable values and protocol metadata are in
evaluation_results.json.
README.md
config.yaml
dataset_statistics.json
evaluation_results.json
summary.jsonl
training_config.original.yaml
checkpoints/
└── steps_100000_pytorch_model.pt
config.yaml and dataset_statistics.json must remain two directory levels
above the checkpoint file; StarVLA resolves both relative to the checkpoint.
Download the repository, then start the StarVLA model server:
huggingface-cli download StarVLA/VLAct-Qwen3VL4B-OFT-RoboDojo \
--local-dir VLAct-Qwen3vl4b-OFT-RoboDojo
CKPT=VLAct-Qwen3vl4b-OFT-RoboDojo/checkpoints/steps_100000_pytorch_model.pt
python deployment/model_server/server_policy.py \
--ckpt_path "$CKPT" \
--port 57700 \
--use_bf16
RoboDojo evaluation must match the training-time input contract:
export STARVLA_CKPT_PATH="$CKPT"
export STARVLA_INCLUDE_STATE=True
export STARVLA_UNNORM_KEY=arx_x5
export STARVLA_EXECUTE_HORIZON=16
Changing state usage, camera ordering, normalization statistics, action order, or execution horizon can materially change the result.
This checkpoint is intended for research on RoboDojo with the ARX X5 dual-arm embodiment. Performance under different camera calibration, state/action ordering, normalization, robot embodiments, or real hardware has not been established.
@article{yang2026beyond,
title={Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models},
author={Yang, Senqiao and Wang, Chengyao and Chen, Yuxin and Wang, Zixuan and Tang, Longxiang and Gui, Haokun and Ye, Jinhui and Lu, Changsheng and Wu, Xiaoyang and Zhu, Mingkang and others},
journal={arXiv preprint arXiv:2608.27550},
year={2026}
}