Chain-of-Thought for Vision-Language-Action Models in Uncertain Scenarios

Donghwa Kang, Sehee Kweon, Wooyul Jung

---

CoT in VLA is an Chain-of-Thought (ECoT) reasoning in vision-language-action (VLA) models in uncertain scenarios. CoT is used to improve success rate and reasoning faithfulness when the uncertainty from the action tokens is high. Experiments in LIBERO simulation shows proper Chain-of-Thought is helpful by improving task success rate and reasoning faithfulness.

Dataset

We train and evaluate our method on the LIBERO-Spatial dataset, which contains 10 manipulation tasks that require spatial reasoning — for example, placing an object relative to another object ("put the bowl on top of the plate," "put the mug to the left of the plate"). These tasks are well-suited for evaluating the effect of reasoning, since the robot must identify spatial relationships before acting.

Why LIBERO-Spatial?

• Reasoning-sensitive tasks: The spatial nature of the tasks allows us to clearly observe the impact of Embodied Chain-of-Thought (ECoT) reasoning on uncertain or ambiguous scenes.
• Randomized evaluation environments: LIBERO generates a new scene configuration (object positions, distractors, etc.) every time an evaluation episode is run. This provides a clean separation between training and evaluation conditions, so reported success rates reflect true generalization rather than memorization.
• Rich annotations: The dataset includes task-, plan-, and subtask-level reasoning annotations, which are necessary for training and evaluating ECoT policies.

We use two versions of LIBERO in this project:

1. Original LIBERO-Spatial (HDF5 format) — used for evaluation in the LIBERO simulator.
2. LIBERO-RLDS (modified, no-noops) — used for training the VLA policy, following the OpenVLA/ECoT pipeline.

---

1. (for evaluation) Original LIBERO-Spatial

The original LIBERO dataset is provided in HDF5 format by the official LIBERO repository. It is required for running evaluation rollouts in the LIBERO simulator.

# Clone the LIBERO repo (provides the simulator + download utility)
git clone https://github.com/Lifelong-Robot-Learning/LIBERO.git
cd LIBERO
pip install -e .

# Download the LIBERO-Spatial split
python benchmark_scripts/download_libero_datasets.py --datasets libero_spatial

# Saved under: ./datasets/libero_spatial/

---

2. (for training) LIBERO-RLDS — modified, no-noops

For training, we use the RLDS-formatted version released by OpenVLA, which removes no-op actions for more efficient policy learning. This is the standard training format used by OpenVLA and ECoT.

Source: openvla/modified_libero_rlds

# Install huggingface CLI if not already installed
pip install -U "huggingface_hub[cli]"

# Download the LIBERO-Spatial (no-noops) RLDS split
huggingface-cli download openvla/modified_libero_rlds \
    --repo-type dataset \
    --include "libero_spatial_no_noops/*" \
    --local-dir ./datasets/modified_libero_rlds

Note: The HuggingFace repo contains multiple splits (libero_spatial_no_noops, libero_object_no_noops, libero_goal_no_noops, libero_10_no_noops). Replace the --include pattern above with the split you want to download. To grab all splits at once, omit the --include flag.

After downloading, the directory structure should look like:
datasets/
├── libero_spatial/                        # original HDF5 (evaluation)
│   └── *.hdf5
└── modified_libero_rlds/                  # RLDS (training)
    └── libero_spatial_no_noops/
    └── 1.0.0/
    ├── dataset_info.json
    ├── features.json
    └── .tfrecord-

Update the dataset paths in your config file accordingly:

# configs/dataset.yaml
eval_dataset_path: ./datasets/libero_spatial/
train_dataset_path: ./datasets/modified_libero_rlds/libero_spatial_no_noops/1.0.0/

Model

We build on two Vision-Language-Action (VLA) architectures: OpenVLA and ECoT (Embodied Chain-of-Thought).

OpenVLA

OpenVLA is a 7B-parameter VLA model built on a vision-language backbone (Prismatic VLM). It takes a single RGB image and a natural language task instruction as input, and directly predicts a 7-DoF robot action (Δx, Δy, Δz, Δroll, Δpitch, Δyaw, gripper).

ECoT (Embodied Chain-of-Thought)

ECoT extends OpenVLA by injecting a chain-of-thought reasoning step before action prediction. Given the same image and instruction, the model first generates structured reasoning tokens — task description, plan, and subtask decomposition — then predicts the action conditioned on this reasoning trace. This explicit reasoning improves performance on tasks that require spatial understanding and multi-step planning.

Pretrained Checkpoints

To download a checkpoint:

# OpenVLA fine-tuned
git clone https://huggingface.co/openvla/openvla-7b-finetuned-libero-spatial
cd openvla-7b-finetuned-libero-spatial && git lfs fetch --all && cd ..

# ECoT LoRA
git clone https://huggingface.co/leepanic/ecot-libero-spatial-r32
cd ecot-libero-spatial-r32 && git lfs fetch --all && cd ..

Quantitative Results

Success Rate Comparison

Qualitative Results

Both demos are evaluated in the LIBERO environment, where the scene is reset with a new configuration each time (different from the training data).

Prerequisites for Training and Evaluation

• OS: Linux (Ubuntu 20.04 / 22.04 recommended)
• GPU: (for training) NVIDIA GPU with ≥ 24 GB VRAM (e.g., RTX 3090 / 4090, A100) (for evaluation) NVIDIA GPU with ≥ 16 GB VRAM (e.g., RTX 3060 / 3070, A100)
• CUDA: 12.1 or 12.4
• Python: 3.10
• Conda (Miniconda or Anaconda)

1. Create the conda environment

conda create -n VLA python=3.10 -y
conda activate VLA

2. Install PyTorch

Install the PyTorch build that matches your CUDA version. See the official selector if you need a different version.

# CUDA 12.4
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia -y

3. Clone and install Adaptive-CoT-in-VLA repository

git clone https://github.com/seram7/Adaptive-CoT-in-VLA.git
cd Adaptive-CoT-in-VLA
pip install -e .

4. Install LIBERO simulator (required for evaluation)

git clone https://github.com/Lifelong-Robot-Learning/LIBERO.git
cd LIBERO
pip install -e .
cd ..

5. Verify installation

python -c "import torch; print('PyTorch:', torch.__version__, '| CUDA available:', torch.cuda.is_available())"
python -c "import flash_attn; print('Flash-Attn:', flash_attn.__version__)"
python -c "import libero; print('LIBERO: OK')"

All three commands should print without error.

Training

Prerequisites

Make sure you have completed the installation steps before proceeding.

Download Base Models and Data

# Install Git LFS
sudo apt update && sudo apt install git-lfs
git lfs install

# Download the base model
git clone https://huggingface.co/openvla/openvla-7b-prismatic
cd openvla-7b-prismatic && git lfs fetch --all && cd ..

# (Optional) Download the ECoT-finetuned base model for LoRA fine-tuning
git clone https://huggingface.co/Embodied-CoT/ecot-openvla-7b-oxe

# Download dataset (written in [Dataset section](#dataset))
git clone https://huggingface.co/datasets/openvla/modified_libero_rlds

Prepare Reasoning Annotations

If using ECoT reasoning, place the reasoning JSON file in the dataset directory:

cp reasoning.json <DATA_ROOT_DIR>/<DATASET_NAME>/reasoning.json

Note: <DATA_ROOT_DIR> is the parent directory containing your RLDS datasets (e.g., ./datasets/modified_libero_rlds), and <DATASET_NAME> is the specific split you're training on (e.g., libero_spatial_no_noops).

---

Full Fine-Tuning

Train from scratch on RLDS-formatted datasets:

torchrun --standalone --nnodes 1 --nproc-per-node 8 vla-scripts/train.py \
  --vla.type "prism-dinosiglip-224px+mx-bridge" \
  --data_root_dir <PATH_TO_DATA> \
  --run_root_dir <PATH_TO_CHECKPOINTS> \
  --wandb_project <WANDB_PROJECT> \
  --wandb_entity <WANDB_ENTITY>

LoRA Fine-Tuning

Parameter-efficient fine-tuning with LoRA on specific datasets:

torchrun --standalone --nnodes 1 --nproc-per-node 2 vla-scripts/finetune.py \
  --vla_path "Embodied-CoT/ecot-openvla-7b-oxe" \
  --data_root_dir <PATH_TO_DATA> \
  --dataset_name <DATASET_NAME> \
  --run_root_dir <PATH_TO_CHECKPOINTS> \
  --adapter_tmp_dir <PATH_TO_ADAPTER_TMP> \
  --lora_rank 32 \
  --batch_size 1 \
  --grad_accumulation_steps 1 \
  --learning_rate 5e-4 \
  --image_aug True \
  --wandb_project <WANDB_PROJECT> \
  --wandb_entity <WANDB_ENTITY> \
  --save_steps 20000

Key Parameters

Memory Requirements

Evaluation

We evaluate both OpenVLA (without CoT) and ECoT (with CoT) on the LIBERO-Spatial benchmark to measure how Chain-of-Thought reasoning affects robot performance under uncertainty.

Each task is evaluated across 10 trials. At every step, we measure the model's uncertainty using action entropy — entropy computed over the 256 action token bins. A higher entropy means the model is more uncertain about which action to take.

Metrics

• Success Rate — whether the robot completes the task
• Action Entropy — uncertainty of the model at each step

Requirements

• GPU: NVIDIA GPU with at least 24GB VRAM (e.g. RTX 3090, A100)
• CUDA: 11.8 or higher

Setup

Make sure LIBERO is installed and the path is correctly set in the script:

sys.path.insert(0, "/your/path/to/LIBERO")

Run Evaluation

Single GPU:

python rollout_ECoT.py

Output

Results are saved under ./rollouts/{date}/{task_description}/:

• task{id}_trial{id}.pt — logits, entropy, and success per rollout
• Replay videos of each episode
• Reasoning text per step

Notes

• Each task runs 10 trials by default
• Max steps per episode: 100 (libero_spatial)
• The scene is reset with a new configuration each trial, different from training data

References

This codebase is built on top of the following excellent works:

• Fast ECoT: Efficient Embodied Chain-of-Thought via Thoughts Reuse by Duan et al. — The original Fast ECoT method for accelerating reasoning in VLA models (arXiv:2506.07639)
• Embodied Chain-of-Thought (ECoT) by Zawalski et al. — The original ECoT reasoning framework for VLA models (arXiv:2407.08693)
• OpenVLA — The base VLA model architecture and training pipeline
• vLLM — High-throughput LLM inference engine used for accelerated reasoning generation

Discussion

Overall, our results suggest that ECoT performs better than OpenVLA in uncertain situations, but the story is more nuanced than a simple comparison.

What we found:

• Action entropy is a reasonable proxy for task difficulty — harder tasks tend to have higher entropy
• CoT reasoning helps the model make better decisions when uncertainty is high
• However, CoT comes with a cost: longer inference time per step

Key insight: This suggests an adaptive strategy — we don't need to always use CoT. Instead, we can monitor the model's entropy in real time and only trigger CoT reasoning when uncertainty is above a certain threshold. This way, we can get the benefits of CoT without paying the full cost on every step.

Limitations:

• Evaluation was limited to LIBERO-Spatial due to computational constraints
• Inference time makes real-time deployment challenging in VLA settings
• Reproducibility was harder than expected — the original repo lacked documentation, which made replication non-trivial

What we learned: Understanding the low-level details of VLA systems — input/output formats, action token structure, and model architecture — was essential to making progress. AI tools helped speed up the process, but only when paired with a solid understanding of the system.