AI539_NLP

Chain of thought (CoT) is a method of scaling compute at inference time. We want to investigate the effects of using intermediate “reasoning tokens” within the VLA to see if it improves performance on downstream tasks without incurring the same penalty to performance as seen in CoT-VLA using intermediate reasoning images

Proposal

See, Parse, Act: Interpretable Vision-Language-Action via Detection Tokens

Abstract

(Brief description of project goals and proposed methods)

• Chain of thought (CoT) is a method of scaling compute at inference time. We want to investigate the effects of using intermediate “reasoning tokens” within the VLA to see if it improves performance on downstream tasks without incurring the same penalty to performance as seen in CoT-VLA using intermediate reasoning images[1].

• We will use a OpenVLA based model, possibly with an adapter seen in VLA-Adapter [2]. The purpose of using the adapter is to reduce cost of training and increase training throughput.

• Our team’s highest-powered GPU is an RTX5090, we will likely be using COE HPC compute nodes available to EECS students if we need additional compute.

• We will not be doing sim-to-real or any other kind of deployment, we will only be doing simulation work with our VLA.

• We will be modeling a Franka Emika Panda arm using MuJoCo and benchmark our model’s performance on tasks using the LIBERO dataset. Starting with the object dataset, eventually benchmarking on all 4 datasets if the project proceeds smoothly.

Starting Ideas

• Converting the output of the last layer of the LLM component of the VLA into text and then feeding it back into the LLM component some number of 1/5/100/n times, the specific number of times is not a number we have at this time. The refined output will then be concatenated with the vision component and sent to the projector and then subsequently the action component.

• Consider the above is modest in its work. We will also probably investigate using the Open-VLA intermediate image generation CoT reasoning interleavened with a similar LLM intermediate labeling of the images generated by the vision component.

• The training of the two models will involve RL. The specific valuable training insight we wish to leverage from SayCan we wish to leverage is the value function [3].

• The performance of the above two experiments will then be benchmarked against a deployment of a vanilla Open-VLA model on the same LIBERO benchmarks.

• There has been work done in this area regarding using policy network outputs for scaling [4]. Perhaps if there is time we will design a third modification to explore how CoT can improve performance or explain ability guided by ACoT-VLA methodologies as well.

Figure 1. The above proposed training scheme used by SayCan to improve LLM "reasoning" regarding tasks given to it.

References

[1] Q. Zhao et al., "CoT-VLA: Visual chain-of-thought reasoning for vision-language-action models," arXiv:2503.22020, 2025. [Online]. Available: https://arxiv.org/abs/2503.22020

[2] Y. Wang et al., "VLA-Adapter: An effective paradigm for tiny-scale vision-language-action model," arXiv:2509.09372, 2025.

[3] M. Ahn et al., "Do as I can and not as I say: Grounding language in robotic affordances," arXiv:2204.01691, 2022.

[4] L. Zhong, Y. Liu, Y. Wei, Z. Xiong, M. Yao, S. Liu, and G. Ren, "ACoT-VLA: Action chain-of-thought for vision-language-action models," arXiv:2601.11404, 2026. [Online]. Available: https://arxiv.org/abs/2601.11404

Professor Lee feedback

Sounds fine. Beyond just feeding its output back in, do you want to have a stretch goal of finetuning this based on the task success? I know datasets like LIBERO have demonstration data and this is generally a verifiable reward setting.

Papers

An archive of relevant papers is located here