PRIMO COT SFT 7B

The stage-1 checkpoint of PRIMO R1, from the paper From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation.

This model has two roles:

1. The SFT-only ablation. It is what you compare against PRIMO-R1-7B to isolate what reinforcement learning contributes.
2. The RL starting point. Stage-2 GRPO training begins from this checkpoint — run_rl_7b.sh reads it as SFT_CKPT. If you want to run your own RL on top of the paper's cold start, this is the model to point at.

For everything else, use PRIMO-R1-7B. It is strictly better at the task.

Model Description

Current video MLLMs often function as passive "Observers" that recognize ongoing events rather than evaluating the current state relative to the final task goal. PRIMO R1 transforms these models into active "Critics" by:

• Reinforcement Learning: Leveraging outcome-based RL to incentivize explicit Chain-of-Thought (CoT) generation for progress estimation.
• Temporal Anchoring: Constructing a structured temporal input that explicitly anchors the video sequence between initial and current state images.
• Process Reasoning: Focusing on evaluating the current state against the intended task goal to detect failures and track progress.

This checkpoint has the temporal anchoring and the structured CoT format, but only the imitation half of the recipe: it was supervised on CoT traces token-by-token rather than optimized against a verifiable progress reward.

What the RL stage actually adds

Progress estimation MRA↑ (paper Table 2). The base model is Qwen2.5-VL-7B-Instruct:

SFT does most of the work in domain but generalizes noticeably worse out of domain — 67.30 MRA on the real-humanoid cross-environment split, where the full model reaches 82.90 average OOD.

The sharper result is on zero-shot failure detection (paper Table 3), where SFT alone regresses below the base model:

Imitating CoT traces overfits the output format at the cost of a capability the base model already had. RL recovers it and then some. This is the main argument for the two-stage recipe, and it is why this checkpoint is published as an ablation rather than as a usable critic.

Resources

Download

16.6 GB, inference files only:

hf download LeonOverload/PRIMO-COT-SFT-7B --local-dir models/PRIMO-COT-SFT-7B

Setup

git clone https://github.com/10-OASIS-01/PRIMO-R1 && cd PRIMO-R1
conda create -n primo-r1 python=3.11 && conda activate primo-r1
bash setup.sh

setup.sh pins vllm==0.7.2, trl==0.16.0, and installs the vendored transformers-main/ tree last. Installing a PyPI transformers over it is the usual cause of shape and processor errors.

Using it as the RL starting point

hf download LeonOverload/PRIMO-COT-SFT-7B --local-dir models/PRIMO-COT-SFT-7B

export SFT_CKPT=models/PRIMO-COT-SFT-7B
export VIDEO_DATA_ROOT=/path/to/PRIMO-Data
bash src/scripts/run_rl_7b.sh

RL training reads its mixture from primo-rl-json — see that card for what to download, and note the behavior-1k size warning there before pulling the full mixture.

Input format

Identical to PRIMO-R1-7B: the content list must be

image (initial frame)  →  video (the clip)  →  image (current frame)  →  text (question)

with the same SYSTEM_PROMPT and QUESTION_TEMPLATE — both imported from primo_prompts in the repo, not retyped — and the same <think><planning>/<observation>/<reasoning></think> + <answer> output contract. A bare video clip degrades output quality silently.

Usage

This example is transcribed from src/eval/eval_interleave.py, the harness used for the paper's numbers.

The prompts and the frame extraction are imported, not pasted — src/primo_prompts.py and src/primo_video_utils.py in the repo are the same objects the eval harness uses, so this example cannot drift out of sync with the checkpoint. setup.sh puts src/ on PYTHONPATH; from elsewhere, sys.path.insert(0, "/path/to/PRIMO-R1/src").

import torch
from transformers import AutoProcessor, AutoTokenizer
from vllm import LLM, SamplingParams
from qwen_vl_utils import process_vision_info

# The single source of truth for the prompt format and the anchor frames.
from primo_prompts import SYSTEM_PROMPT, build_question
from primo_video_utils import extract_frames_on_demand

MODEL_PATH = "models/PRIMO-COT-SFT-7B"   # or "LeonOverload/PRIMO-COT-SFT-7B"
video_path = "path/to/your/episode.mp4"
question = "What is the completion percentage of the task in the video?"
problem_type = "regression"

# (initial state, current state) as PIL images. LRU-cached per video path.
init_img, current_img = extract_frames_on_demand(video_path)

messages = [
    {"role": "system", "content": [{"type": "text", "text": SYSTEM_PROMPT}]},
    {
        "role": "user",
        "content": [
            {"type": "image", "image": init_img},                      # 1. initial state
            {"type": "video", "video": video_path, "nframes": 22},     # 2. the clip
            {"type": "image", "image": current_img},                   # 3. current state
            # QUESTION_TEMPLATE.format(...) + TYPE_TEMPLATE[problem_type]
            {"type": "text", "text": build_question(question, problem_type)},
        ],
    },
]

llm = LLM(
    model=MODEL_PATH,
    tensor_parallel_size=torch.cuda.device_count(),
    max_model_len=16384,
    gpu_memory_utilization=0.8,
    limit_mm_per_prompt={"image": 3, "video": 1},   # 2 anchor frames + 1 video
)

# top_p must stay this low. Larger values produce garbled output on this model.
sampling_params = SamplingParams(temperature=0.1, top_p=0.001, max_tokens=4096)

processor = AutoProcessor.from_pretrained(MODEL_PATH)
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
tokenizer.padding_side = "left"
processor.tokenizer = tokenizer

prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs, video_kwargs = process_vision_info(messages, return_video_kwargs=True)

mm_data = {"video": video_inputs[0]}
if image_inputs:
    mm_data["image"] = image_inputs

outputs = llm.generate(
    [{
        "prompt": prompt,
        "multi_modal_data": mm_data,
        "mm_processor_kwargs": {k: v[0] for k, v in video_kwargs.items()},
    }],
    sampling_params=sampling_params,
)
print(outputs[0].outputs[0].text)

Parse the result with a regex over <answer>...</answer>; for regression and numerical the value is a progress percentage on a 0–100 scale.

Reproducing the ablation

Evaluate both checkpoints on the same splits and compare. Add both to model_paths in the launcher:

# src/eval/src/eval_interleave_local.sh
model_paths=$(cat <<EOF | grep -v '^#' | grep -v '^$'
$MODEL_ROOT/PRIMO-R1-7B
$MODEL_ROOT/PRIMO-COT-SFT-7B
EOF
)

See primo-bench-json for the full setup, including which video group each split needs.

Citations

If you find our work helpful for your research, please consider citing our work.

@misc{liu2026passiveobserveractivecritic,
      title={From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation}, 
      author={Yibin Liu and Yaxing Lyu and Daqi Gao and Zhixuan Liang and Weiliang Tang and Shilong Mu and Xiaokang Yang and Yao Mu},
      year={2026},
      eprint={2603.15600},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2603.15600}, 
}