:fire: Updates

• 2026/06/14: 🔥🔥🔥Release the training and evaluation code for the Qwen3-VL version. See the qwen3vl branch.
• 2026/02/16: 🔥🔥🔥Release the training code for SFT and RL.
• 2026/02/15: 🔥🔥🔥Release the evaluation code for Video-o3.
• 2026/02/10: 🔥🔥🔥Release the checkpoint of Video-o3 (RL) and Video-o3 (SFT+RL).
• 2026/02/10: 🔥🔥🔥Release Seeker-173K, a large-scale dataset comprising 173K high-quality tool-interaction trajectories for effective supervised and reinforcement learning.
• 2026/01/30: 🔥🔥🔥Release the paper of Video-o3, a novel framework that supports native interleaved clue seeking for long video multi-hop reasoning.

📑 Todo List

• Refine the user documentation and tutorials.
• Update with more engaging and interactive demos.
• Provide a streamlined guide for quick inference.

🧠 Motivation: Thinking with Videos

Current Multimodal Large Language Models (MLLMs) struggle with long videos because they typically rely on uniform frame sampling and single-turn inference. This approach often dilutes critical visual evidence within redundant background content.

Video-o3 introduces a paradigm shift by mimicking human behavior. Instead of watching a video passively, it actively explores the content. The model iteratively discovers salient visual clues, inspects key segments with fine-grained detail, and adaptively terminates the search once sufficient evidence is acquired.

Key Features:

• Goal-Driven Exploration: Unlike models that scan the whole video coarsely, Video-o3 starts with a coarse scan and iteratively focuses on informative segments.
• Native Interleaved Tool Use: The model supports "clue seeking" and "answer reasoning" within a single shared context, rather than decoupled modules.

🛠️ Method: Native Interleaved Tool Invocation

Video-o3 is designed to solve the challenges of attention dispersion and contextual efficiency in long-video processing. The framework operates on a Think-and-Tool cycle. The model generates structured directives containing temporal windows and visual token quotas. It dynamically invokes the VideoCrop tool to inspect target segments with adaptive spatiotemporal resolution.

Core Technical Innovations:

• Task-Decoupled Attention Masking: To prevent attention dispersion, TDAM isolates per-step concentration. During clue seeking, the model attends only to the global context; during answering, it focuses on high-resolution tool observations.
• Verifiable Trajectory-Guided Reward: To control context length and cost, we introduce a reward mechanism that balances exploration coverage with reasoning efficiency, encouraging the model to terminate precisely when evidence is sufficient.

📊 Data Construction: Pipeline and Seeker-173K

Training a model to perform native interleaved tool invocation requires high-quality exploration trajectories, which are scarce in existing datasets. To address this, we developed a scalable automated data synthesis pipeline.

About Seeker-173K:

• Structure: The dataset is stratified into a four-quadrant taxonomy based on evidence cardinality and visual saliency, covering tasks from "Single-Clue Direct Answering" to complex "Multi-Clue Tool Invocation".
• Quality: Human verification is enforced through random sampling in all stages. The pipeline rigorously filters out flawed instances, preserving only those with sound logic and factual visual evidence.

📈 Performance

Comparison of our method with existing approaches on video question answering tasks across various benchmarks. Video-o3 significantly outperforms previous methods in long video understanding benchmarks while also demonstrating strong performance in multiple video inference benchmarks.

🖊️ Citation

@article{zeng2026video,
  title={Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning},
  author={Zeng, Xiangyu and Zhang, Zhiqiu and Zhu, Yuhan and Li, Xinhao and Wang, Zikang and Ma, Changlian and Zhang, Qingyu and Huang, Zizheng and Ouyang, Kun and Jiang, Tianxiang and others},
  journal={arXiv preprint arXiv:2601.23224},
  year={2026}
}

:dizzy: Acknowledgement

Thanks to the open source of the following projects:

• Qwen2.5-VL
• Mini-o3
• LongVT