Zebra‑CoT

A diverse large-scale dataset for interleaved vision‑language reasoning traces.

---

Dataset Description

Zebra‑CoT is a diverse large‑scale dataset with 182,384 samples containing logically coherent interleaved text‑image reasoning traces across four major categories: scientific reasoning, 2D visual reasoning, 3D visual reasoning, and visual logic & strategic games.

---

Dataset Structure

Each example in Zebra‑CoT consists of:

• Problem statement: textual description of the question.
• Problem image: Zero or more images accompanying the problem, depending on its nature.
• Reasoning image: At least one or more visual aids that support intermediate reasoning steps during problem solving.
• Text Reasoning Trace: a sequence of text thoughts (THOUGHT x) and corresponding visual sketches or diagrams placeholders, such as <image_start>[problem_image_1]<image_end> and <image_start>[reasoning_image_1]<image_end>.
• Final answer: the solution to the problem.

---

Usage

• To prepare the interleaved text-image traces for training, replace all <image_start>[problem_image_x]<image_end> and <image_start>[reasoning_image_x]<image_end> in the text trace with the actual images. We performed careful data cleaning to make sure each image and image placeholder has a one to one mapping.
• For process supervision related training, you can search for the pattern THOUGHT_x and treat it as a step. We also performed rigorous check to make sure each THOUGHT_x only appears once in a single reasoning trace.
• Additionally, to wrap the text thoughts with thinking tokens such as <think> and </think>, simply look for the adjacent image placeholders such as <image_start>[reasoning_image_i]<image_end> and <image_start>[reasoning_image_{i+1}]<image_end>, and wrap the text within the thinking tokens. You can further remove the THOUGHT_x patterns to create clean thinking flows.

---

Statistics

Statistics are detailed in Table 3 of the paper.

---

Models Finetuned with Zebra‑CoT

• Anole‑Zebra‑CoT: A 7B parameter vision–language model based on Anole‑7B and fine‑tuned on Zebra‑CoT to generate interleaved visual Chain‑of‑Thought (CoT) reasoning.
• Bagel‑Zebra‑CoT: A 7B parameter vision–language model based on Bagel‑7B and fine‑tuned on Zebra‑CoT to generate interleaved visual Chain‑of‑Thought (CoT) reasoning.

---

Citation

If you use Zebra‑CoT, please cite:

@inproceedings{
  li2026zebracot,
  title={Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning},
  author={Ang Li and Charles Wang and Deqing Fu and Kaiyu Yue and Zikui Cai and Wang Bill Zhu and Ollie Liu and Peng Guo and Willie Neiswanger and Furong Huang and Tom Goldstein and Micah Goldblum},
  booktitle={The Fourteenth International Conference on Learning Representations},
  year={2026},
  url={https://openreview.net/forum?id=c6XIVI3TiQ}
}