Seg-Zero Dataset

This repository contains the training data for the Seg-Zero framework, as presented in the paper Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement.

Seg-Zero is a novel framework that demonstrates remarkable generalizability and derives explicit chain-of-thought reasoning for image segmentation tasks through cognitive reinforcement. This dataset facilitates the training of such a system, where a reasoning model interprets user intentions and generates explicit reasoning chains, which are then used by a segmentation model to produce precise pixel-level masks.

Dataset Structure and Features

This dataset is composed of entries with the following features:

• id: A unique identifier for each data instance.
• problem: The input text query or user intention that guides the segmentation.
• solution: The explicit reasoning chain generated by the reasoning model.
• image: The image on which the segmentation task is performed.
• img_height: The height of the image in pixels.
• img_width: The width of the image in pixels.

The dataset contains a train split with 9000 examples.

Seg-Zero Framework Overview

The Seg-Zero framework, which uses this dataset for training, exhibits several key features:

1. Emergent Test-Time Reasoning: It generates a reasoning chain before producing the final segmentation mask.
2. Reinforcement Learning Training: Trained exclusively using reinforcement learning, without any explicit supervised reasoning data.
3. Superior Generalization: Achieves robust zero-shot generalization and superior performance on both in-domain and out-of-domain data compared to supervised fine-tuning.

Code Repository: https://github.com/dvlab-research/Seg-Zero

Overview of Seg-Zero:

Examples of Seg-Zero's Output:

Usage

For detailed instructions on how to use this dataset for training, inference, and evaluation with the Seg-Zero framework, please refer to the official GitHub repository.

Citation

If you use this dataset or the Seg-Zero framework in your research, please cite the associated papers:

@article{liu2025segzero,
  title        = {Seg-Zero: Reasoning-Chain Guided  Segmentation via Cognitive Reinforcement},
  author       = {Liu, Yuqi and Peng, Bohao and Zhong, Zhisheng and Yue, Zihao and Lu, Fanbin and Yu, Bei and Jia, Jiaya},
  journal      = {arXiv preprint arXiv:2503.06520},
  year         = {2025}
}

@article{liu2025visionreasoner,
  title        = {VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning},
  author       = {Liu, Yuqi and Qu, Tianyuan and Zhong, Zhisheng and Peng, Bohao and Liu, Shu and Yu, Bei and Jia, Jiaya},
  journal = {arXiv preprint arXiv:2505.12081},
  year         = {2025}
}