Seg-Zero: Reasoning-Chain Guided Segmentation Dataset

This repository contains a training dataset for the models presented in "Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement" and "VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning".

Seg-Zero is a novel framework that demonstrates remarkable generalizability and derives explicit chain-of-thought reasoning through cognitive reinforcement. It introduces a decoupled architecture consisting of a reasoning model and a segmentation model. The reasoning model interprets user intentions, generates explicit reasoning chains, and produces positional prompts, which are subsequently used by the segmentation model to generate precise pixel-level masks.

The official implementation and training code for Seg-Zero and VisionReasoner can be found on GitHub: https://github.com/dvlab-research/Seg-Zero.

Overview of Seg-Zero

Seg-Zero demonstrates the following features:

1. Seg-Zero exhibits emergent test-time reasoning ability. It generates a reasoning chain before producing the final segmentation mask.
2. Seg-Zero is trained exclusively using reinforcement learning, without any explicit supervised reasoning data.
3. Compared to supervised fine-tuning, our Seg-Zero achieves superior performance on both in-domain and out-of-domain data.

News

• [May 17th, 2025] 🔥 We release VisionReasoner! VisionReasoner supports multi-objects and multi-tasks.
• [March 11th, 2025] 🔥 Paper is coming!
• [March 8th, 2025] 🔥 Seg-Zero is coming! We have released the code and training data.

Dataset Structure

This dataset contains a train split with 7099 examples, totaling approximately 6.67 GB. Each example includes the following features:

• id: A unique identifier for the example.
• problem: The problem description or query text.
• solution: The expected solution or reasoning chain.
• image: The input image.
• img_height: The height of the image.
• img_width: The width of the image.

The data files are located under data/train-* within the repository.

Examples

Sample Usage (Inference)

To use a pre-trained model for inference, first download the models:

mkdir pretrained_models
cd pretrained_models
git lfs install
git clone https://huggingface.co/Ricky06662/VisionReasoner-7B

Then, you can run inference using the provided scripts:

python inference_scripts/infer_multi_object.py

The default question is "What can I have if I'm thirsty?". You can also provide your own image path and text:

python inference_scripts/infer_multi_object.py --image_path "your_image_path" --text "your question text"

Citation

If you find this dataset or the related papers useful for your research, please cite:

@article{liu2025segzero,
  title        = {Seg-Zero: Reasoning-Chain Guided  Segmentation via Cognitive Reinforcement},
  author       = {Liu, Yuqi and Peng, Bohao and Zhong, Zhisheng and Yue, Zihao and Lu, Fanbin and Yu, Bei and Jia, Jiaya},
  journal      = {arXiv preprint arXiv:2503.06520},
  year         = {2025}
}

@article{liu2025visionreasoner,
  title        = {VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning},
  author       = {Liu, Yuqi and Qu, Tianyuan and Zhong, Zhisheng and Peng, Bohao and Liu, Shu and Yu, Bei and Jia, Jiaya},
  journal = {arXiv preprint arXiv:2505.12081},
  year         = {2025}
}