CoT-Self-Instruct: High-Quality Synthetic Data Generation

Generate high-quality synthetic training data using Chain-of-Thought Self-Instruct methodology. This UV script implements the approach from "CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks" (2025).

🚀 Quick Start

# Install UV if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Generate synthetic reasoning data
uv run cot-self-instruct.py \
    --seed-dataset davanstrien/math-examples \
    --output-dataset username/synthetic-math \
    --task-type reasoning \
    --num-samples 1000

# Generate synthetic instruction data
uv run cot-self-instruct.py \
    --seed-dataset wildchat-filtered \
    --output-dataset username/synthetic-instructions \
    --task-type instruction \
    --filter-method rip

📋 Overview

CoT-Self-Instruct uses Chain-of-Thought reasoning to generate high-quality synthetic training data. The method:

1. Analyzes seed examples using step-by-step reasoning
2. Generates new examples of similar quality and complexity
3. Filters outputs using automatic quality metrics

Key Benefits

• Superior Quality: CoT reasoning produces more thoughtful synthetic data
• Versatile: Works for both reasoning tasks (math, logic) and general instructions
• Efficient: Generate thousands of examples from small seed sets
• GPU-Optimized: Uses vLLM for fast inference

🎯 Use Cases

Reasoning Tasks

Generate math problems, logic puzzles, and other verifiable reasoning tasks:

• Questions with step-by-step solutions
• Single, verifiable answers (numbers, yes/no, multiple choice)
• Answer-Consistency filtering ensures correctness

Instruction Following

Generate diverse prompts for general tasks:

• Creative writing, coding, analysis prompts
• Domain-specific instructions
• RIP filtering ensures high quality

🛠️ Installation & Usage

Local Execution

# Basic usage - auto-detects task type
uv run cot-self-instruct.py \
    --seed-dataset your-seed-data \
    --output-dataset your-output-data

# Advanced usage with all options
uv run cot-self-instruct.py \
    --seed-dataset math-seed-data \
    --output-dataset synthetic-math \
    --task-type reasoning \
    --generation-model Qwen/Qwen3-30B-A3B-Thinking-2507 \
    --filter-model Qwen/Qwen3-235B-A22B-Instruct-2507 \
    --num-samples 5000 \
    --filter-method answer-consistency \
    --k-responses 16 \
    --quality-threshold 0.5

HuggingFace Jobs (Recommended for Scale)

# Run on HF Jobs with multi-GPU
hf jobs uv run \
    --flavor l4x4 \
    --image vllm/vllm-openai \
    -e HF_TOKEN=$(python3 -c "from huggingface_hub import get_token; print(get_token())") \
    https://huggingface.co/datasets/uv-scripts/synthetic-data/raw/main/cot-self-instruct.py \
    --seed-dataset your-seed-data \
    --output-dataset your-output-data \
    --num-samples 10000

📊 Example Seed Data

Reasoning Tasks

Your seed dataset should have question and answer columns:

[
  {
    "question": "Find the value of $x$ such that $\\sqrt{x + 7} = 4$.",
    "answer": "9"
  },
  {
    "question": "If $f(x) = 3x^2 - 2x + 1$, what is $f(2)$?",
    "answer": "9"
  }
]

Instruction Tasks

Your seed dataset should have a prompt column:

[
  {
    "prompt": "Write a Python function that checks if a string is a palindrome."
  },
  {
    "prompt": "Explain machine learning to a 10-year-old child."
  }
]

🔧 Parameters

Core Parameters

• --seed-dataset: HuggingFace dataset with seed examples
• --output-dataset: Where to save generated data
• --task-type: reasoning, instruction, or auto (auto-detects)
• --num-samples: Number of synthetic examples to generate (default: 5000)

Model Configuration

• --generation-model: Model for synthetic generation (default: Qwen3-30B-Thinking)
• --filter-model: Model for filtering (defaults to generation model)
• --reward-model: Reward model for RIP filtering

Quality Filtering

• --filter-method: answer-consistency, rip, both, or none
• --k-responses: Number of responses for filtering (default: 16)
• --quality-threshold: Minimum quality score (default: 0.5)

GPU Configuration

• --tensor-parallel-size: Number of GPUs (auto-detected)
• --gpu-memory-utilization: GPU memory usage (default: 0.9)

🏗️ How It Works

1. Seed Selection

• For reasoning: Randomly samples 2 seed questions
• For instructions: Samples from same category (auto-categorized)

2. Chain-of-Thought Generation

The model:

• Analyzes seed examples for patterns and complexity
• Reasons step-by-step about what makes them effective
• Generates new examples following the identified patterns

3. Quality Filtering

Answer-Consistency (Reasoning):

• Generates K responses to each synthetic question
• Computes majority answer
• Keeps only if majority matches generated answer

RIP - Rejecting Instruction Preferences (Both):

• Generates K responses to each prompt
• Scores with reward model
• Uses minimum score as quality indicator

📈 Performance

Based on the paper's results:

• Reasoning tasks: 57.2% vs 44.6% (baseline) on benchmarks
• Instruction tasks: 54.7% vs 46.8% (human data) on evaluations

🤝 Contributing

This is part of the uv-scripts organization. Contributions welcome!

📚 Citation

If you use this script, please cite the original paper:

@article{yu2025cot,
  title={CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks},
  author={Yu, Ping and others},
  journal={arXiv preprint arXiv:2507.23751},
  year={2025}
}

📝 License

Apache 2.0