SCRIBE Multi-Stage Educational Feedback Dataset

This dataset is derived from the work presented in SCRIBE: Structured Chain Reasoning for Interactive Behavior Explanations using Tool Calling (2025). It contains training and evaluation data for developing and benchmarking multi-hop, tool-augmented reasoning models in educational settings.

SCRIBE introduces a framework where smaller open-source LLMs are fine-tuned to provide pedagogically valid, personalized student feedback through iterative reasoning and tool calls. The dataset supports training such models through synthetic but realistic student–feedback interactions.

---

Dataset Structure

We provide four splits, reflecting two stages of fine-tuning and two distinct evaluation sets:

• train_first
  First-stage training set. Data emphasizes initial reasoning and tool selection steps.
• train_second
  Second-stage training set. Data emphasizes multi-hop reasoning, iterative tool calls, and final answer generation.
• test_geo_va
  Testing split from the Éléments de Géomatique (GEO) and Villes Africaines (VA) MOOCs.
• test_ventures
  Testing split from the Launching New Ventures (LNV) MOOC.

Each example includes:

• A student question (e.g., "How can I improve my performance?")
• A feedback report context (course-specific)
• A reasoning chain (step-by-step natural language reasoning)
• Tool calls and outputs (intermediate steps)
• A final model response

---

Data Generation

• Human data collection: Real student questions (28 STEM students across multiple MOOCs).
• Synthetic augmentation: GPT-4o generated ~7,000 additional student-like questions with reasoning traces and tool interactions.
• Filtering: A GPT-as-a-Judge framework validated responses based on relevance, actionability, tool relevance, correctness, and grammar.

Courses included:

• DSP – Digital Signal Processing
• GEO – Éléments de Géomatique
• VA – Villes Africaines
• Ventures – Launching New Ventures (held-out test set)

---