Spark-234K: Skeleton-Guided Scientific Reasoning from Large-Scale Literature

🎉 Accepted to EMNLP 2026 Findings!

Spark-234K is a scientific reasoning dataset containing 234K question-answer pairs synthesized from frontier scientific literature. Instead of directly generating QA pairs from full papers, SPARK first distills each paper into a compact reasoning skeleton—preserving its central claim, supporting evidence, quantitative relations, assumptions, and boundary conditions—and then constructs challenging, self-contained problems from complementary reasoning perspectives. （More meta information is being prepared and will be updated once ready.）

Data Construction

Built on SCI-BASE (~3.36M papers across 10 disciplines), SPARK samples ~370K frontier papers after filtering and discipline-balanced sampling, then proceeds through four stages:

1. Reasoning Skeleton Extraction — Each paper is distilled into a claim-centered skeleton capturing its core conclusion, argumentation steps, observables, and assumptions.
2. Multi-Perspective Question Generation — Questions are generated from multiple complementary reasoning perspectives to maximize difficulty and diversity.
3. Answer Generation & Quality Check — Answers are produced without access to the source paper; each triple is verified for self-containment, source consistency, answer coherence, and reasoning grounding.
4. Deduplication & Decontamination — Semantic deduplication per discipline plus 13-gram exact-match decontamination against downstream benchmarks.

Dataset Composition

Spark-234K spans 10 scientific disciplines sampled in a discipline-balanced manner, with Physics, Mathematics & Computer Science, and Chemistry as the largest shares.

Dataset Quality

Spark-234K emphasizes difficult, diverse, and self-contained reasoning:

• 99.73% self-containment rate.
• Over 93% of questions classified as L4 Multi-Step or L5 Research-Level.
• A stratified human audit over 300 samples across 10 disciplines reports 90.3% agreement with expert answers.

Results

Spark-234K consistently outperforms substantially larger scientific reasoning datasets when used for SFT on Qwen-series models:

For more detailed experimental results and analysis, please refer to our paper.

Citation

If you find Spark-234K useful, please consider citing:

@misc{li2026sparkskeletonguidedreasoningsynthesis,
      title={SPARK: Skeleton-Guided Reasoning Synthesis from Large-Scale Scientific Literature}, 
      author={Yu Li and Wei Li and Xin Gao and Mengyuan Sun and Xiaoyang Wang and Qizhi Pei and Lijun Wu},
      year={2026},
      eprint={2608.30214},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2608.30214}, 
}