Nemotron-Math-Proofs-v3-SFT

Dataset Description:

Nemotron-Math-Proofs-v3-SFT is a long-form mathematical reasoning dataset containing proof-generation, proof-refinement, verification, and meta-verification traces. The release contains 414,890 samples representing 15,818 unique problems after quality filtering.

The source pool contains 15,879 hard proof problems selected from the AoPS subset of nvidia/Nemotron-Math-Proofs-v1. Responses are generated using DeepSeek-V4-Pro in Max inference mode. A four-round generate-verify-refine pipeline produces initial proofs in round 1 and refinement trajectories for problems that remain unsolved in rounds 2 through 4. The pipeline also produces verifier and meta-verifier traces following the proof-generation and proof-verification prompting style described in the DeepSeekMath-V2 paper.

Full details about dataset construction can be found in our technical report An Open Recipe for IMO Gold: Training Nemotron for Olympiad Mathematics.

This dataset is ready for commercial or non-commercial uses.

Dataset Owner(s):

NVIDIA Corporation

Dataset Creation Date:

Created on: 07/01/2026 Last Modified on: 07/01/2026

Versioning:

Nemotron-Math-Proofs-v3-SFT

Previous Version(s):

• nvidia/Nemotron-Math-Proofs-v1 : Previous proof dataset version used as the source of problems for this release.
• nvidia/Nemotron-Math-Proofs-v2 : Previous release version.
• nvidia/Nemotron-Cascade-2-SFT-Data : Contains a natural language proof subset from the same source problems.

Relationship to Previous Version(s): This release substantially expands the prior proof data with a harder source-problem pool, multi-round proof refinement, and additional verification and meta-verification traces. It is intended as a replacement for the prior Nemotron-Math-Proofs-v2 SFT dataset.

License/Terms of Use:

This dataset is governed by the Creative Commons Attribution 4.0 International License (CC BY 4.0).

Intended Usage:

This dataset is intended for:

• Training LLMs to perform structured mathematical reasoning and proof generation.
• Training LLMs to revise proofs using prior proof attempts and verifier feedback.
• Training LLMs to generate proof verification traces and identify gaps in mathematical arguments.
• Training LLMs to assess proof self-evaluations and verifier outputs.
• Building long-context or multi-trajectory reasoning systems for theorem proving.
• Research on proof validity, verifier accuracy, error patterns, and self-verifying mathematical reasoning.

Dataset Characterization

Dataset Composition and Generation

Problem Sourcing

Problems are sourced from nvidia/Nemotron-Math-Proofs-v1, which collected proof-based problems from the AoPS community. A prior pass-rate evaluation is used to select 15,879 relatively hard problems for the generation pipeline.

Multi-Round Trace Generation

DeepSeek-V4-Pro in Max inference mode is used for all four task types. The pipeline follows DeepSeekMath-V2-style proof generation and verification:

• Proof generation: Round 1 generates multiple initial proof attempts for each problem.
• Proof refinement: Rounds 2 through 4 use previous attempts and verifier feedback to produce revised proofs for problems that are not yet strictly solved.
• Verification: Multiple verifier trajectories evaluate candidate proofs and assign a final score of 0, 0.5, or 1.
• Meta-verification: Meta-verifier trajectories assess either a candidate's self-evaluation or a representative verifier evaluation.

Generation uses temperature 1.0 and top-p 0.95. The stage-specific generation limits are 400,000 tokens for initial proofs, 350,000 for refinements, 64,000 for verification, and 96,000 for meta-verification.

Quality Filtering and Balancing

Strict post-processing removes incomplete or length-capped generations, empty or malformed responses, rows that use reasoning fallback, and rows that expose <think> markup in visible answer text. Proof and refinement samples must contain non-empty visible Solution and Self Evaluation sections. Verification and meta-verification samples must contain a parseable final score and use the visible final proof rather than a scratchpad as input.

After final quality filtering, the verification subset contains 117,622 score-0 traces and 118,738 score-0.5 or score-1 traces.

Dataset Fields

The released dataset contains the following fields:

• uuid: Unique identifier for the sample.
• messages: Standardized user/assistant message sequence for LLM training. Assistant messages contain visible content and the corresponding long-form reasoning_content.
• tools: Tool definition list, if any.
• license: Per-sample license label. The released samples use cc-by-4.0.
• metadata: Task metadata, including problem, problem_idx, data_type, recorded input and generated token counts, and optional round or stage information.
• source: Source label for the seed problem. The released samples use AoPS.
• dataset: Dataset/release label. The released samples use Nemotron-Math-Proofs-v3-SFT.
• subset: Compatibility subset label. Use metadata.data_type, not this field, to identify the task type.
• filter_reason, processing_info, match_contexts, and matched_categories: Post-processing and compatibility fields retained by the release pipeline.

The authoritative task type is metadata.data_type, with one of the following values: proof, refinement, verification, or meta_verification.

Data Collection Method

• Hybrid: Manually-Collected, Synthetic, Automated

Labeling Method

• Hybrid: Manually-Labeled, Synthetic, Automated

Dataset Format

• Modality: Text
• Format: JSONL
• Structure: JSONL records with natural-language proof problems and generated proof, refinement, verification, or meta-verification traces.
• Files: The train split is stored as 10 JSONL shards, data/train-00000-of-00010.jsonl through data/train-00009-of-00010.jsonl. Concatenating the shards in order reproduces the original single file byte for byte; load_dataset reads them as one split.

Dataset Quantification

Subset Samples:

Source Pool: 15,879 problems

Unique Problems Represented After Filtering: 15,818

Total Disk Size: 121.95 GiB (130,940,075,840 bytes)

Recorded Generated Token Count: 37,800,710,874

Estimated Input Token Count: 1,246,480,841

Total Recorded Token Count: 39,047,191,715

Generated token counts are taken from the inference metadata. Input token counts are pipeline estimates based on prompt text length rather than a tokenizer re-count.

Reference(s):

• An Open Recipe for IMO Gold: Training Nemotron for Olympiad Mathematics (technical report)
• DeepSeek-V4-Pro
• nvidia/Nemotron-Math-Proofs-v1
• nvidia/Nemotron-Math-Proofs-v2
• DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
• Nemotron-Math paper
• NeMo-Skills
• Nemotron-MathProofs-v1 documentation

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal developer teams to ensure this dataset meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here.