SEED-GRPO

The official repository of SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy Optimization.

We would like to sincerely thank the authors of Dr.GRPO for their excellent work, which greatly inspired and facilitated our implementation of SEED-GRPO.

We sincerely apologize for the delay in releasing this code due to graduation-related affairs. Nevertheless, SEED-GRPO continues to demonstrate strong performance.

🌟 Main Results

---

🧩 Framework Overview

---

Usage

We strongly recommend using Python 3.10.
Python 3.11 or above may cause issues when installing launchpad.

The installation process is identical to that of Dr.GRPO.

⚙️ Installation We strongly recommend using Python 3.10; otherwise Python 3.11 or above may cause issues when installing launchpad.

Training:

bash run_debug.sh

Evaluation:

 bash eval.sh

All experiments were conducted on 8× NVIDIA A800 GPUs. Training typically reaches 1000 steps within 3–4 days under default hyperparameters. If you are using H-series GPUs, expect roughly 2× faster convergence.

About Semantic Uncertainty

For mathematical problems, due to the varying lengths of the sampled answers, using an entailment model (e.g., robertav2) for meaning clustering is not feasible. Moreover, relying on commercial model APIs for answer clustering is both slow and costly.

Therefore, for math questions, we simply consider the final answer: if two responses produce the same answer, we cluster them together. While this approach works in practice, it ignores the reasoning process behind the answers. We welcome any better ideas or suggestions for handling this issue.

If you find this repo useful for your research, feel free to give us a star ⭐ and cite our paper:

@article{chen2025seed,
  title={Seed-grpo: Semantic entropy enhanced grpo for uncertainty-aware policy optimization},
  author={Chen, Minghan and Chen, Guikun and Wang, Wenguan and Yang, Yi},
  journal={arXiv preprint arXiv:2505.12346},
  year={2025}
}