OpenCoder Dataset

The OpenCoder dataset is composed of the following datasets:

• opc-sft-stage1: the sft data used for opencoder sft-stage1
• opc-sft-stage2: the sft data used for opencoder sft-stage2
• opc-annealing-corpus: the synthetic data & algorithmic corpus used for opencoder annealing <-- you are here
• fineweb-code-corpus: the code-related page recalled from fineweb
• fineweb-math-corpus: the math-related page recalled from fineweb
• refineCode-code-corpus-meta: the meta-data of RefineCode

Detailed information about the data can be found in our paper.

opc-annealing-corpus summary

This corpus is an additional component incorporated into OpenCoder during the annealing phase, beyond the original distribution:

• algorithmic_corpus: Algorithm-related code sampled from The Stack v2.
• synthetic_code_snippet: High-quality code snippets generated by rewriting algorithmic_corpus as seeds.
• synthetic_qa: High-quality Q&A pairs generated by adapting algorithmic_corpus as seeds.

Our ablation experiments validated the effectiveness of this batch of synthetic data.

Citation Information

Please consider citing our paper if you find this dataset useful:

@inproceedings{Huang2024OpenCoderTO,
  title = {OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models},
  author = {Siming Huang and Tianhao Cheng and Jason Klein Liu and Jiaran Hao and Liuyihan Song and Yang Xu and J. Yang and J. H. Liu and Chenchen Zhang and Linzheng Chai and Ruifeng Yuan and Zhaoxiang Zhang and Jie Fu and Qian Liu and Ge Zhang and Zili Wang and Yuan Qi and Yinghui Xu and Wei Chu},
  year = {2024},
  url = {https://arxiv.org/pdf/2411.04905}
}