Bespoke-Stratos-17k

We replicated and improved the Berkeley Sky-T1 data pipeline using SFT distillation data from DeepSeek-R1 to create Bespoke-Stratos-17k -- a reasoning dataset of questions, reasoning traces, and answers.

This data was used to train:

1. Bespoke-Stratos-32B, a 32B reasoning model which is a fine-tune of Qwen-2.5-32B-Instruct
2. Bespoke-Stratos-7B, a 7B reasoning model which is a fine-tune of Qwen-2.5-7B-Instruct.

Metrics for Bespoke-Stratos-32B

Metrics for Bespoke-Stratos-7B

Details

The code for curating the data is here. Please also refer to Sky-T1’s codebase for the training and evaluation code.

Similarly to Sky-T1_data_17k, this dataset contains 5k coding data from APPs and TACO, and 10k math data from AIME, MATH, and Olympiads subsets of the NuminaMATH dataset, and 1k science and puzzle data from STILL-2. Note that the exact problems included may differ due to the rejection sampling process.

We used Bespoke Curator to create the synthetic reasoning dataset. We ported the Sky-T1 data pipeline into Curator, which helped us generate the reasoning dataset within 1.5 hours with DeepSeek-R1 at a cost of $800 without hiccups.

Rejection sampling involves filtering out reasoning traces with incorrect solutions. This is challenging for code verification, which we speed up using a Ray cluster. We are currently integrating code execution verifier directly in Curator, so stay tuned.

We followed the same recipe as the Sky-T1, but with the following differences:

• We used DeepSeek-R1 as the teacher reasoning model instead of QwQ.
• The Sky-T1 recipe used gpt-4o-mini to reformat QwQ’s traces, whereas we did not reformat DeepSeek-R1’s. We found that DeepSeek-R1’s reasoning traces were sufficiently well-formatted and coherent for parsing and finetuning even without an intermediate reformatting step.
• We used gpt-4o-mini instead of Sky-T1’s parsing logic to filter out incorrect math solutions. Using gpt-4o-mini allowed us to reduce the number of false negatives, increasing the number of retained correct solutions from 25% to 73%.

Citation

@misc{bespoke_stratos,  
    author = {Bespoke Labs},  
    title = {Bespoke-Stratos: The unreasonable effectiveness of reasoning distillation},  
    howpublished = {https://www.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation},  
    note = {Accessed: 2025-01-22},  
    year = {2025}
}

Acknowledgement

We are standing on the shoulders of giants. Bespoke Labs would like to thank Berkeley Sky Computing Lab for their work on Sky-T1 and for releasing the code and data, Deepseek for releasing the Deepseek-R1 model, and the Datacomp community for insightful discussions.

To be in the loop, please sign up to be notified at https://bespokelabs.ai/newsletter