Llama-3.3-70B-Inst-awq Responses for RefAlign Safety Alignment

This dataset contains responses generated for the paper Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data, which introduces the RefAlign alignment algorithm.

Code Repository: https://github.com/mzhaoshuai/RefAlign

This dataset specifically consists of responses generated by the casperhansen/llama-3.3-70b-instruct-awq model, given the prompts from the PKU-Alignment/PKU-SafeRLHF dataset.

During generation, we use three random seeds (13, 21) to generate two responses and use the model itself to choose the helpful/harmless response.

This results in the saferlhf.json file, available at: https://huggingface.co/datasets/mzhaoshuai/Llama-3.3-70B-Inst-awq_SafeRLHF/blob/main/saferlhf.json.

Citation

If you use this dataset, please cite the original paper:

@article{zhao2025learning,
  title={Learning from reference answers: Versatile language model alignment without binary human preference data},
  author={Zhao, Shuai and Xu, Yunqiu and Zhu, Linchao and Yang, Yi},
  journal={arXiv preprint arXiv:2504.09895},
  year={2025}
}