RefAlign: LLM Alignment Dataset

This dataset is used in the paper Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data.

Code: https://github.com/mzhaoshuai/RefAlign

This dataset is modified from https://huggingface.co/datasets/princeton-nlp/llama3-ultrafeedback. We use the BERTScore to choose the chosen and rejected responses.

Item with key ['Llama3.3-70B-Inst-Awq'] is the reference answers generated by https://huggingface.co/casperhansen/llama-3.3-70b-instruct-awq.

Item with key ['bertscore_R_scores'] is the Recall score from BERTScore.

Item with key ['bertscore_P_scores'] is the Precision score from BERTScore.

We select the chosen and rejected responses with the Recall score.

The test split is meaningless; it is a copy of 1K samples from the train set. It is for code compatibility purpose.