RefAlign: RL with Similarity-based Rewards

GitHub repository: https://github.com/mzhaoshuai/RefAlign

Paper: Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data.

This is the model aligned with RefAlign described in the paper Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data.

It is primarily aligned for safety.

The training data is https://huggingface.co/datasets/mzhaoshuai/Llama-3.3-70B-Inst-awq_SafeRLHF.

For the project code, please refer to the GitHub repository.

When conducting Reinforcement Learning with Similarity-based Rewards, the reward function is Meteor.