README
Fair RLHF
We propose two optimizations for improving fairness in RLHF -- an exploratory intrinsic reward in the PPO training of LLMs (explored in past papers), and an info-theoretic constraint enforcing invariance of the reward model distribution with respect to different categories of bias (our novelty)
To run our fair RLHF reward model training pipeline,
./train_rm.sh
Contributors
10 commits
Similar repos
Languages
Python
97.9%
Shell
2.1%
README
Fair RLHF
We propose two optimizations for improving fairness in RLHF -- an exploratory intrinsic reward in the PPO training of LLMs (explored in past papers), and an info-theoretic constraint enforcing invariance of the reward model distribution with respect to different categories of bias (our novelty)
To run our fair RLHF reward model training pipeline,
./train_rm.sh
Similar repos
Contributors
10 commits
Languages
Python
97.9%
Shell
2.1%