Dataset Card for OffsetBias

Dataset Description:

• 💻 Repository: https://github.com/ncsoft/offsetbias
• 📜 Paper: OffsetBias: Leveraging Debiased Data for Tuning Evaluators

Dataset Summary

OffsetBias is a pairwise preference dataset intended to reduce common biases inherent in judge models (language models specialized in evaluation). The dataset is introduced in paper OffsetBias: Leveraging Debiased Data for Tuning Evaluators. OffsetBias contains 8,504 samples for training judge (or reward) models.

Dataset Structure

Data Instance

{
    "instruction": "Rewrite the following sentence, paying attention to spelling and grammar. Distance learning is harder then in-person classes", 
    "output_1": "Distance learning is harder than in-person classes.", 
    "output_2": "The original sentence is grammatically correct and the punctuation is also appropriate. However, if you wish to slightly rephrase it for clarity or emphasis, one possibility is:\n\n\"Studying online can be more challenging than in traditional classroom settings.\" \n\nThis slight tweak emphasizes the comparison between \"studying online\" and \"studying... in traditional classroom settings.\"", 
    "label": 1
}

Data Fields

• instruction: a string, instruction that responses need to answer
• output_1: a string, first candidate response to the instruction
• output_2: a string, second candidate response to the instruction
• label: an integer of 1 or 2, indicating the better response number

Dataset Construction

The instructions are taken from source datasets including Alpaca, Ultrachat, Evol-Instruct and Flan. The outputs are created using GPT-3.5, GPT-4 and Claude-3-Opus models. See paper for details about data construction methods.

Considerations for Using the Data

This dataset is intended to be a collection of counter-examples to existing common biases in evaluation models. Thus using this dataset alone for training is not suitable for training an evaluation model, as it would introduce another bias. It is best used as an additional dataset that can complement other general-purpose evaluation datasets.

Citation

BibTeX:

@misc{park2024offsetbias,
      title={OffsetBias: Leveraging Debiased Data for Tuning Evaluators},
      author={Junsoo Park and Seungyeon Jwa and Meiying Ren and Daeyoung Kim and Sanghyuk Choi},
      year={2024},
      eprint={2407.06551},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}