Dataset Card for Orca DPO Pair

Dataset Description

This is a pre-processed version of the OpenOrca dataset.

The original OpenOrca dataset is a collection of augmented FLAN data that aligns, as best as possible, with the distributions outlined in the Orca paper. It has been instrumental in generating high-performing preference-tuned model checkpoints and serves as a valuable resource for all NLP researchers and developers!

Dataset Summary

The OrcaDPO Pair dataset is a subset of the OpenOrca dataset suitable for DPO preference tuning. The dataset is stored in parquet format with each entry using the following schema: :

{
'prompt': 'Read the following paragraph and determine if the hypothesis is true:\n\nWorld leaders expressed concern on Thursday that North Ko...'
'chosen': [
           {'content': 'You are a helpful assistant, who always provide explanation. Think like you are answering to a five year old.',
            'role': 'system'
           },
           {'content': 'Read the following paragraph and determine if the hypothesis is true...',
            'role': 'user'
           },
           {'content': 'Okay little buddy, let\'s look at this...',
            'role': 'assistant'
           }
          ],
'rejected': [
             {'content': 'You are a helpful assistant, who always provide explanation. Think like you are answering to a five year old.',
              'role': 'system'
             },
             {'content': 'Read the following paragraph and determine if the hypothesis is true...',
              'role': 'user'
             },
             {'content': 'Oh my gosh! Let me see if I can help you with that! ...',
              'role': 'assistant'
             }
            ],
}

Data Splits

The dataset consists of two splits, "train_prefs" and "test_prefs":

Usage

To load the dataset, run:

from datasets import load_dataset

ds = load_dataset("HuggingFaceH4/orca_dpo_pairs")

Languages

The language of the data is primarily English.

Dataset Creation

Curation Rationale

The dataset was created to provide a source of augmented text data for researchers and developers. The datapoints are intended primarily to provide an enhancement of the core FLAN Collection data which relies upon the detailed step-by-step reasoning capabilities of GPT-3.5 and GPT-4. This "reasoning trace" augmentation has demonstrated exceptional results, allowing a LLaMA-13B model trained with this data to rival or beat GPT-3.5 on broad sets of hard reasoning tasks which all models below 100B parameters had previously performed dramatically worse on.

Source Data

The data is generated using techniques in alignment with the distributions outlined in the Orca paper, except as noted below:

1. There is not enough CoT data in the FLAN Collection to generate 150K zero-shot entries, as the paper purports to use. We suspect this portion was either undocumented or misrepresented. We have used the ~75K points available.
2. We used the pre-generated FLAN Collection datasets hosted on HuggingFace under conceptofmind, e.g. conceptofmind/flan2021. These are referenced by the official FLAN Collection repo as the preferred data source. However, these are a subset of the full FLAN Collection data, and have less than the required entries for the flan2021 and t0 submixes, by ~1.25M and 200k respectively.

Combined, this gave us ~1.5M fewer data points than in the original Orca paper. Completing the set is an ongoing work.

Dataset Use

Use Cases

The dataset can be used for tasks related to language understanding, natural language processing, machine learning model training, and model performance evaluation.

Usage Caveats

Given that this is a work-in-progress dataset, it is recommended to regularly check for updates and improvements. Further, the data should be used following the guidelines and recommendations outlined in the Orca paper.

Getting Started

This dataset is organized to be naively loaded via the Hugging Face datasets library. We recommend using streaming due to the large size of the files. Regular updates and data generation progress can be monitored through the OpenOrca repository on Hugging Face.

Citation

@misc{OpenOrca,
  title = {OpenOrca: An Open Dataset of GPT Augmented FLAN Reasoning Traces},
  author = {Wing Lian and Bleys Goodson and Eugene Pentland and Austin Cook and Chanvichet Vong and "Teknium"},
  year = {2023},
  publisher = {HuggingFace},
  journal = {HuggingFace repository},
  howpublished = {\url{https://https://huggingface.co/Open-Orca/OpenOrca}},
}