Kun: Answer Polishment Saves Your Time for Using Intruction Backtranslation on Self-Alignment

Table of Contents

• Overview
• Dataset Description
• Usage
• Citation
• Acknowledgments

Overview

The COIG-Kun dataset, part of the COIG-Kun GitHub project, consists of instructional data used for training language models. This dataset was developed following the methodology inspired by Meta's "Self-Alignment with Instruction Backtranslation" and adapted for optimal performance in training label, point, and answer models.

Dataset Description

Language

• The dataset contains instructions primarily in Chinese.

Dataset Structure

• Data Instances: Each data instance is structured in a JSON format with two fields: instruction and output.
  • Example: {"instruction": "如何评价祁又一自编自导的电影《鸽子小姐》？", "output": "《鸽子小姐》是一部由祁又一自编自导的电影。..."}
• Data Split: The dataset is comprised of three subsets:
  • wudao.jsonl: 139,852 instances
  • wanjuan.jsonl: 328,294 instances
  • skypile.jsonl: 71,560 instances

Data Characteristics

• The dataset is designed to provide high-quality instructional data for language model training, focusing on enhancing the quality and applicability of the data.

Methodology

Our approach closely follows the self-alignment method ådescribed by Meta, with adaptations to optimize the process:

1. Seed Data Selection and Model Training: Initially, appropriate seed data are selected and inverted to train a Label Model on a base model(Yi Base). Concurrently, using the same seed data, a Primary Chat model is trained following the Supervised Fine-Tuning (SFT) method typical of chat models.

2. Labeling Unlabeled Data: The Label Model is then used to annotate preliminarily cleansed Primary data. Cleansing involves filtering based on perplexity (ppl) and length, discarding data exceeding 512 tokens.

3. Instruction Data Generation: Post-annotation, we obtain our first version of Labeled data. Unlike the original project where both instruction and output data pairs are fed into Primary Chat Model for scoring, our replication revealed limitations in Primary Chat's ability to discern high-quality instructions. We innovated by scoring only the instruction component, effectively filtering out noise and selecting high-quality instructions.

4. Output Data Refinement: Upon manual inspection, we identified a mismatch between the Primary Data (used as output) and the standard requirements for output in instruction data. To address this, we introduced an additional step: refining the output data. Using Primary Chat's capabilities, the output (originally unlabeled data) is adjusted according to the instructions, making it more suitable as output for the instruction data.

5. Framework Completion: Our methodology concludes with the acquisition of a substantial volume of instructional data, achieved with minimal resource expenditure.

Usage

Using the Data

• The dataset can be used for training and fine-tuning language models, specifically focusing on instruction understanding and response generation.
• Users are encouraged to refer to the project documentation for detailed instructions on utilizing the dataset in the training process.

Citation

If you use this dataset in your research, please cite it as follows:

@misc{COIG-Kun,
  title={Kun: Answer Polishment Saves Your Time for Using Intruction Backtranslation on Self-Alignment},
  author={Tianyu, Zheng* and Shuyue, Guo* and Xingwei, Qu and Xinrun, Du and Wenhu, Chen and Jie, Fu and Wenhao, Huang and Ge, Zhang},
  year={2023},
  publisher={GitHub},
  journal={GitHub repository},
  howpublished={https://github.com/Zheng0428/COIG-Kun}
}

Acknowledgments

This dataset was created by a dedicated team at [M-A-P]. We acknowledge the contributions of all individuals and organizations that made this project possible.