
GenQA consists of over 10M cleaned and deduplicated instruction samples generated from a handful of carefully designed prompts. Our experiments indicate that prompt engineering alone can yield millions of diverse training samples with quality as good as (or in some cases surpassing) high-cost human labelling. To the best of our knowledge, this is the largest finetuning dataset written autonomously by an LLM without conditioning on human questions or using complex multi-stage pipelines. This dataset is also valuable to combine with other existing datasets, or to quickly generate synthetic data to cover blindspots of existing data sources.
Each split can be automatically downloaded and loaded for browsing, processing, or training in a few lines of code using the datasets library.
Here is an example of downloading the dataset and then selecting the "academic" split. Optionally, pass the split argument to load_dataset directly.
>>> from datasets import load_dataset
>>> ds = load_dataset("tomg-group-umd/GenQA")
>>> ds
DatasetDict({
code: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 513483
})
dialog: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 819154
})
general: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 304920
})
math: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 515509
})
mmlu: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 2409841
})
multiple_choice: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 372610
})
writing: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 932362
})
academic: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 4210076
})
task: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 1004179
})
})
>>> ds["academic"]
Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 4210076
})
The dataset contains both single turn and multi-turn conversations in many of the splits. For ease of use, each conversation (contained in the text field) is formatted in the same way as a list of json objects.
This format is chosen for compatibility with the transformers library's built in support for chat templates.
Here is an example from the dialog split:
>>> ds["dialog"][123]["text"]
[{'content': 'What were the Opium Wars?', 'role': 'user'},
{'content': 'The Opium Wars were a series of conflicts in the 19th century between the British Empire and the Qing dynasty of China over the British desire to sell opium to China. The First Opium War (1839-1842) ended with the Treaty of Nanking, which gave the British control over Hong Kong and opened five Chinese ports to foreign trade. The Second Opium War (1856-1860) ended with the Treaty of Tientsin, which gave the British and French further concessions in China, including the right to station troops in Beijing. The Opium Wars had a profound impact on China, weakening the Qing dynasty and leading to the rise of foreign imperialism in the country.',
'role': 'assistant'},
{'content': 'What role did opium play in the wars?', 'role': 'user'},
{'content': "Opium played a central role in the Opium Wars. The British East India Company had been selling opium to China since the late 18th century, and by the early 19th century, opium had become one of China's most valuable imports. However, the Qing government was concerned about the growing opium addiction problem in the country and in 1839, it banned the import of opium. The British government, which was heavily dependent on the opium trade for revenue, refused to accept the ban, and this led to the outbreak of the First Opium War.",
'role': 'assistant'},
{'content': 'How did the wars affect China?', 'role': 'user'},
{'content': 'The Opium Wars had a devastating impact on China. The wars weakened the Qing dynasty and led to the rise of foreign imperialism in the country. The Treaty of Nanking gave the British control over Hong Kong and opened five Chinese ports to foreign trade. This led to an influx of foreign goods and ideas into China, which had a profound impact on Chinese society. The wars also led to the loss of Chinese territory, as the British acquired Hong Kong and other territories in Southeast Asia.',
'role': 'assistant'},
{'content': 'What were the long-term consequences of the Opium Wars?',
'role': 'user'},
{'content': 'The Opium Wars had a number of long-term consequences for China. The wars weakened the Qing dynasty and led to the rise of foreign imperialism in the country. The wars also led to the loss of Chinese territory, as the British acquired Hong Kong and other territories in Southeast Asia. The wars also had a significant impact on Chinese society, as they led to an influx of foreign goods and ideas into the country. This led to a number of changes in Chinese society, including the growth of nationalism and the decline of traditional Chinese values.',
'role': 'assistant'}]
The associated manuscript contains a detailed explanation of each split. Each split pertains to a different range of topics. Each instance within a split contains simulated conversations between a human user and an LLM virtual assistant. Many splits contain instances with multiple turns in which the human and machine alternate questions and responses. Each sample contains Unicode formatted text representing a conversation between a user and an assistant. A unified format is used for both single and multi-turn conversations.
The dataset is intended for researchers and individuals in the academic community focused on solving problems related to large language model (LLM) finetuning at scale. We hope this enables the performance of controlled scientific studies that result in better science and ultimately new techniques to address the existing deficiencies with existing models.
This dataset is suitable for use cases involving open research on industrial-scale finetuning practices, leveraging its scale and diversity to advance knowledge in this area.
The dataset should not be used for applications requiring verified factual accuracy, critical decision-making, or any malicious or unethical activities.
Detailed information on how each split was curated is provided in the manuscript and paper.
Dataset fields:
GenQA was created to demonstrate the effectiveness of autonomously written instruction datasets, and to produce a research instruction dataset of size comparable to commercial instruction sets
The data consists of question and answer pairs simulating conversation between a user and an assistant.
Questions were written by the Gemini Language model and deduplicated in the processing stage. All collection and processing stages were done using Python.
Correspondence to Tom Goldstein (tomg@umd.edu)
N/A.
N/A.
N/A.
N/A.
The dataset may contain factual inaccuracies and was not manually checked for factual correctness.
Users should be aware of the potential factual inaccuracies and exercise caution when using the dataset by verifying critical information and mitigatating any potential biases and errors.
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
14 commits
5 commits

GenQA consists of over 10M cleaned and deduplicated instruction samples generated from a handful of carefully designed prompts. Our experiments indicate that prompt engineering alone can yield millions of diverse training samples with quality as good as (or in some cases surpassing) high-cost human labelling. To the best of our knowledge, this is the largest finetuning dataset written autonomously by an LLM without conditioning on human questions or using complex multi-stage pipelines. This dataset is also valuable to combine with other existing datasets, or to quickly generate synthetic data to cover blindspots of existing data sources.
Each split can be automatically downloaded and loaded for browsing, processing, or training in a few lines of code using the datasets library.
Here is an example of downloading the dataset and then selecting the "academic" split. Optionally, pass the split argument to load_dataset directly.
>>> from datasets import load_dataset
>>> ds = load_dataset("tomg-group-umd/GenQA")
>>> ds
DatasetDict({
code: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 513483
})
dialog: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 819154
})
general: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 304920
})
math: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 515509
})
mmlu: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 2409841
})
multiple_choice: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 372610
})
writing: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 932362
})
academic: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 4210076
})
task: Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 1004179
})
})
>>> ds["academic"]
Dataset({
features: ['text', 'prompt', 'template', 'idx', 'category'],
num_rows: 4210076
})
The dataset contains both single turn and multi-turn conversations in many of the splits. For ease of use, each conversation (contained in the text field) is formatted in the same way as a list of json objects.
This format is chosen for compatibility with the transformers library's built in support for chat templates.
Here is an example from the dialog split:
>>> ds["dialog"][123]["text"]
[{'content': 'What were the Opium Wars?', 'role': 'user'},
{'content': 'The Opium Wars were a series of conflicts in the 19th century between the British Empire and the Qing dynasty of China over the British desire to sell opium to China. The First Opium War (1839-1842) ended with the Treaty of Nanking, which gave the British control over Hong Kong and opened five Chinese ports to foreign trade. The Second Opium War (1856-1860) ended with the Treaty of Tientsin, which gave the British and French further concessions in China, including the right to station troops in Beijing. The Opium Wars had a profound impact on China, weakening the Qing dynasty and leading to the rise of foreign imperialism in the country.',
'role': 'assistant'},
{'content': 'What role did opium play in the wars?', 'role': 'user'},
{'content': "Opium played a central role in the Opium Wars. The British East India Company had been selling opium to China since the late 18th century, and by the early 19th century, opium had become one of China's most valuable imports. However, the Qing government was concerned about the growing opium addiction problem in the country and in 1839, it banned the import of opium. The British government, which was heavily dependent on the opium trade for revenue, refused to accept the ban, and this led to the outbreak of the First Opium War.",
'role': 'assistant'},
{'content': 'How did the wars affect China?', 'role': 'user'},
{'content': 'The Opium Wars had a devastating impact on China. The wars weakened the Qing dynasty and led to the rise of foreign imperialism in the country. The Treaty of Nanking gave the British control over Hong Kong and opened five Chinese ports to foreign trade. This led to an influx of foreign goods and ideas into China, which had a profound impact on Chinese society. The wars also led to the loss of Chinese territory, as the British acquired Hong Kong and other territories in Southeast Asia.',
'role': 'assistant'},
{'content': 'What were the long-term consequences of the Opium Wars?',
'role': 'user'},
{'content': 'The Opium Wars had a number of long-term consequences for China. The wars weakened the Qing dynasty and led to the rise of foreign imperialism in the country. The wars also led to the loss of Chinese territory, as the British acquired Hong Kong and other territories in Southeast Asia. The wars also had a significant impact on Chinese society, as they led to an influx of foreign goods and ideas into the country. This led to a number of changes in Chinese society, including the growth of nationalism and the decline of traditional Chinese values.',
'role': 'assistant'}]
The associated manuscript contains a detailed explanation of each split. Each split pertains to a different range of topics. Each instance within a split contains simulated conversations between a human user and an LLM virtual assistant. Many splits contain instances with multiple turns in which the human and machine alternate questions and responses. Each sample contains Unicode formatted text representing a conversation between a user and an assistant. A unified format is used for both single and multi-turn conversations.
The dataset is intended for researchers and individuals in the academic community focused on solving problems related to large language model (LLM) finetuning at scale. We hope this enables the performance of controlled scientific studies that result in better science and ultimately new techniques to address the existing deficiencies with existing models.
This dataset is suitable for use cases involving open research on industrial-scale finetuning practices, leveraging its scale and diversity to advance knowledge in this area.
The dataset should not be used for applications requiring verified factual accuracy, critical decision-making, or any malicious or unethical activities.
Detailed information on how each split was curated is provided in the manuscript and paper.
Dataset fields:
GenQA was created to demonstrate the effectiveness of autonomously written instruction datasets, and to produce a research instruction dataset of size comparable to commercial instruction sets
The data consists of question and answer pairs simulating conversation between a user and an assistant.
Questions were written by the Gemini Language model and deduplicated in the processing stage. All collection and processing stages were done using Python.
Correspondence to Tom Goldstein (tomg@umd.edu)
N/A.
N/A.
N/A.
N/A.
The dataset may contain factual inaccuracies and was not manually checked for factual correctness.
Users should be aware of the potential factual inaccuracies and exercise caution when using the dataset by verifying critical information and mitigatating any potential biases and errors.
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
14 commits
5 commits