This dataset comprises concatenated long conversation data, each entry approximately 16k tokens, sourced from AlpacaGPT-4 and ShareGPT. We use Vicuna's scripts to pre-process the ShareGPT data and retain only very lengthy conversations. We also keep all the samples whose responses are generated by GPT-4. Fine-tuning Llama 2 on this dataset can often yield performance comparable to that of Vicuna v1.5 with a 16k context length.

Usage

from datasets import load_dataset

dataset = load_dataset("L4NLP/chunkllama-long-dialog", split="train")

print(dataset[0].keys()) # dict_keys(['inputs' (user queries), 'outputs' (chatbot responses), 'length' (conversation length), 'source']