Gutenberg DPO

Overview

This is a dataset meant to enhance novel writing capabilities of LLMs, by using public domain books from Project Gutenberg

Process

First, the each book is parsed, split into chapters, cleaned up from the original format (remove superfluous newlines, illustration tags, etc.).

Once we have chapters, an LLM is prompted with each chapter to create a synthetic prompt that would result in that chapter being written. Each chapter has a summary created as well, so that the prompts for each chapter after the also include a summary of the previous chapter to provide additional context.

We then use the synthetic prompt with previous chapter summary to write the chapter with an LLM (llama-2-13b-chat, bagel-7b-v0.1, dolphin-2.2-34b). The human written text, that is, the original chapter, is used as the "chosen" value, and the LLM written chapter is used as the rejected value.

Books used

These books were chosen main because they appeared in the popular section on project gutenberg, and they function correctly with the chapterize library.

• Huckleberry Finn
• Treasure Island
• Anna Karenina
• Uncle Tom’s Cabin
• Wuthering Heights
• Madame Bovary
• The Turn of the Screw
• The War of the Worlds
• A Study in Scarlet
• Middlemarch
• Pride and Prejudice
• The Brothers Karamazov
• Through the Looking Glass
• Moby Dick
• Frankenstein
• A Tale of Two Cities