Gacha is a corpus of 100 texts from Project Gutenberg, collected for the purpose of evaluating chunking algorithms.

It's heavily inspired by the work of LumberChunker. The paper "LumberChunker: Long-Form Narrative Document Segmentation" released a dataset named "GutenQA" which has the same 100 texts as this corpus. However, because the GutenQA dataset does not contain the original texts, it is hard to extend it to newer methods of chunking as well as newer models as they come out. Chunking is an active area of research and new methods are being developed all the time.

We wished to evaluate the performance of chunking algorithms and having a dataset that contains the original texts is crucial for this. To this end, we spent significant time and effort on creating this dataset over the course of a few weeks. The exact books were extracted from the Gutenberg Project, manually cleaned for any formatting issues, metadata extracted via LLMs (authors, titles, publication dates, etc.) and the questions from the GutenQA corpus were aligned to fit the new texts. Given the amount of time and work that went into creating this corpus, we decided to share this corpus with the community as Gacha.

💡 Info: The Gacha dataset is currently licensed under the CC-BY-NC-SA-4.0 license, but we plan to release it under a more permissive license in the future.

Usage

You can load up the corpus and questions via Huggingface's datasets library and use them as you see fit.

from datasets import load_dataset

corpus = load_dataset("chonkie-ai/gacha", "corpus", split="train")
questions = load_dataset("chonkie-ai/gacha", "questions", split="train")

Splits and Structure

The dataset is split into 3 parts:

• raw: The raw texts from the Gutenberg Project.
• corpus: The corpus of 100 texts.
• questions: The questions from the GutenQA corpus.

Raw

The raw config contains the following features:

• text: The raw text from the Gutenberg Project.

Here's an example from the raw config:

{
  "text": "The Project Gutenberg eBook of A Christmas Carol in Prose; Being a Ghost Story of Christmas..."
}

Corpus

The corpus config contains the following features:

• author: The author of the book.
• title: The title of the book.
• date: The release date of the book in DD-MM-YYYY format.
• text: The text of the book cleaned up for any formatting issues.

Here's an example of the corpus split:

{
  "author" : "Charles Dickens",
  "title" : "A Christmas Carol",
  "date" : "11-08-2004",
  "text" : "I HAVE ordained..."
}

Questions

The questions config contains the following features:

• title: The title of the book.
• author: The author of the book.
• date: The release date of the book in DD-MM-YYYY format.
• question: The question from the GutenQA corpus.
• answer: The answer to the question from the GutenQA corpus.

Here's an example of the questions split:

{
  "title": "A Christmas Carol",
  "author": "Charles Dickens",
  "date": "11-08-2004",
  "question": "What is the main character's name?",
  "answer": "Ebenezer Scrooge",
  "chunk-must-contain": "Ebenezer Scrooge"
}

Misc.

A couple of Misc. details about the corpus.

List of the 100 Books in the Corpus

The corpus contains 100 english texts from Project Gutenberg.

Author	Year	Title
Mark Twain	2004	Adventures of Huckleberry Finn
Lewis Carroll	2008	Alice's Adventures in Wonderland
L. Tolstoy	2010	Anna Karenina
L. M. Montgomery	2008	Anne of Green Gables
Jules Verne	1994	Around the World in Eighty Days
Charles Dickens	2004	A Christmas Carol
Mark Twain	2004	A Connecticut Yankee in King Arthur's Court
W. H. Hudson	2005	A Crystal Age
Walter Scott	2006	A Legend of Montrose
James Joyce	2003	A Portrait of the Artist as a Young Man
E. M. Forster	2001	A Room with a View
Arthur Conan Doyle	1995	A Study in Scarlet
Charles Dickens	1994	A Tale of Two Cities
F. W. Nietzsche	2003	Beyond Good and Evil
Joseph Sheridan Le Fanu	2003	Carmilla
E. C. Gaskell	2018	Cranford
Fyodor Dostoyevsky	2006	Crime and Punishment
H. R. Garis	2005	Daddy Takes Us to the Garden
Charles Dickens	1996	David Copperfield
Bram Stoker	1995	Dracula
James Joyce	2003	Dubliners
Jane Austen	1994	Emma
M. W. Shelly	1993	Frankenstein
Charles Dickens	1998	Great Expectations
Grimm Brothers	2001	Grimm's Fairy Stories
Jonathan Swift	1997	Gulliver's Travels
Joseph Conrad	2006	Heart of Darkness
H. A. Jacobs	2004	Incidents in the Life of a Slave Girl
Charlotte Bronte	1998	Jane Eyre
Victor Hugo	2008	Les Miserables
L. M. Alcott	1996	Little Women
Alexandre Dumas	1995	Louise de la Vallière
Gustave Flaubert	2006	Madame Bovary
Marcus Aurelius	2001	Meditations
George Eliot	1994	Middlemarch
Herman Melville	2001	Moby Dick
Richard Wagner	2004	My Life - Volume 1
Frederick Douglass	2006	Narrative of the Life of Frederick Douglass
Charles Dickens	1996	Oliver Twist
Jane Austen	1994	Persuasion
J. M. Barrie	2008	Peter Pan
Jane Austen	1998	Pride and Prejudice
Max Brand	2006	Riders of the Silences
John Locke	2005	Second Treatise of Government
Jane Austen	1994	Sense and Sensibility
Alexandre Dumas	2001	Ten Years Later
T. Smollett	2004	The Adventures of Ferdinand Count Fathom
T. Smollett	2004	The Adventures of Roderick Random
Arthur C. Doyle	1999	The Adventures of Sherlock Holmes
Mark Twain	2004	The Adventures of Tom Sawyer
Alexadre Dumas	1997	The Black Tulip
L. M. Montgomery	2022	The Blue Castle
Arthur Quiller-Couch	2006	The Blue Pavilions
Fyodor Dostoyevsky	2006	The Brothers Karamazov
Jack London	2008	The Call of the Wild
St. Augustine	2002	The Confessions of St. Augustine
Alexandre Dumas	1998	The Count of Monte Cristo
Elizabeth Von Arnim	2003	The Enchanted April
Maurice Leblanc	2004	The Extraordinary Adventures of Arsène Lupin, Gentleman-Burglar
Fyodor Dostoyevsky	2000	The Gambler
F. S. Fitzgerald	2004	The Great Gatsby
Arthur C. Doyle	2001	The Hound of the Baskervilles
Robert W. Chambers	2005	The King in Yellow
Daniel Defoe	1996	The Life and Adventures of Robinson Crusoe
Alexandre Dumas	1995	The Man in the Iron Mask
Agatha Christie	2022	The Murder of Roger Ackroyd
Agatha Christie	2019	The Murder of the Links
Homer	1999	The Odyssey
Oscar Wilde	1994	The Picture of Dorian Gray
Niccolo Machiavelli	2006	The Prince
Mark Twain	2004	The Prince and the Pauper
Bertrand Russell	2004	The Problems of Philosophy
Khalil Gibran	2002	The Prophet
Jose Rizal	2004	The Reign of Greed
Plato	1998	The Republic
Anonymous	2009	The Romance of Lust
Nathaniel Hawthorne	2004	The Scarlet Letter
Arthur C. Doyle	2000	The Sign of Four
W. E. B. Du Bois	1996	The Souls of Black Folk
R. L. Stevenson	2008	The Strange Case of Dr. Jekyll and Mr. Hyde
Ernest Hemingway	2022	The Sun Also Rises
Alexandre Dumas	1998	The Three Musketeers
H. G. Wells	2004	The Time Machine
Franz Kafka	2005	The Trial
Henry James	1995	The Turn of the Screw
Alexandre Dumas	2001	The Vicomte de Bragelonne
H. G. Wells	2004	The War of the Worlds
L. F. Baum	1993	The Wonderful Wizard of Oz
F. W. Nietzsche	2003	Thus Spoke Zarathustra
R. L. Stevenson	2006	Treasure Island
Jules Verne	1994	Twenty Thousand Leagues under the Sea
Alexandre Dumas	1998	Twenty Years After
James Joyce	2003	Ulysses
H. B. Stowe	2006	Uncle Tom's Cabin
H. D. Thoreau	1995	Walden and On The Duty of Civil Disobedience
Leo Tolstoy	2001	War and Peace
Max Brand	2006	Way of the Lawless
Fyodor Dostoyevsky	2011	White Nights and Other Stories
A. A. Milne	2022	Winnie-the-Pooh
Emily Bronte	1996	Wuthering Heights

A few more books that are available in the extra folder:

Author	Year	Title
Solomon Northup	2014	Twelve Years a Slave
Richard Wagner	2004	My Life - Volume 2

Citation

This corpus is heavily inspired by the work of LumberChunker. If you use this corpus, please cite the following paper:

@misc{duarte2024lumberchunker,
      title={LumberChunker: Long-Form Narrative Document Segmentation}, 
      author={André V. Duarte and João Marques and Miguel Graça and Miguel Freire and Lei Li and Arlindo L. Oliveira},
      year={2024},
      eprint={2406.17526},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2406.17526}, 
}