MajinBook

This document outlines the structure and metadata schemas of the datasets released with MajinBook: An open catalogue of digitally mediated world literature published in Journal of Cultural Analytics by Antoine Mazières and Thierry Poibeau.

Abstract: This data paper introduces MajinBook, an open catalogue designed to facilitate the use of shadow libraries, such as Library Genesis and Z-Library, for computational social science and cultural analytics. By linking metadata from these vast, crowd-sourced archives with structured bibliographic data from Goodreads, we create a high-precision corpus of over 539,000 references to digitally mediated English-language books. Spanning three centuries and reflecting a contemporary selection bias, these entries are enriched with first publication dates, genres, and popularity metrics such as ratings and reviews. Our methodology prioritises natively digital EPUB files to ensure machine-readable quality, while addressing biases in traditional corpora such as HathiTrust, and includes secondary datasets for French-, German-, and Spanish-language works. We evaluate the linkage strategy for accuracy, release all underlying data openly, and discuss the project’s legal permissibility under EU and U.S. frameworks for text and data mining in research.

The data is available on Zenodo and HuggingFace.

The paper is available on the website of the Journal of Cultural Analytics.

All files are in the JSON Lines text file format.

1. The MajinBook's Catalogue

This section describes the primary high-precision English catalogue introduced in the paper. The secondary datasets in French, German and Spanish follow the same model.

Files

• Primary dataset: majinbook_eng.jsonl (English) — 539,530 lines.
• Secondary datasets:
  • majinbook_fra.jsonl (French) — 47,960 lines.
  • majinbook_deu.jsonl (German) — 35,559 lines.
  • majinbook_spa.jsonl (Spanish) — 30,169 lines.

JSON Record Example

{
  "first_pub_year": 1913,
  "authors": [
    [
      233619,
      "Marcel Proust"
    ]
  ],
  "genres": [
    "Classics",
    "Literature",
    "Philosophy",
    "20th Century",
    "Novels",
    "Fiction",
    "Literary Fiction",
    "Classic Literature",
    "France",
    "French Literature"
  ],
  "n_reviews": 356,
  "n_ratings": 3635,
  "rating": 4.28,
  "title": "Remembrance of Things Past: Volume I - Swann's Way & Within a Budding Grove",
  "work_id": 45683795,
  "zlibrary_ids": [
    11588490
  ],
  "libgen_ids": null
}

Schema Description

Notes:

1. Coverage is for the primary dataset only (English). Coverage varies for secondary datasets regarding genres and n_reviews, see the paper.
2. Goodreads Author ID (int) corresponds to the ID of the Author found in the URL of their profile, as in goodreads.com/author/show/233619
3. Goodreads Work ID, as in goodreads.com/work/editions/45683795
4. zlibrary_ids and libgen_ids cannot both be null.

2. Underlying datasets

This section describes the metadata datasets collected and processed to construct the MajinBook catalogue.

2.1 Shadow Library metadata

Z-Library

File: zlibrary.jsonl — 8,097,488 lines

JSON Record Example

{
  "id": 11588490,
  "title": "Remembrance of Things Past, Volume I",
  "authors": "Marcel Proust",
  "pub_year": 2011,
  "lang_iso": "eng",
  "publishers": "Knopf Doubleday Publishing Group",
  "isbns": [
    "9780307808554"
  ]
}

Schema Description

Library Genesis (LibGen)

File: libgen.jsonl — 3,032,730 lines

JSON Record Example

{
  "id": "bc0d6411ddd67b2c2d43b39bc8472cfc",
  "collection": "fiction",
  "title": "Alice's Adventures in Wonderland and What the Tortoise Said to Achilles and Other Riddles",
  "lang_iso": "eng",
  "authors": [
    "Lewis Carroll"
  ],
  "pub_year": 2012,
  "publisher": "West Margin Press",
  "isbns": [
    "9780882408712"
  ],
  "asin": "B007HOO22Y"
}

Schema Description

Goodreads

Goodreads Works

File: goodreads_works.jsonl — 4,778,124 lines

JSON Record Example

{
  "id": 55548884,
  "suggestions": [
    2998,
    5326,
    5659,
    ...
  ],
  "first_pub_year": 1865,
  "n_ratings": 418248,
  "rating": 3.99,
  "main_authors": [
    8164
  ]
}

Schema Description

Goodreads Editions

File: goodreads_editions.jsonl — 28,105,913 lines

JSON Record Example

{
  "id": 60671823,
  "work_id": 55548884,
  "title": "Alice's Adventures in Wonderland (Hardcover)",
  "authors": [
    8164,
    59749
  ],
  "pub_year": null,
  "lang": "English",
  "lang_iso": "eng",
  "publisher": "Pan Macmillan",
  "isbn": "9781529002461",
  "asin": null,
  "rating": 4.08,
  "n_ratings": 71699
}

Schema Description

Goodreads Authors

File: goodreads_authors.jsonl — 2,150,522 lines

JSON Record Example

{
  "id": 8164,
  "full_name": "Lewis Carroll",
  "birth_place": "Daresbury, Cheshire, England, The United Kingdom",
  "birth_year": 1832,
  "death_year": 1898,
  "works": [
    87631877,
    164806665,
    155893772,
    ...
  ]
}

Schema Description

Minhash signatures

File: minhash_signatures.jsonl — 11,130,005 lines

Example

{
  "source": "libgen",
  "id": "0af71d22806865ed567bc0fe2bc9be2a",
  "minhash_signature": [
    87053,
    62778,
    73327,
    ...
  ]
}

Fields