Post-OCR Correction of Historical German Periodicals using LLMs

This repository contains the scripts and evaluation results for the paper "Post-OCR Correction of Historical German Periodicals using LLMs" (Danilova & Aangenendt, 2025), presented at RESOURCEFUL, NoDaLiDa-2025.

Main Files

• prepare_dataset.ipynb: Prepares the dataset for post-OCR correction. It downloads the data, preprocesses it, aligns the OCR text with the ground truth text, calculates the character error rate (CER), prepares train-test split
• llama-2.py: Fine-tunes the Llama 2 model for OCR correction using the prepared dataset (re-used).
• bart.py: Fine-tunes the BART model for OCR correction using the prepared dataset (re-used).
• inference.ipynb: Evaluates the performance of the models and calculates the CER reduction.
• analysis.ipynb: Analyzes the inference results and calculates various metrics.

Data Availability

The training dataset is available upon request. Please contact the author of this repository.

License

This project is licensed under the terms of the MIT license.

If you use the evaluation data from this repository to evaluate your own models, please cite our paper referenced below.

Reference

Danilova, V., & Aangenendt, G. (2025). Post-OCR Correction of Historical German Periodicals using LLMs. In Špela Arhar Holdt, Nikolai Ilinykh, Barbara Scalvini, Micaella Bruton, Iben Nyholm Debess, & Crina Madalina Tudor (Eds.), Proceedings of the Third Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2025) (pp. 120–129). Tallinn, Estonia: University of Tartu Library. https://aclanthology.org/2025.resourceful-1.26/ (ISBN: 978-9908-53-121-2)