Code, technical report and findings: GitHub muscgab/manga-ocr-CPT · Weights: Hugging Face · ModelScope
OCR for Japanese manga text boxes. This is mangaOCR after continued pre-training on our training data, with the architecture unchanged (ViT-B/16 encoder, 2-layer BERT decoder, autoregressive, 111M parameters) and a new vocabulary of 6,817 characters. Input: stretched to 224 × 224 (14 × 14 patches), pixel-identical to the original mangaOCR ViTImageProcessor.
The control of the two models: its input is exactly that of the original mangaOCR, so it shows the effect of changing only the data. For general use, manga-ocr-base-CPT-flex is better on elongated and horizontal boxes.
This model comes from an experimental study; see the technical report.
JMangaBench_Read v1.3, 7,789 boxes, RIN v1 normalisation, batch 1, RTX 2080 Ti.
| Model | Score ↑ | CER % ↓ | EM % ↑ | EM printed | EM non-printed |
|---|---|---|---|---|---|
| manga-ocr-base-CPT | 97.43 | 1.77 | 91.00 | 93.82 | 82.32 |
| manga-ocr-base-CPT-flex | 97.45 | 1.53 | 91.42 | 94.16 | 83.00 |
| mangaOCR (original, package defaults) | 90.23 | 4.84 | 78.25 | 88.04 | 48.17 |
RTX 2080 Ti, Read v1.3 speed protocol, preprocessing included; a page (8.7 boxes on average) is one batch.
| Inference path | Per crop (batch 1) | Page p50 | Peak throughput |
|---|---|---|---|
| CUDA path (CUDA graphs + GPU preprocessing) | 10.4 ms | 38.1 ms | 277 crops/s (batch 32) |
| Plain PyTorch path | 27.6 ms | 69.7 ms | 234 crops/s (batch 64) |
| Original manga-ocr package (fp16) | 84.9 ms | 242.4 ms | 42 crops/s (batch 56) |
Both paths give identical outputs on all 7,789 boxes.
The inference code is on GitHub; the weights are downloaded from this repository on first use.
pip install git+https://github.com/muscgab/manga-ocr-CPT
from manga_ocr_cpt import MangaOcrCPT
ocr = MangaOcrCPT("224") # or a local directory holding this repository
ocr("crop.png") # one crop -> str
ocr(["crop1.png", "crop2.png"]) # the crops of one page -> list[str]
Inputs are cropped text boxes (file paths, PIL images or uint8 RGB arrays); there is no text detection. With an NVIDIA GPU the CUDA path is used, otherwise plain PyTorch (CPU, Apple MPS, ...).
kha-white/manga-ocr-base; the 5,133 characters already in the original vocabulary keep their embeddings.| File | Contents |
|---|---|
model.safetensors | Weights, fp32 (444 MB) |
config.json | Architecture and input settings (input.fit selects the input) |
nar_vocabulary_6817.json | Vocabulary (6,817 characters) |
LICENSE_MODEL | CC BY-NC-SA 4.0 |
NOTICE.md | Provenance and acknowledgements |
SHA256SUMS | Checksums |
LICENSE_MODEL); commercial use requires separate permission. The training data AnimeText-OCR is CC BY-NC-SA 4.0.NOTICE.md.Code, technical report and findings: GitHub muscgab/manga-ocr-CPT · Weights: Hugging Face · ModelScope
OCR for Japanese manga text boxes. This is mangaOCR after continued pre-training on our training data, with the architecture unchanged (ViT-B/16 encoder, 2-layer BERT decoder, autoregressive, 111M parameters) and a new vocabulary of 6,817 characters. Input: stretched to 224 × 224 (14 × 14 patches), pixel-identical to the original mangaOCR ViTImageProcessor.
The control of the two models: its input is exactly that of the original mangaOCR, so it shows the effect of changing only the data. For general use, manga-ocr-base-CPT-flex is better on elongated and horizontal boxes.
This model comes from an experimental study; see the technical report.
JMangaBench_Read v1.3, 7,789 boxes, RIN v1 normalisation, batch 1, RTX 2080 Ti.
| Model | Score ↑ | CER % ↓ | EM % ↑ | EM printed | EM non-printed |
|---|---|---|---|---|---|
| manga-ocr-base-CPT | 97.43 | 1.77 | 91.00 | 93.82 | 82.32 |
| manga-ocr-base-CPT-flex | 97.45 | 1.53 | 91.42 | 94.16 | 83.00 |
| mangaOCR (original, package defaults) | 90.23 | 4.84 | 78.25 | 88.04 | 48.17 |
RTX 2080 Ti, Read v1.3 speed protocol, preprocessing included; a page (8.7 boxes on average) is one batch.
| Inference path | Per crop (batch 1) | Page p50 | Peak throughput |
|---|---|---|---|
| CUDA path (CUDA graphs + GPU preprocessing) | 10.4 ms | 38.1 ms | 277 crops/s (batch 32) |
| Plain PyTorch path | 27.6 ms | 69.7 ms | 234 crops/s (batch 64) |
| Original manga-ocr package (fp16) | 84.9 ms | 242.4 ms | 42 crops/s (batch 56) |
Both paths give identical outputs on all 7,789 boxes.
The inference code is on GitHub; the weights are downloaded from this repository on first use.
pip install git+https://github.com/muscgab/manga-ocr-CPT
from manga_ocr_cpt import MangaOcrCPT
ocr = MangaOcrCPT("224") # or a local directory holding this repository
ocr("crop.png") # one crop -> str
ocr(["crop1.png", "crop2.png"]) # the crops of one page -> list[str]
Inputs are cropped text boxes (file paths, PIL images or uint8 RGB arrays); there is no text detection. With an NVIDIA GPU the CUDA path is used, otherwise plain PyTorch (CPU, Apple MPS, ...).
kha-white/manga-ocr-base; the 5,133 characters already in the original vocabulary keep their embeddings.| File | Contents |
|---|---|
model.safetensors | Weights, fp32 (444 MB) |
config.json | Architecture and input settings (input.fit selects the input) |
nar_vocabulary_6817.json | Vocabulary (6,817 characters) |
LICENSE_MODEL | CC BY-NC-SA 4.0 |
NOTICE.md | Provenance and acknowledgements |
SHA256SUMS | Checksums |
LICENSE_MODEL); commercial use requires separate permission. The training data AnimeText-OCR is CC BY-NC-SA 4.0.NOTICE.md.