Manga Translator

Local web app + Python pipeline for translating Japanese manga to Vietnamese. Japanese is the primary, actively optimized path: its workflow uses Comic ONNX for text-region detection and MangaOCR for recognition.

English is available only as a convenience, best-effort source option. It is not a primary target for OCR accuracy, reading-order handling, or translation quality. Korean and Simplified Chinese are likewise experimental OCR options.

Pipeline: Phase A detect/OCR/inpaint once per source page → Phase B translate/render per target language → ZIP/CBZ export.

UI: Next.js + shadcn at http://localhost:3000. FastAPI runs locally at http://127.0.0.1:8000 and serves the pipeline over HTTP and WebSocket. No desktop shell or .exe is required.

Requirements

• Windows (primary target) with Python 3.11.
• Node.js 20+ for the frontend.
• Internet for MangaOCR/IOPaint first-time downloads and Claude/Gemini/OpenAI API calls. EasyOCR models for English, Korean, and Simplified Chinese are stored in assets/models/easyocr for offline use.
• API key from at least one LLM provider.

Translation scope

The project is designed first and foremost for Japanese manga. For the most reliable results, select Nguồn: Nhật and the appropriate Japanese reading direction for the source chapter.

Nguồn: Anh exists for occasional English pages, but should be treated as a secondary fallback rather than a fully tuned English-comics pipeline. It uses left-to-right ordering and EasyOCR-based recognition, so dense lettering, stylized fonts, and complex English layouts may require more manual review.

Setup

Python

Use .venv for all dependencies:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

If your Python distribution creates a POSIX-style venv layout on Windows, run pip through the venv interpreter directly:

.\.venv\bin\python.exe -m pip install --upgrade pip
.\.venv\bin\python.exe -m pip install -r requirements.txt

Frontend

cd frontend
npm install

GPU acceleration (dev, NVIDIA only)

Optional CUDA acceleration for the four local model stages — comic text detection (ONNX), MangaOcr, EasyOCR, and LaMa inpaint (IOPaint).

Requirements:

• NVIDIA GPU with a recent driver. ONNX Runtime 1.27 needs CUDA 13 + cuDNN 9 — check yours with nvidia-smi (look for the CUDA UMD Version line). No CUDA Toolkit install is needed; all DLLs come from pip wheels.
• ~3 GB of extra disk (the torch cu121 wheel is the bulk).

Install (do NOT combine with -r requirements.txt in one pip command — the CPU lock is incremental and cannot be fully re-resolved; see the header of requirements-gpu.txt):

.\.venv\Scripts\Activate.ps1
python -m pip uninstall onnxruntime
python -m pip install -r requirements-gpu.txt

Verify:

python -c "import torch; print(torch.cuda.is_available())"
python -c "import onnxruntime as ort; print(ort.get_available_providers())"

Both must report CUDA (True and CUDAExecutionProvider).

Behavior at runtime:

• Device selection is automatic and cached for the process; the backend logs one line at startup: Device: cuda — <GPU name> (ONNX: CUDAExecutionProvider, CPUExecutionProvider) or Device: cpu — <reason>.
• Any CUDA failure (broken driver, OOM at model load, missing DLL) falls back to CPU automatically with a warning — the pipeline never crashes because of a device problem. EasyOCR and the ONNX detector cache the downgrade for the rest of the process.
• Force CPU regardless of hardware with MANGA_TRANSLATOR_DEVICE=cpu (also excludes the CUDA ONNX provider).
• Revert to CPU builds with the revert command in requirements-gpu.txt.

Run (dev mode)

Start backend and frontend with the PowerShell script

After completing the Python and frontend setup, run this command from the project root:

.\run-dev.ps1

The script starts the FastAPI backend and Next.js frontend in two separate PowerShell windows. The default addresses are:

• Frontend: http://localhost:3000
• Backend API and Swagger: http://127.0.0.1:8000/docs

If PowerShell blocks local scripts, run it for this invocation only:

powershell -ExecutionPolicy Bypass -File .\run-dev.ps1

You can choose different ports when the defaults are already in use:

.\run-dev.ps1 -BackendPort 8001 -FrontendPort 3001

The script requires frontend\node_modules; run npm install in frontend first if it reports that the directory is missing. It uses .venv\Scripts\python.exe when available, otherwise py -3.11.

Run services manually

Use separate terminals for the following commands.

Terminal 1 — Backend

py -3.11 -m uvicorn backend.main:app --reload --port 8000

Swagger UI: http://127.0.0.1:8000/docs

Terminal 2 — Frontend (web)

cd frontend
npm run dev

Open http://localhost:3000.

The app opens in the Project Library. Open a saved project or choose a manga folder to create one; opening a new project does not start the pipeline. Settings are available before a project is selected. Use "Back to Library" to switch projects; switching is disabled while a job is running (Cancel remains available). Refreshing the browser returns to the Library.

The Library folder-picker button calls the local backend, which opens a Windows folder dialog on the same machine and fills the path field. You can still enter a path manually. The dialog is unavailable on headless/non-Windows hosts, but manual path entry remains available. The picker starts in the path already entered (or the last chosen folder).

Opening a previously translated source folder automatically reloads its saved pages and language-specific rendered images. Project ownership is recorded in projects.json under the data directory and project.json inside its output folder. Different source folders with the same basename receive separate output folders. Existing output is adopted only when saved originals match the source images; uncertain legacy output is left untouched and a separate project is created with a warning. Keep the source folder at the same path to reopen it. IOPaint loads in the background; the browser and basic settings remain usable while its model starts. Font discovery is cached and rechecked when font files change.

Configuration

The settings UI (Cài đặt tab) writes config.json in the project root by default. Set MANGA_TRANSLATOR_DATA_DIR to use another data directory; translations.db and output images follow the same setting.

Before starting a chapter, choose the source language and reading direction in the Workspace sidebar. The primary Japanese path uses Comic ONNX detection and MangaOCR recognition. For occasional English pages, select Nguồn: Anh + Trái → phải so Phase A uses left-to-right ordering. English, Korean, and Simplified Chinese are secondary/experimental paths: they use one full-page Comic ONNX pass, bounded multi-scale tiles, and a language-specific EasyOCR CRAFT pass fused as secondary evidence. EasyOCR recognizes the resulting crops.

English, Korean, and Simplified Chinese EasyOCR readers use the checked-in assets/models/easyocr models with downloads disabled. An explicit MANGA_TRANSLATOR_EASYOCR_MODEL_DIR overrides this path. Japanese EasyOCR still uses its own model download when selected.

Build and serve locally

# Backend tests
py -3.11 -m pytest tests/ -v

# Frontend lint + build
cd frontend
npm run lint
npm run build

npm run start

Run the backend separately with py -3.11 -m uvicorn backend.main:app --host 127.0.0.1 --port 8000 from the project root. npm run start serves the exported frontend at localhost:3000 using the locally installed static server. Build with NEXT_PUBLIC_API_URL set to the desired backend URL if it is not the default http://127.0.0.1:8000.

To refresh the checked-in EasyOCR assets from their verified upstream archives, run .\fetch-easyocr-models.ps1 from the project root.

Architecture

Browser → Next.js static export (frontend/out, localhost:3000)
  └── HTTP/WS → FastAPI (127.0.0.1:8000)
       └── Pipeline: Phase A detect/OCR/inpaint → Phase B translate/render

The Python pipeline under pipeline/ is the source of truth for AI logic. UI changes do not touch it. Adding a new LLM provider = one new function in pipeline/translate.py.

Tests

py -3.11 -m pytest -q

The CPU-only PR gate runs synthetic pipeline fixtures plus the P0-P4 contracts on Windows and Linux without model downloads or provider calls. External manga pages stay outside Git; point MANGA_BENCH_ROOT at the corpus and run py -3.11 -m benchmarks.runner to produce metrics.json, report.html, and debug overlays.

Fonts

The project includes only allow-listed fonts whose binary hash and license are declared in assets/fonts/manifest.json. Noto Sans Condensed SemiBold is the licensed text fallback and Noto Sans Symbols 2 supplies missing symbol glyphs. Local fonts (.ttf or .otf) can be copied into the fonts directory under the data directory; unmanifested files in assets/fonts are treated as local-only. They are listed in Settings after a backend restart. A local Yuki-Burobu.ttf is preferred on first setup, but no Yuki font is included without redistribution rights.

Notes

• First manga-ocr run may download model weights (~400 MB).
• The local model assets include EasyOCR CRAFT plus English, Korean, and Simplified Chinese Gen2 recognizers (about 136.3 MB). Their official source URLs, archive/file checksums, and Apache-2.0 license are recorded in assets/models/easyocr/manifest.json.
• opencv-contrib-python and opencv-python-headless are locked to 4.10.0.84. IOPaint declares the separate opencv-python distribution, so pip may also install that package transitively; it is intentionally not a second direct project pin.
• Frontend URL http://localhost:3000 must match MANGA_FRONTEND_ORIGIN on the backend (set automatically by run-dev.ps1). If using a custom port, set both it and NEXT_PUBLIC_API_URL consistently before building.