Razčlenjevalnik Revij (RaRe)

RaRe is a parsing toolkit for Slovene magazines (primarily Glasbena Mladina), built on top of a fork of layoutparser. It exposes two tracks for parsing PDFs and comparing models on annotated data:

• Pipeline track — DLA model → reading-order (additional OCR → figure linking →) assembled GlasanaDocument → HTML / Markdown / JSON.
• VLM track — vision-language model (cloud or locally-served) producing HTML / Markdown / JSON directly.

These are furthermore enriched with named entities, and classification.

Pipelines track assumes presence of previously OCR-ed PDFs. If empty, the text layer can be re-read using --ocr flag.

Installation

Before running either of these, check Additional model-specific setup below for additional per-model requirements. Then run (if the code below doesn't include the desired model, no additional dependencies are needed):

pip install -e .                   # core package + 'rare' command

# Model specific dependencies
pip install -e ".[detr]" # + DETR dependencies
pip install -e ".[dit]" # + DiT dependencies
pip install -e ".[doclayout-yolo]" # + DocLayout-YOLO dependencies
pip install -e ".[faster-rcnn]" # + Faster R-CNN dependencies
pip install -e ".[layoutlmv3]" # + LayoutLMv3 dependencies 
pip install -e ".[mask-rcnn]" # + Mask R-CNN dependencies
pip install -e ".[pp-doclayoutv3]" --extra-index-url https://download.pytorch.org/whl/cpu # + PP-DocLayoutV3 dependencies
pip install -e ".[rf-detr]" # + RF-DETR dependencies
pip install -e ".[vgt]" # + VGT dependencies

pip install -e ".[marker]" # + Marker dependencies

Furthermore, install LayoutParser fork:

pip install -e layout-parser

It is recommended to create a separate Conda environment for each of the intended models in order to avoid library and version clashes between different model's dependencies.

Usage

The single rare command exposes three subcommands.

rare parse — parse a PDF

# Pipeline track
rare parse <pdf> --layout doclayout-yolo --order top-bottom

# VLM track (mutually exclusive with --layout)
rare parse <pdf> --vlm claude

# Pipeline track, with per-page Markdown for OmniDocBench's end2end evaluator
rare parse <pdf> --layout doclayout-yolo --emit-omnidocbench

# Choose the NER backend used by the linking stage
rare parse <pdf> --layout doclayout-yolo --ner rudar-slv

# Re-read regions the PDF's text layer left empty (running headers, by default)
rare parse <pdf> --layout doclayout-yolo --ocr tesseract

# Discover backends
rare parse --list-models

rare evaluate — score one model against a dataset

# Pipeline track
rare evaluate --track pipeline --dataset glasbena_mladina \
    --layout doclayout-yolo --order top-bottom \
    [--run-id myrun-2026-05] [--limit 5]

# VLM track 
rare evaluate --track vlm --dataset glasbena_mladina \
    --vlm claude \
    [--pdfs-dir dataset/pdfs] [--run-id myrun-2026-05]

Each invocation runs one model. Re-invoke with the same --run-id to accumulate models; report.md produces the manual evaluation report.

Outputs are stored in outputs/evaluations/<run_id>/{report.md, scores.csv, per_model/}.

rare evaluate --track figure-link --dataset glasbena_mladina \
    --pdfs-dir datasets/glasbena_mladina/pdfs/eval

rare evaluate --track article-genre --classification gams \
    --dataset glasbena_mladina

OmniDocBench evaluation (--run-omnidocbench)

Both tracks can run OmniDocBench's end-to-end evaluator, which emits text_block and reading_order NED.

On the pipeline track, mode can be selected with --omnidocbench-eval:

The two passes ship in different images, so they are configured separately:

• --omnidocbench-image for end-to-end,
• --omnidocbench-layout-image for detection.

OmniDocBench end2end evaluation

Before running, pull the following image:

docker pull ghcr.io/zeng-weijun/omnidocbench-eval:repro-ubuntu2204

Then run:

rare evaluate --track pipeline --dataset glasbena_mladina \
    --layout doclayout-yolo --order top-bottom \
    --run-omnidocbench --pdfs-dir datasets/glasbena_mladina/pdfs

rare evaluate --track vlm --dataset glasbena_mladina \
    --vlm dots-ocr \
    --run-omnidocbench --pdfs-dir datasets/glasbena_mladina/pdfs

OmniDocBench Layout detection evaluation

The pipeline track can also run OmniDocBench's layout evaluator, including mAP. Pass --run-omnidocbench. Use --omnidocbench-image to override the image.

Inside OmnoDocBench-Dockerfile, build the Docker image:

docker build -t omnidocbench-v15 .

Then run:

rare evaluate --track pipeline --dataset glasbena_mladina \
    --layout doclayout-yolo --order top-bottom \
    --run-omnidocbench --pdfs-dir datasets/glasbena_mladina/pdfs

rare tools — annotation utilities

rare tools -m count-annotations -a dataset/annotations.json
rare tools -m join-annotations -p results/<doc_hash>/ -o merged.json
rare tools -m prepare-annotations -a merged.json -o cleaned.json
rare tools -m review-annotations -a cleaned.json -s reviewed/

rare tools -h prints the full flag list (same as the old helper.py).

Supported Models

The supported models (and therefore given Python version recommendations) were tested using:

• Ubuntu 24.04
• CUDA 12.8

Pipeline track — layout backends

* Included in LayoutParser with detectron2

Pipeline track — reading-order backends

VLM track

Outputs

Main result of the parser is <stem>_articles.json, which contains one entry per article with its items in reading order. It can be used to render articles into formats such as Markdown and HTML.

Project Structure

rare/                         # installable package — entry point: rare = "rare.cli:main"
├── cli.py                    # rare parse | evaluate | tools
├── doc/{schema,renderers}.py # GlasanaDocument + 43 region classes + HTML/MD renderers
├── models/
│   ├── base.py               # LayoutBackend / ReadingOrderBackend / VLMBackend protocols
│   ├── registry.py           # lazy registry; sets LAYOUTPARSER_BACKEND env var
│   ├── layout/               # layout detection model/method classes
│   ├── order/                # order detection model/method classes
│   └── vlm/                  # visual language model document parsing classes
├── parse/                    # PDF → pages → layout → order → text → GlasanaDocument
├── evaluate/                 # dataset loaders + pipeline/VLM/figure-link metrics + runner + report
├── tools/_helper.py          # annotation utilities
└── utils/                    # eval / display / file / conversion / character helpers
configs/                      # JSON configs per model
data/                         # default path for model weights and model files
datasets/                     # default path for datasets
layout-parser/                # git submodule (layoutparser fork)
outputs/                      # outputs/parsed/* + outputs/evaluations/*

Additional model-specific setup

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html

python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html

pip install torch==1.10.0+cu111 torchvision==0.11.1+cu111 -f https://download.pytorch.org/whl/torch_stable.html

python -m pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.10/index.html

git clone https://github.com/microsoft/unilm.git

pip install -e .

python -m pip install paddlepaddle-gpu==3.3.1 -i https://www.paddlepaddle.org.cn/packages/stable/cu126/

conda install nvidia::cuda-nvcc==12.8.93

pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 --index-url https://download.pytorch.org/whl/cu128

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

python -m pip install detectron2==0.6 -f  https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html

pip install transformers

python create_grid_input.py \
--pdf 'path-to-pdf-file' \
--output 'path-to-output-folder' \
--tokenizer 'google-bert/bert-base-uncased' \
--model 'doclaynet'

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu128

pip install psutil
pip install flash-attn --no-build-isolation

pip install -e .

pip install uv # If uv is not previously installed
uv pip install vllm --torch-backend=cu128

conda install nvidia::cuda-nvcc==12.8.93

CUDA_VISIBLE_DEVICES=0 vllm serve rednote-hilab/dots.mocr --tensor-parallel-size 1 --gpu-memory-utilization 0.9 --chat-template-content-format string --served-model-name model --trust-remote-code

pip install "vllm<0.20"

pip install zai-sdk

conda install nvidia::cuda-nvcc==12.8.93

vllm serve zai-org/GLM-OCR --allowed-local-media-path / --port 8080

pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu128

pip uninstall vllm
pip install "vllm<0.20"

pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 --index-url https://download.pytorch.org/whl/cu128

pip install --upgrade pip
pip install uv
uv pip install -U "mineru[all]"

pip install torch==2.9.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

uv pip install "mineru[core,vllm]"

pip install albumentations timm open_clip_torch

hf download nvidia/NVIDIA-Nemotron-Parse-v1.2 chat_template.jinja --local-dir . 

vllm serve nvidia/NVIDIA-Nemotron-Parse-v1.2 \
    --dtype bfloat16 \
    --max-num-seqs 8 \
    --limit-mm-per-prompt '{"image": 1}' \
    --trust-remote-code \
    --port 8000 \
    --chat-template chat_template.jinja

 python -m pip install paddlepaddle-gpu==3.3.1 -i https://www.paddlepaddle.org.cn/packages/stable/cu126/

python -m pip install "paddleocr[all]"

vllm serve Qwen/Qwen3-VL-8B-Instruct \
  --gpu-memory-utilization 0.95 \
  --max-model-len 12288 \ 
  --max-num-seqs 2 \
  --limit-mm-per-prompt '{"image": 1}' \
  --mm-processor-kwargs '{"max_pixels": 1003520}'

Note: for Claude, ChatGPT and Gemini, the user must have an account and API key, which is pasted into the config file, which must then be passed as a parameter. The examples for each of them are present in ther respective configs directories.

Evaluation results

The following results were obtained by evaluating detections made by the following models on ground truths of manually annotated Glasbena Mladina magazines.

Note: NED - Normalized edit distance.

Layout Analysis

Two approaches to evaluation are can be computed:

• manual (hand written functions for computation of mAP), however we don't report the results here and
• using OmniDocBench — run as part of rare evaluate via --run-omnidocbench (see the usage section).

The results in the following table were obtained using OmniDocBench's evaluation.

Reading Order

VLM

Specialized VLMs:

* Only results successfully parsed were scored against ground truth.

General VLMs

* As of 15.7.2026, unoptimized (no use of cache), using similar resolution as seen on OmniDocBench dataset images.

Retried OCR results

Text block NED scores for our pipeline implementation with and without OCR retrying, using our trained DocLayout-YOLO model and LayoutReader for reading order.

Page wise breakdown of NED scores for best VLMs

Comparison of the best performing VLMs compared to our implementation, given with normalized edit distance by page type.

Text = text blocks, Order = reading order.

YP = Youtu-Parsing, DLY+LR = DocLayout-YOLO + LayoutReader.

* Due to ground truth being obtained using bounding boxes but same OCR-ed letters (barring retrying of failed extractions), the text block NED is not directly comparable to the other VLMs, but is included for reference.

Figure linking results

Ablation study, scores figures, captions and figure bylines correctly linked to their respective articles using proximity, NER and additional heuristics.

Classification results

Number of correctly classified articles.

Demo

To be added.

TODO

Top priority:

• Add mappings from other datasets (PubLayNet, DocBank, DocLayNet) to OmniDocBench schema for evaluation
• Add OmniDocBench evaluation support for pipeline track
  • Fix classes and other issues in OmniDocBench layout evaluation
• Add specialized VLM support:
  • Marker
• Add general VLM support, among others:
  • Qwen3-VL
  • GPT 5.5
  • Gemini Pro 3.1
  • Anthropic Claude Opus 4.8
• Evaluate all models:
  • Pipeline
  • Specialized VLM
  • General VLM

Limitations and Further Work

Pipeline based track:

• Built layout detection and reading order detection tasks are evaluated separately (reading order is evaluated using ground bounding boxes).
• Currently, RaRe only supports inference; possible extension includes training of the available models.
• VLM track currently only supports Youtu-parse. Further improvement could see integration of a larger number of models into rare, so they can be output in an arbitrary format (such as JSON, HTML etc.)

Acknowledgements

Thanks for the work of the authors of these projects:

• PaddleX — the improved XY-Cut reading-order backend is vendored from PaddleX (Apache-2.0); see NOTICE and licenses/LICENSE-PADDLEX.
• OmniDocBench — the end-to-end Edit-distance evaluator (run via --run-omnidocbench) and the specialized VLM img2md parsing backends are adapted from OmniDocBench (Apache-2.0); see NOTICE and licenses/LICENSE-OMNIDOCBENCH.
• layoutreader — the layoutreader reading-order backend uses the LayoutLMv3 inference helpers and the hantian/layoutreader checkpoint from Hantian Pang's faster LayoutReader (CC BY-NC-SA 4.0); neither is redistributed by RaRe, see NOTICE and the licensing note below.
• DocLayout-YOLO

Citation

@phdthesis{
    Kuzman_2026,
    title={Samodejno razčlenjevanje kompleksnih postavitev dokumentov},
    url={https://hdl.handle.net/20.500.12556/RUL-189793},
    abstractNote={Razčlenjevanje kompleksnih vrst dokumentov, kot so revije, je zahtevnejše od razčlenjevanja preprostejših dokumentov, na primer znanstvenih člankov. Primerne zbirke za razčlenjevanje slovenskih dokumentov doslej še ni bilo. V sklopu magistrskega dela smo z ročnim anotiranjem 35 dokumentov, ki vsebujejo 881 slik, ustvarili prvo takšno zbirko v slovenščini. Na njej smo ovrednotili celovite modele za razčlenjevanje in posamezne dele cevovodnega pristopa; najbolje se je izkazal Youtu-Parsing z oceno razdalje urejanja 0,031. Na zbirki smo naučili model DocLayout-YOLO, ki dosega mAP 64,3 %. Razvili smo tudi cevovod, ki s temi modeli iz dokumentov izlušči članke ter jih obogati z metapodatki.},
    author={Kuzman, Luka},
    year={2026}
}

@software{Pang_Faster_LayoutReader_based_2024,
  author  = {Pang, Hantian},
  month   = feb,
  title   = {{Faster LayoutReader based on LayoutLMv3}},
  url     = {https://github.com/ppaanngggg/layoutreader},
  version = {1.0.0},
  year    = {2024}
}

@inproceedings{wang-etal-2021-layoutreader,
  title     = {{L}ayout{R}eader: Pre-training of Text and Layout for Reading Order Detection},
  author    = {Wang, Zilong and Xu, Yiheng and Cui, Lei and Shang, Jingbo and Wei, Furu},
  booktitle = {Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing},
  month     = nov,
  year      = {2021},
  address   = {Online and Punta Cana, Dominican Republic},
  publisher = {Association for Computational Linguistics},
  url       = {https://aclanthology.org/2021.emnlp-main.389/},
  doi       = {10.18653/v1/2021.emnlp-main.389},
  pages     = {4735--4744}
}