Main Leaderboard

42 models · 3 tracks · 🏆 Search & sort the leaderboard →

Top 10 by Avg₃, highest first. Bold = best; underlined = runner-up. ‡ = author-reported result.

CSV · JSON · Evaluation protocol · Submit results

Show all 42 models

Rank	Model	Model type	Params	Avg₃ ↑	Clean ↑	Digital ↑	Real ↑
1	TeleOCR ‡	Pipeline / Multi-stage	1.2B	78.41	86.90	77.47	70.85
2	OvisOCR2 ‡	End-to-End	0.8B	75.06	81.55	77.09	66.56
3	Qwen3.5-122B-A10B	General VLM	122B/10B	74.11	76.14	76.34	69.85
4	FD-RL	End-to-End	4B	73.92	78.38	76.33	67.04
5	Logics-Parsing-v2	End-to-End	4B	72.61	76.35	73.85	67.64
6	Qwen3.5-9B	General VLM	9B	70.89	73.87	73.34	65.45
7	Gemini-3.1-Pro	General VLM	—	70.43	70.04	69.28	71.98
8	DotsMOCR	Pipeline / Multi-stage	3B	70.39	76.27	73.16	61.73
9	Kimi K2.6	General VLM	1T/32B	70.10	72.32	69.95	68.02
10	MinerU2.5-Pro	Pipeline / Multi-stage	1.2B	70.07	75.87	71.77	62.56
11	Qwen3.5-4B	General VLM	4B	69.82	73.45	72.53	63.47
12	Qwen3.5-27B	General VLM	27B	69.57	72.07	70.73	65.92
13	OCRVerse	End-to-End	4B	69.40	73.18	71.36	63.66
14	Qwen3-VL-8B	General VLM	8B	69.07	72.44	72.03	62.73
15	YouTu-Parsing	Pipeline / Multi-stage	2B	68.32	75.02	69.66	60.29
16	MinerU2.5	Pipeline / Multi-stage	1.2B	67.66	74.90	68.92	59.15
17	Qwen3-VL-4B	General VLM	4B	67.50	72.04	70.84	59.61
18	PaddleOCR-VL-1.5	Pipeline / Multi-stage	0.9B	66.75	73.01	66.73	60.50
19	Qwen3.5-397B-A17B	General VLM	397B/17B	66.72	69.12	68.34	62.70
20	Qwen3.5-35B-A3B	General VLM	35B/3B	65.68	68.40	68.04	60.59
21	FireRed-OCR	End-to-End	2B	65.57	70.81	68.49	57.42
22	dots.ocr	End-to-End	2.9B	64.55	72.01	65.95	55.68
23	olmOCR-2-7B	End-to-End	7B	63.78	69.36	65.87	56.10
24	GLM-OCR	Pipeline / Multi-stage	0.9B	63.34	68.65	63.06	58.31
25	Qwen3.5-2B	General VLM	2B	62.46	66.24	65.22	55.92
26	Qwen3-VL-2B	General VLM	2B	62.09	66.37	65.81	54.09
27	HunyuanOCR	End-to-End	1B	60.56	65.61	61.49	54.58
28	Nanonets-OCR2	End-to-End	3B	58.36	64.83	61.23	49.03
29	Dolphin-v2	Pipeline / Multi-stage	3B	57.02	65.90	60.24	44.92
30	Qwen3.5-0.8B	General VLM	0.8B	55.99	60.77	59.28	47.93
31	olmOCR-7B	End-to-End	7B	55.90	62.56	57.84	47.30
32	MonkeyOCR-pro-3B	Pipeline / Multi-stage	3B	55.37	62.23	57.40	46.49
33	MonkeyOCR-pro-1.2B	Pipeline / Multi-stage	1.2B	53.54	61.09	55.72	43.82
34	OpenDoc-0.1B †	End-to-End	0.1B	52.00	60.28	52.46	44.27
35	Qianfan-OCR	End-to-End	4B	51.04	57.22	50.85	45.06
36	Step3-VL	General VLM	10B	50.48	53.65	52.74	45.06
37	DeepSeek-OCR-2	End-to-End	3B	49.51	55.53	49.41	43.60
38	UniRec-0.1B	End-to-End	0.1B	48.59	58.91	52.42	34.44
39	DeepSeek-OCR	End-to-End	3B	46.98	53.50	46.95	40.48
40	MiniCPM-V-4.5	General VLM	8B	46.26	51.81	49.38	37.59
41	OCRFlux-3B	End-to-End	3B	42.06	47.14	41.82	37.21
42	OpenOCR	Pipeline / Multi-stage	0.1B	29.49	32.70	30.03	25.73

Clean: full metrics for all 42 models

Rank	Model	Model type	Params	Overall ↑	TextEdit ↓	FormulaCDM ↑	TableTEDS ↑	ROEdit ↓
1	TeleOCR ‡	Pipeline / Multi-stage	1.2B	86.90	0.111	81.01	91.09	—
2	OvisOCR2 ‡	End-to-End	0.8B	81.55	—	—	—	—
3	FD-RL	End-to-End	4B	78.38	0.193	68.21	86.22	0.334
4	Logics-Parsing-v2	End-to-End	4B	76.35	0.213	67.67	82.67	0.342
5	DotsMOCR	Pipeline / Multi-stage	3B	76.27	0.151	66.23	77.65	0.273
6	Qwen3.5-122B-A10B	General VLM	122B/10B	76.14	0.226	67.96	83.03	0.375
7	MinerU2.5-Pro	Pipeline / Multi-stage	1.2B	75.87	0.222	65.14	84.68	0.346
8	YouTu-Parsing	Pipeline / Multi-stage	2B	75.02	0.230	67.34	80.74	0.358
9	MinerU2.5	Pipeline / Multi-stage	1.2B	74.90	0.184	62.08	81.04	0.327
10	Qwen3.5-9B	General VLM	9B	73.87	0.254	67.60	79.39	0.388
11	Qwen3.5-4B	General VLM	4B	73.45	0.276	69.96	78.02	0.410
12	OCRVerse	End-to-End	4B	73.18	0.273	63.78	83.09	0.393
13	PaddleOCR-VL-1.5	Pipeline / Multi-stage	0.9B	73.01	0.266	63.53	82.12	0.428
14	Qwen3-VL-8B	General VLM	8B	72.44	0.261	65.10	78.35	0.411
15	Kimi K2.6	General VLM	1T/32B	72.32	0.303	66.93	80.30	0.466
16	Qwen3.5-27B	General VLM	27B	72.07	0.227	66.36	72.51	0.362
17	Qwen3-VL-4B	General VLM	4B	72.04	0.262	65.10	77.17	0.418
18	dots.ocr	End-to-End	2.9B	72.01	0.248	61.37	79.51	0.379
19	FireRed-OCR	End-to-End	2B	70.81	0.287	63.86	77.23	0.396
20	Gemini-3.1-Pro	General VLM	—	70.04	0.306	65.63	75.08	0.409
21	olmOCR-2-7B	End-to-End	7B	69.36	0.284	56.89	79.59	0.358
22	Qwen3.5-397B-A17B	General VLM	397B/17B	69.12	0.233	65.26	65.40	0.366
23	GLM-OCR	Pipeline / Multi-stage	0.9B	68.65	0.314	57.89	79.44	0.470
24	Qwen3.5-35B-A3B	General VLM	35B/3B	68.40	0.232	64.94	63.45	0.374
25	Qwen3-VL-2B	General VLM	2B	66.37	0.300	59.04	70.03	0.439
26	Qwen3.5-2B	General VLM	2B	66.24	0.348	62.84	70.70	0.473
27	Dolphin-v2	Pipeline / Multi-stage	3B	65.90	0.342	59.80	72.12	0.429
28	HunyuanOCR	End-to-End	1B	65.61	0.269	55.74	68.02	0.382
29	Nanonets-OCR2	End-to-End	3B	64.83	0.254	44.98	74.94	0.377
30	olmOCR-7B	End-to-End	7B	62.56	0.388	58.69	67.77	0.466
31	MonkeyOCR-pro-3B	Pipeline / Multi-stage	3B	62.23	0.346	48.46	72.83	0.492
32	MonkeyOCR-pro-1.2B	Pipeline / Multi-stage	1.2B	61.09	0.358	47.43	71.60	0.498
33	Qwen3.5-0.8B	General VLM	0.8B	60.77	0.376	54.39	65.54	0.500
34	OpenDoc-0.1B †	End-to-End	0.1B	60.28	0.411	53.09	68.86	0.519
35	UniRec-0.1B	End-to-End	0.1B	58.91	0.422	51.31	67.60	0.526
36	Qianfan-OCR	End-to-End	4B	57.22	0.370	49.79	58.83	0.443
37	DeepSeek-OCR-2	End-to-End	3B	55.53	0.354	46.00	56.01	0.466
38	Step3-VL	General VLM	10B	53.65	0.496	53.41	57.16	0.509
39	DeepSeek-OCR	End-to-End	3B	53.50	0.419	45.39	57.06	0.514
40	MiniCPM-V-4.5	General VLM	8B	51.81	0.439	45.97	53.36	0.481
41	OCRFlux-3B	End-to-End	3B	47.14	0.454	38.35	48.46	0.424
42	OpenOCR	Pipeline / Multi-stage	0.1B	32.70	0.354	33.50	0.00	0.507

Digital-degraded: full metrics for all 42 models

Rank	Model	Model type	Params	Overall ↑	TextEdit ↓	FormulaCDM ↑	TableTEDS ↑	ROEdit ↓
1	TeleOCR ‡	Pipeline / Multi-stage	1.2B	77.47	0.206	72.59	80.45	—
2	OvisOCR2 ‡	End-to-End	0.8B	77.09	—	—	—	—
3	Qwen3.5-122B-A10B	General VLM	122B/10B	76.34	0.220	67.82	83.21	0.366
4	FD-RL	End-to-End	4B	76.33	0.214	67.16	83.22	0.350
5	Logics-Parsing-v2	End-to-End	4B	73.85	0.248	67.33	79.02	0.375
6	Qwen3.5-9B	General VLM	9B	73.34	0.260	67.00	79.01	0.396
7	DotsMOCR	Pipeline / Multi-stage	3B	73.16	0.198	64.32	74.95	0.309
8	Qwen3.5-4B	General VLM	4B	72.53	0.281	68.88	76.78	0.412
9	Qwen3-VL-8B	General VLM	8B	72.03	0.266	64.88	77.82	0.409
10	MinerU2.5-Pro	Pipeline / Multi-stage	1.2B	71.77	0.272	61.79	80.73	0.378
11	OCRVerse	End-to-End	4B	71.36	0.302	63.95	80.36	0.415
12	Qwen3-VL-4B	General VLM	4B	70.84	0.272	63.54	76.13	0.425
13	Qwen3.5-27B	General VLM	27B	70.73	0.236	64.61	71.17	0.367
14	Kimi K2.6	General VLM	1T/32B	69.95	0.322	64.69	77.31	0.475
15	YouTu-Parsing	Pipeline / Multi-stage	2B	69.66	0.270	61.44	74.49	0.388
16	Gemini-3.1-Pro	General VLM	—	69.28	0.322	65.81	74.24	0.417
17	MinerU2.5	Pipeline / Multi-stage	1.2B	68.92	0.245	56.99	74.24	0.374
18	FireRed-OCR	End-to-End	2B	68.49	0.319	62.64	74.77	0.422
19	Qwen3.5-397B-A17B	General VLM	397B/17B	68.34	0.244	63.91	65.53	0.376
20	Qwen3.5-35B-A3B	General VLM	35B/3B	68.04	0.245	64.78	63.86	0.379
21	PaddleOCR-VL-1.5	Pipeline / Multi-stage	0.9B	66.73	0.339	58.03	76.07	0.478
22	dots.ocr	End-to-End	2.9B	65.95	0.307	56.67	71.86	0.417
23	olmOCR-2-7B	End-to-End	7B	65.87	0.318	54.57	74.81	0.378
24	Qwen3-VL-2B	General VLM	2B	65.81	0.314	60.25	68.52	0.448
25	Qwen3.5-2B	General VLM	2B	65.22	0.350	58.30	72.36	0.477
26	GLM-OCR	Pipeline / Multi-stage	0.9B	63.06	0.383	53.23	74.21	0.520
27	HunyuanOCR	End-to-End	1B	61.49	0.308	51.62	63.68	0.400
28	Nanonets-OCR2	End-to-End	3B	61.23	0.307	45.40	68.97	0.408
29	Dolphin-v2	Pipeline / Multi-stage	3B	60.24	0.393	52.20	67.86	0.461
30	Qwen3.5-0.8B	General VLM	0.8B	59.28	0.386	54.22	62.22	0.510
31	olmOCR-7B	End-to-End	7B	57.84	0.436	55.44	61.66	0.499
32	MonkeyOCR-pro-3B	Pipeline / Multi-stage	3B	57.40	0.397	45.57	66.32	0.526
33	MonkeyOCR-pro-1.2B	Pipeline / Multi-stage	1.2B	55.72	0.416	43.91	64.83	0.529
34	Step3-VL	General VLM	10B	52.74	0.516	53.62	56.15	0.529
35	OpenDoc-0.1B †	End-to-End	0.1B	52.46	0.501	48.41	59.04	0.577
36	UniRec-0.1B	End-to-End	0.1B	52.42	0.501	48.37	59.04	0.578
37	Qianfan-OCR	End-to-End	4B	50.85	0.438	44.41	51.96	0.485
38	DeepSeek-OCR-2	End-to-End	3B	49.41	0.412	40.78	48.67	0.493
39	MiniCPM-V-4.5	General VLM	8B	49.38	0.461	42.79	51.50	0.489
40	DeepSeek-OCR	End-to-End	3B	46.95	0.478	39.99	48.64	0.548
41	OCRFlux-3B	End-to-End	3B	41.82	0.486	31.90	42.17	0.437
42	OpenOCR	Pipeline / Multi-stage	0.1B	30.03	0.410	31.09	0.00	0.541

Real-degraded: full metrics for all 42 models

Rank	Model	Model type	Params	Overall ↑	TextEdit ↓	FormulaCDM ↑	TableTEDS ↑	ROEdit ↓
1	Gemini-3.1-Pro	General VLM	—	71.98	0.300	68.62	77.26	0.386
2	TeleOCR ‡	Pipeline / Multi-stage	1.2B	70.85	0.302	65.11	77.66	—
3	Qwen3.5-122B-A10B	General VLM	122B/10B	69.85	0.281	62.19	75.44	0.401
4	Kimi K2.6	General VLM	1T/32B	68.02	0.335	62.44	75.14	0.481
5	Logics-Parsing-v2	End-to-End	4B	67.64	0.304	61.65	71.64	0.416
6	FD-RL	End-to-End	4B	67.04	0.298	58.82	72.08	0.391
7	OvisOCR2 ‡	End-to-End	0.8B	66.56	—	—	—	—
8	Qwen3.5-27B	General VLM	27B	65.92	0.283	61.23	64.82	0.390
9	Qwen3.5-9B	General VLM	9B	65.45	0.332	60.91	68.59	0.437
10	OCRVerse	End-to-End	4B	63.66	0.363	57.03	70.30	0.452
11	Qwen3.5-4B	General VLM	4B	63.47	0.380	61.27	67.17	0.477
12	Qwen3-VL-8B	General VLM	8B	62.73	0.342	55.55	66.81	0.448
13	Qwen3.5-397B-A17B	General VLM	397B/17B	62.70	0.287	60.70	56.12	0.399
14	MinerU2.5-Pro	Pipeline / Multi-stage	1.2B	62.56	0.375	52.70	72.47	0.446
15	DotsMOCR	Pipeline / Multi-stage	3B	61.73	0.312	54.39	61.97	0.393
16	Qwen3.5-35B-A3B	General VLM	35B/3B	60.59	0.310	59.68	53.07	0.419
17	PaddleOCR-VL-1.5	Pipeline / Multi-stage	0.9B	60.50	0.398	54.00	67.33	0.510
18	YouTu-Parsing	Pipeline / Multi-stage	2B	60.29	0.360	52.20	64.69	0.430
19	Qwen3-VL-4B	General VLM	4B	59.61	0.378	55.15	61.47	0.480
20	MinerU2.5	Pipeline / Multi-stage	1.2B	59.15	0.370	49.01	65.41	0.446
21	GLM-OCR	Pipeline / Multi-stage	0.9B	58.31	0.433	50.34	67.83	0.543
22	FireRed-OCR	End-to-End	2B	57.42	0.415	51.60	62.16	0.474
23	olmOCR-2-7B	End-to-End	7B	56.10	0.417	48.79	61.25	0.439
24	Qwen3.5-2B	General VLM	2B	55.92	0.440	50.99	60.79	0.521
25	dots.ocr	End-to-End	2.9B	55.68	0.403	47.70	59.63	0.467
26	HunyuanOCR	End-to-End	1B	54.58	0.421	48.30	57.54	0.459
27	Qwen3-VL-2B	General VLM	2B	54.09	0.428	51.05	53.99	0.511
28	Nanonets-OCR2	End-to-End	3B	49.03	0.435	35.50	55.09	0.468
29	Qwen3.5-0.8B	General VLM	0.8B	47.93	0.498	44.60	48.98	0.557
30	olmOCR-7B	End-to-End	7B	47.30	0.542	46.26	49.80	0.568
31	MonkeyOCR-pro-3B	Pipeline / Multi-stage	3B	46.49	0.511	38.18	52.43	0.600
32	Qianfan-OCR	End-to-End	4B	45.06	0.494	39.08	45.53	0.509
33	Step3-VL	General VLM	10B	45.06	0.579	45.42	47.66	0.573
34	Dolphin-v2	Pipeline / Multi-stage	3B	44.92	0.553	39.98	50.04	0.558
35	OpenDoc-0.1B †	End-to-End	0.1B	44.27	0.547	38.46	49.06	0.603
36	MonkeyOCR-pro-1.2B	Pipeline / Multi-stage	1.2B	43.82	0.556	36.94	50.07	0.609
37	DeepSeek-OCR-2	End-to-End	3B	43.60	0.486	37.30	42.06	0.533
38	DeepSeek-OCR	End-to-End	3B	40.48	0.537	34.04	41.12	0.575
39	MiniCPM-V-4.5	General VLM	8B	37.59	0.583	32.01	39.06	0.552
40	OCRFlux-3B	End-to-End	3B	37.21	0.559	32.65	34.87	0.491
41	UniRec-0.1B	End-to-End	0.1B	34.44	0.658	30.97	38.16	0.685
42	OpenOCR	Pipeline / Multi-stage	0.1B	25.73	0.486	25.81	0.00	0.591

‡ Author-reported results added after the paper: TeleOCR, OvisOCR2. TeleOCR was previously named NaviDC-OCR. TeleOCR's Avg₃ (78.41) is the mean of its reported track scores. Unreported component metrics are shown as —.

Source and evaluation notes document the added results, rounding, and evaluation disclosures.

† OpenDoc-0.1B retains the published Avg3 of 52.00. The mean of its displayed track scores is 52.34; this transcription preserves the paper value pending an erratum.

Original paper Table 2 (40 baselines)

Evaluation protocol

• Overall ↑ = (100 × (1 − TextEdit) + FormulaCDM + TableTEDS) / 3.
• Avg₃ ↑ is the arithmetic mean of the three track Overall scores, with the published values preserved above.
• TextEdit ↓ and ROEdit ↓ use the 0–1 scale; FormulaCDM ↑, TableTEDS ↑, Overall ↑, and Avg₃ ↑ use the 0–100 scale. Reading order is reported separately from Overall.
• The benchmark has 1,475 pages per track. The 40 paper baselines retain their original scores and protocol. TeleOCR and OvisOCR2 are transcribed from the linked author reports; their exact GT revisions are not specified in those public score tables. The current GT download has continued to receive annotation updates. New submissions should identify their GT revision and page manifest.
• For paper-compatible scoring, follow the OmniDocBench export and evaluation instructions. The lightweight public scorer is for development checks and uses a different scoring implementation.

Source: paper Table 2 and versioned published table.

Submit results

Open a discussion or a dataset pull request with the following information:

1. Model name, exact checkpoint or API version, parameter count, and model/code link.
2. GT revision, page manifest, and results for Clean, Digital-degraded, and Real-degraded, including all five metrics per track.
3. Inference settings, evaluator version/configuration, and prediction or evaluation-log links where available.
4. A clear distinction between author-reported results and results reproduced by the benchmark maintainers.

Maintainers review submissions before adding them to the published leaderboard. Evaluation revisions and result sources are recorded explicitly.

PureDocBench

How far is document parsing from solved?
A source-traceable benchmark for OCR and document parsing across clean, digitally degraded, and real-degraded document settings.

中文说明 | Leaderboard | Dataset | Paper | GT Review & Corrections

Benchmark Overview

PureDocBench uses HTML/CSS document sources as hidden anchors: each page is rendered into images and annotated from the same structured source. This gives a benchmark where text, tables, formulas, captions, and reading order can be scored with less post-hoc annotation noise.

PureDocBench 是一个源可追踪的 OCR / 文档解析 benchmark。数据由 HTML/CSS 源文件渲染而来，GT 标注从同源结构中抽取，覆盖 clean、digital-degraded、real-degraded 三条图像轨道。

Updates

• 2026-09-16: Published the Hugging Face leaderboard with 40 paper baselines and author-reported TeleOCR and OvisOCR2 results, plus CSV/JSON downloads.

• Current GT: The stable alias is puredocbench-gt-latest; download gt/puredocbench_gt_latest.tar.gz and check gt/latest.json for the exact revision and timestamp.

• 2026-06-14: Updated GT annotations and opened the GT Review app for community corrections.

• 2026-05-08: Initial public release of PureDocBench, including the paper PDF and full dataset on Hugging Face.

GT Annotation Examples

The examples below show colored coordinate boxes over clean rendered pages from an academic paper, a patent form, and a tuition invoice.

At A Glance

Diagnostics

The diagnostic panel shows where current systems still have headroom. Formula recognition is the largest single bottleneck, and real degradation changes rankings more sharply than digital degradation.

Case Studies

The four case studies below are all taken from the paper. They show failures that aggregate scores can hide: notation loss, reading-order mistakes, annotation contamination, table-structure errors, character-level corruption, and missing visual authentication cues.

Case 1: Academic

Case 2: Business

Case 3: Finance

Case 4: Certificate

Appendix Highlights

The appendix documents the degradation design, per-category behavior, and source-validity checks used to make the benchmark reproducible.

Download

The full image/GT/HTML release is hosted on Hugging Face:

# After downloading all files from Hugging Face:
shasum -a 256 -c SHA256SUMS.txt
cat pdb_full.tar.part-* | tar -xf -

Latest GT-only annotations are available separately:

• gt/puredocbench_gt_latest.tar.gz
• gt/puredocbench_gt_latest.sha256
• gt/latest.json
• gt/changed_cases.json

If you already downloaded an older GT archive, replace it with gt/puredocbench_gt_latest.tar.gz.

Verify the split archive and reconstructed release:

python scripts/verify_split_archive.py /path/to/downloaded/files

python scripts/validate_release_manifest.py \
  --release-root /path/to/puredocbench \
  --manifest manifests/release_manifest_candidate_1475.csv

GT Review

Stable GT alias: puredocbench-gt-latest (gt/puredocbench_gt_latest.tar.gz). The exact revision is recorded in gt/latest.json; dates are kept as provenance metadata rather than embedded in public URLs or directory names. Use the review app to inspect annotations and export correction patches.

• Public review app: Open GT Review App
• Repository file: review/index.html
• Correction guide: docs/ANNOTATION_CORRECTIONS.md
• Submit a correction: New GT annotation correction issue (English or Chinese)

Local launch:

mkdir -p review/assets
ln -s /path/to/puredocbench/images/clean review/assets/images
python3 -m http.server 8767 --directory review

Open:

http://127.0.0.1:8767/index.html

Static app URL:

https://zhihengli-casia.github.io/PureDocBench/review/

The GitHub repository does not include the full image release. For visual review on GitHub Pages, click Load Images and select the downloaded images/clean folder. Local launch can also use the symlink above.

GT Coordinates

If you need spatial labels, regenerate clean-render coordinates from the HTML/CSS sources:

python scripts/add_gt_coordinates.py \
  --release-root /path/to/puredocbench \
  --manifest manifests/release_manifest_candidate_1475.csv \
  --in-place \
  --include-bbox \
  --include-coordinate-system \
  --report coordinate_report.json

python scripts/validate_release_manifest.py \
  --release-root /path/to/puredocbench \
  --manifest manifests/release_manifest_candidate_1475.csv \
  --require-coordinates \
  --require-bbox

The script follows the OmniDocBench GT convention and adds a rectangular poly field to each layout_dets item. poly is a flat list of clean-image pixel coordinates in top-left, top-right, bottom-right, bottom-left order: [x1, y1, x2, y1, x2, y2, x1, y2]. A derived bbox: [x1, y1, x2, y2] can also be written with --include-bbox, but poly is the primary coordinate field. Run playwright install chromium first if the Playwright browser is not installed, or pass --browser-channel chrome to use a local Chrome installation.

Inference And Scoring

PureDocBench includes a public CLI for model-agnostic inference, lightweight scoring, and OmniDocBench export:

pip install -e .

puredocbench infer \
  --images /path/to/puredocbench/images/clean \
  --output-dir predictions/my_model_clean \
  --command-template 'python my_model_infer.py --image {image} --out {output}'

puredocbench score \
  --release-root /path/to/puredocbench \
  --manifest manifests/release_manifest_candidate_1475.csv \
  --pred-dir predictions/my_model_clean \
  --track clean \
  --out-dir scores/my_model_clean

See docs/INFERENCE_SCORING.md for the full interface and OmniDocBench export path.

Repository Contents

manifests/                         Release and sample manifests
metadata/                          Dataset card and Croissant metadata
scripts/                           Rendering, degradation, validation, leaderboard tools
puredocbench/                      Public inference, scoring, and OmniDocBench export CLI
model_inference/                   Sanitized model inference configs and runners
supplemental_inference_scoring/    API/local inference and scoring utilities
assets/figures/                    Figures from the paper
paper/                             Paper PDF

Quick Start

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
playwright install chromium

Render one HTML page:

python scripts/render_single_image.py \
  --html /path/to/page.html \
  --out /path/to/page.png \
  --dpi 300

Apply a deterministic degradation profile:

python scripts/apply_degradation_ablation.py \
  --input /path/to/clean_images \
  --output /path/to/degraded_images \
  --profile full_medium

License

• Dataset assets are released under CC BY 4.0; see LICENSE_DATA.
• Code in this repository is released under the license in LICENSE.
• Model weights are not redistributed.

Citation

@misc{puredocbench,
  title        = {How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings},
  author       = {Li, Zhiheng and collaborators},
  year         = {2026},
  howpublished = {\url{https://github.com/zhihengli-casia/puredocbench}},
  note         = {Dataset and benchmark release}
}