FinixDocBench

Language: English | 中文

This repository contains a compliance-reviewed public subset of FinixDocBench, the financial-domain document parsing benchmark introduced in the technical report "FinixDoc: Rethinking Financial Document Parsing Beyond Saturated Benchmarks".

The benchmark focuses on document parsing conditions that are common in real financial workflows but underrepresented in saturated clean-document benchmarks: digitally native insurance clauses, noisy camera-captured medical receipts, ultra-long pages, and very large dense tables. The expected model outputs are page-level Markdown and, where available, structured JSON layout annotations.

Project links:

• Project page: https://finix.alipay.com/
• Technical report: https://arxiv.org/abs/2608.22842
• Hugging Face dataset: https://huggingface.co/datasets/inclusionAI/FinixDocBench
• License: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International

Released Subset Contents

The broader FinixDocBench benchmark contains 5,000 pages across five tracks. This public release contains 742 page samples. Track 3 is split into two directories so that ultra-long pages and large-table pages can be evaluated separately.

The released files are organized as follows:

The FinixDigital package here is a 242-page insurance-terms subset of the broader 500-page FinixDigital track discussed in the technical report.

Repository Structure

FinixDocBench/
  README.md
  README_zh.md
  LICENSE.md
  CITATION.cff
  dataset_manifest.jsonl
  metadata.jsonl
  matrix.pdf
  matrix.png
  track1_finixdigital_242_insurance_terms/
    images/
    mds/
    jsons/
  track2_finixphoto_300/
    images/
    mds/
    jsons/
  track3_finixhuge_100_long/
    images/
    mds/
  track3_finixhuge_100_table/
    images/
    mds/
  FinixDocBench_Eval_for_Markdown/
    README.md
    requirements.txt
    run_eval.py
    finixdoc_md_eval/

Each sample is matched by file stem. For example, abc123.png, abc123.md, and abc123.json describe the same page when all three files are present.

The dataset_manifest.jsonl file provides one row per sample with relative paths, track metadata, image dimensions, and basic annotation counts. It is intended as a lightweight index for users who want to load the release programmatically.

Tasks

This FinixDocBench release supports three complementary task settings.

1. Full-Page Markdown Parsing

Given a page image, a model should produce a complete page-level Markdown reconstruction. This task is available for all public tracks.

The Markdown ground truth preserves text order, headings, tables, and other page-level structure. HTML <table> blocks are used where table structure, merged cells, or dense financial layouts need to be represented more faithfully than plain Markdown tables.

2. Structured Layout Parsing

Given a page image, a model should produce structured page elements with category labels, bounding boxes, transcribed content, and reading order. This task is available for FinixDigital and FinixPhoto, which include jsons/ annotations.

The public JSON files use pixel-space bounding boxes in the original image coordinate system. Each JSON file includes page metadata plus a layout list.

3. Ultra-Large Page Processability

FinixHuge-Long and FinixHuge-Table evaluate whether a system can return a syntactically valid, non-empty, page-level Markdown result for oversized documents. These pages stress page resolution, output length, table complexity, and reading-order preservation.

Because FinixHuge is Markdown-only in this release, it is best evaluated with Markdown metrics plus a success-rate style processability check.

Annotation Schema

FinixDigital and FinixPhoto use a unified 10-class page-element schema:

page-header
page-footer
title
section-header
text
table
figure
caption
footnote
other

Top-level JSON fields:

Each layout item contains:

Example:

{
  "width": 993,
  "height": 1404,
  "resized_width": 992,
  "resized_height": 1408,
  "max_pixels": 16777216,
  "min_pixels": 4096,
  "layout": [
    {
      "category": "section-header",
      "bbox": [82, 364, 223, 394],
      "content": "## 2.3 责任免除",
      "order": 5
    },
    {
      "category": "table",
      "bbox": [337, 156, 916, 295],
      "content": "<table>...</table>",
      "order": 3
    }
  ]
}

Dataset Statistics

Category counts for the structured JSON tracks:

Loading Examples

Load the manifest with the Hugging Face datasets library:

from datasets import load_dataset

manifest = load_dataset(
    "json",
    data_files="dataset_manifest.jsonl",
    split="train",
)

print(manifest[0])

Read a sample locally after cloning the repository:

from pathlib import Path
from PIL import Image
import json

repo = Path("FinixDocBench")
row = manifest[0]

image = Image.open(repo / row["image_path"])
markdown = (repo / row["markdown_path"]).read_text(encoding="utf-8")

annotation = None
if row["json_path"] is not None:
    annotation = json.loads((repo / row["json_path"]).read_text(encoding="utf-8"))

Evaluation

The repository includes a lightweight Markdown evaluator:

cd FinixDocBench_Eval_for_Markdown
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Run evaluation on a track by providing a ground-truth Markdown directory and a prediction Markdown directory with matching file names:

python run_eval.py \
  --gt_dir ../track2_finixphoto_300/mds \
  --pred_dir /path/to/predicted_mds \
  --output_json outputs/finixphoto_result.json

The Markdown evaluator reports:

The overall score is:

overall = ((1 - text_block_Edit_dist) * 100
         + (1 - reading_order_Edit_dist) * 100
         + table_TEDS) / 3

For FinixHuge, users should additionally report a success rate: the fraction of pages for which the system returns a syntactically valid, non-empty page-level Markdown result without runtime failure, severe truncation, or format errors that prevent downstream evaluation.

Structured JSON annotations are provided for FinixDigital and FinixPhoto. This repository currently ships the Markdown evaluator; if you report structured layout metrics, please describe the evaluator, matching rules, and coordinate convention used.

Reference Results from the Technical Report

The following values are copied from the FinixDoc technical report for context. They correspond to the benchmark protocol reported in the paper and should not be treated as precomputed scores for every subset in this release unless the same split and evaluation protocol are reproduced.

FinixDigital

FinixPhoto

FinixHuge

Intended Uses

This dataset is intended for:

• Evaluating OCR and document parsing systems on financial-domain documents.
• Testing full-page Markdown reconstruction.
• Testing layout parsing, table parsing, bounding boxes, and reading-order recovery on FinixDigital and FinixPhoto.
• Measuring robustness on noisy camera-captured receipt images.
• Evaluating end-to-end processability on ultra-large document pages.

Out-of-Scope Uses

This dataset is not intended for:

• Individual profiling or personal information extraction.
• Automated financial, medical, insurance, legal, employment, credit, or similarly consequential decision-making.
• Reporting benchmark numbers after using benchmark labels or ground truth for training, fine-tuning, data augmentation, or prompt optimization.
• Claiming complete coverage of all financial document scenarios.

Limitations

FinixDocBench is an evaluation benchmark, not a comprehensive training corpus. This release covers selected high-value financial document parsing scenarios and does not include the private FinixInner track.

FinixPhoto is derived from public-scenario medical receipt sources and re-annotated under the FinixDocBench schema. Prior exposure of some external models to the original public sources cannot be fully ruled out.

FinixHuge emphasizes system-level processability with Markdown-only public annotations. Direct single-pass model comparisons may understate or overstate practical usability if failed pages, truncation, or invalid outputs are not reported consistently.

Some page images may be very large. Users should use image loading libraries carefully and configure decompression or pixel limits intentionally when evaluating FinixHuge.

License

This FinixDocBench release is distributed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0).

See LICENSE.md for the human-readable license notice and the official Creative Commons license link.

Citation

If you use this FinixDocBench release, please cite:

@misc{wang2026finixdoc,
  title        = {FinixDoc: Rethinking Financial Document Parsing Beyond Saturated Benchmarks},
  author       = {Hang Wang and Jin Zhang and Guoliang Xu and Pengyue Lu and Yao Li and Zijiao Zhang and Tianyu Huang and Weiqi Xiong and Yulong Wang and Chuqiao Lu and Wenkang Huang and Kai Yang and Yadong Li and Hui Li and Xingzhong Xu and Xiao Xu},
  year         = {2026},
  institution  = {Ant Group},
  url          = {https://finix.alipay.com}
}

Contact

For questions about the benchmark, please contact the FinixDoc authors through the project page or the Ant Group Hugging Face organization.