SPARTAN — Structured Parsing and Relevant Table Analysis

Heuristic, training-free table detection and extraction for PDF documents. No GPU, no model training, fully explainable.

Open source and open to contributions. Bug reports, difficult sample documents, new OCR engines and better heuristics are all welcome — see Contributing.

Reference implementation of:

Nandurbarkar, S., Chaudhari, A. Y. & Mulla, R. SPARTAN: automated table detection and extraction from documents using advanced OpenCV heuristics and OCR techniques. Scientific Reports 16, 14954 (2026). https://doi.org/10.1038/s41598-026-44325-7

---

Highlights

• Exact text for born-digital PDFs. Characters come from the PDF text layer and are mapped onto the detected cell grid, so part numbers are reproduced exactly (no 0/O or 1/I OCR confusions). OCR (Tesseract, offline) is used only for scanned pages or for regions that have no text layer.
• Bordered, borderless and fill-delimited tables: morphological boundary detector, alignment-based borderless detector with pseudo boundaries, and adaptive thresholding for tables whose cells are delimited only by background colours.
• Complex structure: nested (multi-level) headers, row/column-spanning cells, title rows, section rows, key-value tables and form grids, header-less list tables, and tables drawn as several separate boxes.
• Document-level post-processing: tables continuing across pages are merged (repeated header rows removed), and repeated page headers/footers are detected and de-duplicated.
• Optional PCN schema mapping (the paper's "LLM hook"). Deterministic keyword matching first; a local LLM through Ollama (or Azure OpenAI) fills the remaining gaps.
• CLI, Python API and REST API (FastAPI); JSON and CSV output; debug visualisations.

Installation

Python 3.10 or newer.

pip install spartan-tables                  # core: born-digital PDFs
pip install "spartan-tables[ocr]"           # + Tesseract OCR for scanned pages
pip install "spartan-tables[all]"           # + OCR, REST API, local LLM, memory stats

Extras: ocr, api, llm, azure, metrics, all. The package is imported as spartan.

From source (for development):

git clone https://github.com/shlok-n/spartan.git spartan && cd spartan
python -m venv .venv && source .venv/bin/activate     # Windows: .venv\Scripts\activate
pip install -e ".[all,dev]"                           # or: pip install -r requirements.txt

Optional components:

Born-digital PDFs need neither OCR nor an LLM.

Configuration and credentials

Runtime settings are read from environment variables prefixed with SPARTAN_, or from a .env file in the working directory:

cp .env.example .env      # then edit .env and add your own values

The repository contains no credentials. Cloud services are optional; to use them, put your own endpoint and key in .env (git-ignored). For example:

SPARTAN_AZURE_VISION_ENDPOINT=https://<your-resource>.cognitiveservices.azure.com/
SPARTAN_AZURE_VISION_KEY=<your-azure-vision-key>

If a selected service is not configured, SPARTAN logs a warning explaining which variables to set and continues without it. See .env.example for every setting.

OCR engines

OCR is only needed for pages without a text layer (scans) or with --text-source ocr. Choose the engine with --ocr-engine (or PipelineConfig(ocr_engine=...)):

--ocr-scope controls how OCR is applied: page (one call per page, words assigned to cells geometrically; the default) or the paper's cell strategies single, multi (cells stacked in chunks of --chunk-size with the row-separator image) and auto.

Using another OCR engine

Any object with a name, image_to_words(image) and image_to_text(image) works (image is a BGR NumPy array). examples/custom_ocr_engine.py wraps EasyOCR in about 40 lines. Use it without modifying the package:

python -m spartan --file scan.pdf --ocr-engine examples.custom_ocr_engine:EasyOcrEngine

or register it under a short name from Python:

from spartan import PipelineConfig, process_pdf
from spartan.ocr import register_ocr_engine
from examples.custom_ocr_engine import EasyOcrEngine

register_ocr_engine("easyocr", EasyOcrEngine)
document = process_pdf("scan.pdf", PipelineConfig(ocr_engine="easyocr"))

Quick start

# Standard profile (pre-processing + region segmentation)
python -m spartan --file notice.pdf

# Every PDF in a folder, with CSV export and debug images
python -m spartan --file ./pdfs --csv --save

# Multi-column documents (e.g. scientific papers)
python -m spartan --file paper.pdf --preprocess --column --region

# Map to the PCN schema (Ollama by default, deterministic only with --llm-provider none)
python -m spartan --file notice.pdf --pcn

Results are written to output/<document>/<document>.json (plus csv/table_<n>.csv and <document>_pcn.json when requested).

Using SPARTAN in your code

import spartan

document = spartan.process_pdf("notice.pdf")
for table in document["tables"]:
    print(table["title"], table["columns"], len(table["rows"]))

Tables load directly into pandas:

import pandas as pd

frames = [pd.DataFrame(t["rows"], columns=t["columns"]) for t in document["tables"]]

Options are set with PipelineConfig (all fields have sensible defaults):

from spartan import PipelineConfig, SpartanPipeline, TextSourceMode

config = PipelineConfig(
    column_detection=True,               # multi-column layouts
    text_source=TextSourceMode.AUTO,     # PDF text layer, OCR only where needed
    ocr_engine="tesseract",              # or "azure", "none", "package.module:Class"
    dpi=300,
)
pipeline = SpartanPipeline(config)       # reuse one pipeline for many files
for path in ["a.pdf", "b.pdf"]:
    document = pipeline.process(path)

process_pdf returns a JSON-serialisable dict (format below). SpartanPipeline.process also includes the internal Table objects under document["_tables"], and SpartanPipeline.process_and_save writes JSON/CSV files like the CLI does.

CLI reference (paper Table 2)

As in the paper, a step runs only when its flag is given. Without any step flag, the standard profile --preprocess --region (the evaluation configuration) is used.

Output format

{
  "document_name": "Analog Devices",
  "tables": [{
    "id": "table_2",
    "pages": [1, 2],                      // merged across the page break
    "type": "normal",                     // normal | nested | key_value
    "title": "PDN 23_0061 Discontinued Parts Material List",
    "title_source": "table",              // table | context | none
    "columns": ["Model", "Product Family", "Replacement Part", "Pin To Pin Compatible", "Comments"],
    "header_rows": 1,
    "rows": [["ADAU1328BSTZ", "ADAU1328", "AD1938WBSTZ", "Yes", "None"], ...],
    "merged_cells": [{"row": 4, "col": 2, "row_span": 2, "col_span": 1, "merge_type": "row", "text": "..."}],
    "pairs": [...],                       // key_value tables only
    "column_widths": [...], "detection_method": "boundary", "text_source": "pdf_text_layer",
    "sources": [{"page": 1, "segment": "table_2", "bbox": [...]}, ...]
  }],
  "text": [{"page": 1, "id": "pre_text", "kind": "pre_text", "text": "...", "boilerplate": false}],
  "full_text": "...",
  "pages": [...],                         // per-page segments in reading order, timings
  "processing": {"pages": 2, "seconds": 2.4, "seconds_per_page": 1.2, "peak_rss_mb": 185.0},
  "pcn": {...}                            // only with --pcn
}

Merged (spanning) cells are expanded: their value is repeated in every row and column they cover, as described in the paper, and listed in merged_cells.

REST API

uvicorn spartan.api:app --host 127.0.0.1 --port 8000     # or: python -m spartan.api

Uploads must be PDFs (checked by content) and are limited to SPARTAN_MAX_UPLOAD_MB. Errors are returned as {"error": "..."}.

PCN schema mapping

spartan/assets/fixed_schema.json defines the output fields and spartan/assets/configuration.json lists example labels for each field. Mapping order:

1. Table columns are matched to the table fields (Impacted_Part, Replacement_Part, Pin_to_Pin_Match). Part fields also require the column content to look like part numbers. Repeated column groups (EOL Devices | Replacements | EOL Devices | …) are unrolled.
2. Tables without matching columns, but with a title such as "Affected Part Numbers", contribute all their cells to Impacted_Part.
3. Scalar fields come from key-value tables, matching table columns and Label: value lines. Dates must be parseable and are normalised to YYYY-MM-DD.
4. Only what is still missing is sent to the LLM, with JSON-schema constrained output at temperature 0.

How each value was obtained is recorded in document["pcn_provenance"].

Architecture

Testing

python -m pytest

Unit tests use synthetic images and need no external services. End-to-end tests run on real PCN/PDN PDFs from ../Dataset (or the folder in SPARTAN_DATASET_DIR) and are skipped when it is absent; the documents are third-party and are not distributed with this repository.

Performance (23 PCN/PDN PDFs, ~90 pages, CPU only)

About 0.3–1.5 s per page (the first pages include start-up), and about 280 MB peak memory, without OCR. Measured on the development laptop. The LLM step adds roughly a minute per document with an 8B model on CPU.

Known limitations

• Scanned pages need the Tesseract binary; without it, tables on such pages are detected but their cells stay empty (a warning is logged).
• Heuristics assume roughly upright pages; heavily skewed scans are not de-skewed.
• Word splitting relies on glyph geometry; PDFs that break a text run in the middle of a word can still produce a stray split (e.g. "Withdrawa l").
• Small borderless blocks whose first row is a set of Label: headers can be misread as key-value tables.
• The borderless detector needs at least three aligned rows and two columns.

Contributing

Contributions of any size are welcome, and you don't need permission to start:

• Found a table SPARTAN gets wrong? Open an issue with the document or a screenshot. Hard examples are the most valuable contribution.
• Want to code? Fork the repository, make your change with a test, and open a pull request. CONTRIBUTING.md explains the setup and lists open tasks, from small fixes to de-skewing and benchmark evaluation.
• Using SPARTAN somewhere? Let us know in an issue or discussion; it helps prioritise.

Please follow the Code of Conduct. Report security issues privately as described in SECURITY.md.

License

Released under the MIT License.

Citation

If you use SPARTAN in your work, please cite the paper (see also CITATION.cff):

@article{nandurbarkar2026spartan,
  title   = {{SPARTAN}: automated table detection and extraction from documents using
             advanced {OpenCV} heuristics and {OCR} techniques},
  author  = {Nandurbarkar, S. and Chaudhari, A. Y. and Mulla, R.},
  journal = {Scientific Reports},
  volume  = {16},
  pages   = {14954},
  year    = {2026},
  doi     = {10.1038/s41598-026-44325-7}
}