NOVA

NOVA extracts evidence from PDFs, proposes traceable filenames, and routes uncertain documents to review. It is designed as a handoff-friendly document-processing pipeline: original files are copied by default, every decision cites source pages, and extraction/model upgrades are isolated behind interfaces.

The default extractor is portable pypdf with optional page-level Tesseract OCR. NVIDIA NeMo Retriever 26.5 can be enabled for page elements, tables, charts, infographics, and complex layouts without changing naming or processing code.

Bundled NeMo Retriever

The upstream NVIDIA NeMo Retriever repository is included under nemo_retriever/, with its history retained in this repository. NVIDIA's production container remains the root Dockerfile; the smaller document-naming container is docker/docnaming.Dockerfile.

What changed from the original script

• No user-specific paths or import-time filesystem writes
• Full-document, page-level extraction instead of a three-page text snippet
• Stable JSON schema for text, table, chart, image, and infographic elements
• Content hashes, page provenance, evidence IDs, confidence, and review reason codes
• Immutable copy mode by default; destructive move mode is explicit
• Optional NeMo Retriever and Ollama adapters with lightweight fallbacks
• Repeatable CLI, evaluation harness, tests, CI, container definition, and handoff guide
• The original process_pdfs.py remains available as a legacy compatibility path

Requirements

• Python 3.10-3.12 for the core package
• Python 3.12 for NeMo Retriever 26.5
• Poppler and Tesseract only when using OCR fallback
• An NVIDIA GPU and the appropriate NVIDIA services/credentials for accelerated NeMo extraction

Do not send sensitive or controlled documents to hosted endpoints without authorization. API keys belong in environment variables, never in this repository. The auto backend therefore remains on local pypdf unless DOCNAMING_ALLOW_NEMO_AUTO=true is explicitly set. --extractor nemo is also an explicit opt-in.

Install

Core pipeline:

py -3.12 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"

Add OCR support:

python -m pip install -e ".[ocr,dev]"

Add NeMo Retriever on a compatible GPU environment:

python -m pip install -e .\nemo_retriever
python -m pip install -e ".[dev]"

To use the supported PyPI release instead of the bundled source, install .[nemo,dev].

NVIDIA's production stack is primarily Linux/container oriented. On Windows, use WSL2 or a Linux GPU host for NeMo and retain Windows-native pypdf/Tesseract as the fallback.

Configure

Copy .env.example values into your shell, deployment environment, or service configuration. The application does not automatically load .env files so secrets cannot be imported unexpectedly.

The safe project-local defaults are:

data/inbox
data/processed
data/needs-review
data/artifacts

Check readiness:

docnaming doctor

Use

Process the configured inbox:

docnaming --extractor auto process

Process specific files or folders:

docnaming --extractor pypdf process C:\scans\one.pdf C:\scans\batch

Inspect extraction without copying a PDF:

docnaming --extractor nemo extract C:\scans\complex.pdf --json-output complex.extraction.json

Watch the inbox:

docnaming --extractor auto watch --poll-interval 5 --settle-seconds 2

Watch mode waits for each PDF's size and modification time to stabilize and retries transient failures rather than permanently marking the file as processed.

Run the evaluation corpus:

docnaming --extractor pypdf evaluate .\evals\cases.local.jsonl --report evaluation-report.json

Move mode is intentionally explicit:

docnaming --move-source process C:\scans\approved-batch

Outputs

For each document, the pipeline produces:

• artifacts/extractions/<sha256>.json: normalized document elements
• artifacts/decisions/<sha256>.json: proposed name, confidence, evidence pages, and reason codes
• artifacts/manifest.jsonl: append-only processing history
• A copied PDF in processed or needs-review

Reason codes:

• P01_PLACEHOLDER_TITLE: no meaningful title
• P02_TITLE_NOT_SUPPORTED: proposed title is weakly supported by evidence
• P04_DATE_NOT_FOUND: required date was not found
• P05_LOW_CONFIDENCE: confidence is below the configured threshold

Low extraction quality lowers confidence. The pipeline does not treat poor OCR as permission to trust an unsupported model output.

NeMo Retriever integration

The adapter follows the NeMo Retriever 26.5 graph API and enables text, tables, charts, infographics, and table structure using the pdfium layout path. Because NeMo can call hosted endpoints, auto selects it only when both the package is installed and DOCNAMING_ALLOW_NEMO_AUTO=true; otherwise it stays on pypdf. An auto-selected NeMo failure is recorded and retried with pypdf, while explicit --extractor nemo failures remain visible.

The adapter normalizes NeMo results into the internal schema. Before a deployment upgrade, run fixture tests and a representative PDF corpus because NVIDIA may change result fields between releases.

See Architecture, NeMo setup, and Handoff runbook.

Development

$env:PYTHONPATH = "$PWD\src"
python -m pytest

The committed training_dataset.json and naming_examples.csv contain useful historical naming examples. They are not a complete extraction or retrieval benchmark; use evals/ for a fixed, reviewer-approved corpus.