Document AI with Hugging Face Transformers

Document AI s a term that has become popular over the last 3 years. It defines machine learning models, tasks, and techniques to classify, parse, and extract information from documents in digital and print forms, like invoices, receipts, licenses, contracts, and business reports.

This repository contains different example and tutorials on how to get started with Document AI and Transformers. Below you can also find a compendium of available models, tasks, datasets and other resources.

Training

• fine-tuning donut with SROIE
• fine-tuning LayoutLM with FUNSD
• fine-tuning LiLT with FUNSD

Inference

• Donut
• LayoutLM
• LiLT

Data-processing

• convert FUNSD to donut document for vqa

Demos/Spaces

Community:

• fedihch/InvoiceReceiptClassifierDemo
• nielsr/LayoutLMv2-FUNSD
• katanaml/LayoutLMv2-CORD
• nielsr/TrOCR-handwritten
• keras-io/ocr-for-captcha
• nielsr/dit-document-layout-analysis
• PatrickTyBrown/LoanDocumentClassifier
• Theivaprakasham/layoutlmv2_invoice
• TMsp/invoice_processing_layoutlmv3_custom
• Epoching/DocumentQA
• impira/docquery

popular models are layoutlm.... and Donut which we will use today get a first impression of how you can build you own document AI System using Hugging Face Transformers.

Machine Learning Models (Transformers)

Below you can find a table of the currently available Transformers models, who are achieving state-of-the-art performance on Document AI tasks.

Tasks

Document AI includes the following use cases and tasks:

• document classification (image-classification)
• document parsing (form understanding & information extraction)
• visual question answering
• table detection/layout analysis
• optical character recognition (OCR)

Datasets

APIs and existing Solutuions

• Amazon Textract
• Google Cloud Document AI
• Azure Form Recognizer

Other Tools

• SynthDoG 🐶: Synthetic Document Generator

Resources

OCR-Free Document Understanding with Donut