Scene Text Recognition Recommendations

Everything about Scene Text Recognition

SOTA • Papers • Datasets • Code • Our Framework

Contents

• 1. Papers
• 2. Datasets
  • 2.1 Synthetic Datasets
  • 2.2 Benchmarks
• 3. Public Code
  • 3.1 Frameworks
  • 3.2 Algorithms
• 4. SOTA

---

Checkout Our New Work！

Revisiting Scene Text Recognition: A Data Perspective

---

1. Papers

All Papers Can be Find Here

• Latest Papers:

up to (2023-11-29)

• ICCV-2023: Self-supervised Character-to-Character Distillation for Text Recognition

up to (2023-8-11)

• ICCV-2023: A Benchmark for Chinese-English Scene Text Image Super-resolution

up to (2023-7-25)

• ACMMM-2023: Relational Contrastive Learning for Scene Text Recognition
• arXiv-2023: HiREN: Towards Higher Supervision Quality for Better Scene Text Image Super-Resolution
• arXiv-2023: Context Perception Parallel Decoder for Scene Text Recognition
• IJCAI-2023: Towards Robust Scene Text Image Super-resolution via Explicit Location Enhancement

up to (2023-7-25)

• ICCV-2023: MRN: Multiplexed Routing Network for Incremental Multilingual Text Recognition

up to (2023-7-20)

• ICCV-2023: Revisiting Scene Text Recognition: A Data Perspective
• arXiv-2023: DiffusionSTR: Diffusion Model for Scene Text Recognition

up to (2023-6-1)

• arXiv-2023:GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generation
• arXiv-2023:TextDiffuser: Diffusion Models as Text Painters
• arXiv-2023:DiffUTE: Universal Text Editing Diffusion Model
• arXiv-2023:GlyphControl: Glyph Conditional Control for Visual Text Generation

up to (2023-5-16)

• IJCAI-2023:TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition
• IJCAI-2022:Linguistic More: Taking a Further Step toward Efficient and Accurate Scene Text Recognition
• ICDAR-2023:Scene Text Recognition with Image-Text Matching-guided Dictionary
• arXiv-2023:Improving Scene Text Recognition for Character-Level Long-Tailed Distribution

up to (2023-3-16)

• arXiv-2023:CLIPTER: Looking at the Bigger Picture in Scene Text Recognition
• ECCVW-2022:On calibration of scene-text recognition models
• Others:STR transformer: a cross-domain transformer for scene text recognition
• TIP-2023:Text prior guided scene text image super-resolution
• Neurocomputing-2023:DPF-S2S: A novel dual-pathway-fusion-based sequence-to-sequence text recognition model
• WACV-2023:Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition
• PR-2023:Towards open-set text recognition via label-to-prototype learning

up to (2022-12-29)

• BMVC-2022:Visual-semantic transformer for scene text recognition
• BMVC-2022:Parallel and Robust Text Rectifier for Scene Text Recognition
• ICFHR-2022:A Vision Transformer Based Scene Text Recognizer with Multi-grained Encoding and Decoding
• ECCV-2022:TextAdaIN: Paying Attention to Shortcut Learning in Text Recognizers

up to (2022-11-1)

• arxiv-2022: IterVM: Iterative Vision Modeling Module for Scene Text Recognition
• Applied intelligence:Scene text recognition based on two-stage attention and multi-branch feature fusion module
• ICPR-2022: Portmanteauing Features for Scene Text Recognition
• ECCV-2022: Pure Transformer with Integrated Experts for Scene Text Recognition
• BMCV-2022: Masked Vision-Language Transformers for Scene Text Recognition

up to (2022-11-1)

• AAAI-2022:Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition
• ECCV-2022:Background-Insensitive Scene Text Recognition with Text Semantic Segmentation
• ACCESS-2022:Scene Text Recognition with Semantics
• TIP-2022:PETR: Rethinking the Capability of Transformer-Based Language Model in Scene Text Recognition
• TMM-2022:Dual Relation Network for Scene Text Recognition

up to (2022-9-20)

• ECCV-2022:Levenshtein OCR
• ECCV-2022:Multi-Granularity Prediction for Scene Text Recognition
• arXiv-2022:A Scene-Text Synthesis Engine Achieved Through Learning from Decomposed Real-World Data
• arXiv-2022:Scene Text Recognition with Single-Point Decoding Network
• ECCV-2022-Technical-Report:Vision-Language Adaptive Mutual Decoder for OOV-STR
• WACV-2023:Seq-UPS: Sequential Uncertainty-aware Pseudo-label Selection for Semi-Supervised Text Recognition
• ECCV-2022-Technical-Report:1st Place Solution to ECCV 2022 Challenge on Out of Vocabulary Scene Text Understanding: End-to-End Recognition of Out of Vocabulary Words
• ECCV-2022-Technical-Report:Runner-Up Solution to ECCV 2022 Challenge on Out of Vocabulary Scene Text Understanding: Cropped Word Recognition

up to (2022-8-9)

• ECCV-2022:Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognition

up to (2022-7-24)

• ECCV-2022:SGBANet: Semantic GAN and Balanced Attention Network for Arbitrarily Oriented Scene Text Recognition
• ECCV-2022:Scene Text Recognition with Permuted Autoregressive Sequence Models

up to (2022-7-9)

• arXiv-2022:MaskOCR: Text Recognition with Masked Encoder-Decoder Pretraining
• ACM-MM22:Reading and Writing: Discriminative and Generative Modeling for Self-Supervised Text Recognition

up to (2022-5-12)

• arXiv-2022:Multimodal Semi-Supervised Learning for Text Recognition
• IJCAI-2022:SVTR: Scene Text Recognition with a Single Visual Model

2. Datasets

All Datasets Can be Find Here

2.1 Synthetic Training Datasets

---

2.2 Benchmarks

---

2.3 Other Real Datasets

• The Real Datasets refer to this repo ku21fan/STR-Fewer-Labels

3 Public Code

3.1 Frameworks

PaddleOCR (百度)

• PaddlePaddle/PaddleOCR
• 特性 (截取至PaddleOCR)：
  • 使用百度自研深度学习框架PaddlePaddle搭建
  • PP-OCR系列高质量预训练模型，准确的识别效果
    • 超轻量PP-OCRv2系列：检测（3.1M）+ 方向分类器（1.4M）+ 识别（8.5M）= 13.0M
    • 超轻量PP-OCR mobile移动端系列：检测（3.0M）+方向分类器（1.4M）+ 识别（5.0M）= 9.4M
    • 通用PPOCR server系列：检测（47.1M）+方向分类器（1.4M）+ 识别（94.9M）= 143.4M
    • 支持中英文数字组合识别、竖排文本识别、长文本识别
    • 支持多语言识别：韩语、日语、德语、法语
    • 丰富易用的OCR相关工具组件
  • 半自动数据标注工具PPOCRLabel：支持快速高效的数据标注
    • 数据合成工具Style-Text：批量合成大量与目标场景类似的图像
    • 文档分析能力PP-Structure：版面分析与表格识别
    • 支持用户自定义训练，提供丰富的预测推理部署方案
    • 支持PIP快速安装使用
    • 可运行于Linux、Windows、MacOS等多种系统
• 支持算法(识别):
  • CRNN
  • Rosetta
  • STAR-Net
  • RARE
  • SRN
  • NRTR

---

MMOCR (OpenMMLab)

• open-mmlab/mmocr
• 特性(截取至MMOCR):
  • MMOCR 是基于 PyTorch 和 mmdetection 的开源工具箱，专注于文本检测，文本识别以及相应的下游任务，如关键信息提取。 它是 OpenMMLab 项目的一部分。
  • 该工具箱不仅支持文本检测和文本识别，还支持其下游任务，例如关键信息提取。
• 支持算法(识别)
  • ABINet (CVPR'2021)
  • CRNN (TPAMI'2016)
  • MASTER (PR'2021)
  • NRTR (ICDAR'2019)
  • RobustScanner (ECCV'2020)
  • SAR (AAAI'2019)
  • SATRN (CVPR'2020 Workshop on Text and Documents in the Deep Learning Era)
  • SegOCR (Manuscript'2021)

---

Deep Text Recognition Benchmark (ClovaAI)

• clovaai/deep-text-recognition-benchmark
• 特性:
  • Offical Pytorch implementation of What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis
  • 可自定义四阶段组件，如CRNN，ASTER
  • 容易上手，推荐使用

---

DAVAR-Lab-OCR (海康威视)

• hikopensource/DAVAR-Lab-OCR
• 特性:
  • 基于mmocr搭建，复现了一些算法，同时将来会用于海康自研算法开源
• 支持算法(识别)
  • Attention(CVPR 2016)
  • CRNN(TPAMI 2017)
  • ACE(CVPR 2019)
  • SPIN(AAAI 2021)
  • RF-Learning(ICDAR 2021)

---

3.2. Algorithms

CRNN

• Lua, Offical, 1.9k⭐: bgshih/crnn
  • 官方实现版本，使用Lua
• Pytorch, 1.9k⭐: meijeru/crnn.pytorch
  • 推荐使用🀄
• Tensorflow, 972⭐:MaybeShewill-CV/CRNN_Tensorflow
• Pytorch, 1.4k⭐:Sierkinhance/CRNN_Chinese_Characters_Rec
  • 用于中文识别版本的CRNN

---

ASTER

• Tensorflow, official, 651⭐: bgshih/aster
  • 官方实现版本，使用Tensorflow
• Pytorch, 535⭐:ayumuymk/aster.pytorch
  • Pytorch版本，准确率相较原文有明显提升

---

MORANv2

• Pytorch, official, 572⭐:Canjie-Luo/MORAN_v2
  • MORAN v2版本。更加稳定的单阶段训练，更换ResNet做backbone，使用双向解码器

---

4. SOTAs

All the models are evaluated in a lexicon-free manner

Baek's Reimplementation Version