SensenGao/Multimodal-RAG-Survey-For-Document

[ACL2026 Main] Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding

21

7 commits

updated Jun 6, 2026

See the code

README

Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding

ACL 2026 Main Conference

arXiv Project Page GitHub Stars

Sensen Gao1*, Shanshan Zhao2†, Xu Jiang3, Lunhao Duan4*, Yong Xien Chng3*,
Qing-Guo Chen2, Weihua Luo2, Kaifu Zhang2, Jia-Wang Bian1, Mingming Gong1,5†

1MBZUAI   2Alibaba International Digital Commerce Group   3Tsinghua University   4Wuhan University   5University of Melbourne

* Work done during an internship at Alibaba International Digital Commerce Group. † Corresponding authors.


Multimodal RAG for Document Understanding

This repository maintains a curated list of methods, datasets, and benchmarks for Multimodal Retrieval-Augmented Generation (RAG) in Document Understanding, based on our ACL 2026 survey. We will keep updating this list. Feel free to open an issue or PR if we missed any relevant work!

Updates

  • Ongoing - We keep this list actively maintained! If you have a recent paper, or notice any relevant work we missed, feel free to contact us or open an issue/PR — we'll include it in a future update.
  • 2026.04 - Paper accepted at ACL 2026 Main Conference!
  • 2025.10 - Paper available on arXiv.

Table of Contents

Overview

A taxonomy of multimodal RAG for document understanding: retrieval domain (open vs. closed), retrieval modality (image vs. image+text), retrieval granularity (page- vs. element-level), and hybrid graph-/agent-based enhancements.

Methods

Open-Domain Methods

MethodVenueModalityGranularityTrainingPaper
DSEEMNLP 2024ImagePageYesLink
ColPaliICLR 2025ImagePageYesLink
ColQwen2ICLR 2025ImagePageYesLink
VisRAGICLR 2025ImagePageYesLink
M3DocRAGPreprintImagePageNoLink
VisDoMRAGNAACL 2025Image+TextPageNoLink
GMECVPR 2025Image+TextPageYesLink
ViDoRAGEMNLP 2025Image+TextPageNoLink
HM-RAGACM MM 2025Image+TextPageNoLink
VDocRAGCVPR 2025ImagePageYesLink
VRAG-RLPreprintImageElementYesLink
CoRe-MMRAGACL 2025Image+TextPageYesLink
Light-ColPaliACL 2025ImagePageYesLink
MM-R5PreprintImagePageYesLink
SimpleDocPreprintImage+TextPageNoLink
DocVQA-RAPICIC 2025ImageElementNoLink
RL-QRPreprintImagePageYesLink
Patho-AgenticRAGPreprintImagePageYesLink
M2IO-R1PreprintImage+TextPageYesLink
mKG-RAGPreprintImage+TextElementYesLink
DB3Team-RAGPreprintImage+TextPageYesLink
PREMIREMNLP 2025Image+TextElementNoLink
CMRAGPreprintImage+TextPageNoLink
MoLoRAGEMNLP 2025ImagePageYesLink
SERVALPreprintImagePageNoLink
MetaEmbedPreprintImagePageYesLink
DocPrunerPreprintImagePageYesLink
RECONPreprintImage+TextElementNoLink
LAD-RAGPreprintImage+TextElementNoLink
HEAVENPreprintImagePageNoLink
MARAACM MM 2025ImageElementYesLink
HPC-ColPaliPreprintImagePageYesLink
RegionRAGPreprintImageElementYesLink
IndustryRAGEMNLP Industry 2025ImagePageNoLink
COLMATEEMNLP Industry 2025ImagePageYesLink
LILaCEMNLP 2025ImageElementNoLink
HKRAGPreprintImageElementYesLink
SLEUTHPreprintImagePageNoLink
SnappyPreprintImageElementNoLink

Closed-Domain Methods

MethodVenueModalityGranularityTrainingPaper
CREAMACM MM 2024Image+TextPageYesLink
SV-RAGICLR 2025ImagePageYesLink
FRAGPreprintImagePageNoLink
MG-RAGPreprintImage+TextElementNoLink
VisChunkPreprintImage+TextPageNoLink
MMRAG-DocQAPreprintImage+TextElementNoLink
ReDocRAGICDAR WML 2025ImagePageYesLink
DREAMACM MM 2025ImagePageYesLink
HEARACM MMW 2025Image+TextPageNoLink

Graph-based Methods

MethodVenueKey IdeaPaper
HM-RAGACM MM 2025Hierarchical multi-agent framework with graph databases for structured relation captureLink
mKG-RAGPreprintMultimodal knowledge graphs aligning entities across vision and textLink
DB3Team-RAGPreprintImage-indexed knowledge graphs for domain-specific retrievalLink
MoLoRAGEMNLP 2025Page graphs encoding logical connections via graph traversalLink
RECONPreprintGlobal multimodal document graph linking intra-page and inter-page relationsLink
LAD-RAGPreprintLayout-aware component graphs with dynamic traversalLink
LILaCEMNLP 2025Layered component graph with late interaction subgraph retrievalLink

Agent-based Methods

MethodVenueKey IdeaPaper
ViDoRAGEMNLP 2025Iterative agent workflow with exploration, summarization, and reflectionLink
HM-RAGACM MM 2025Hierarchical multi-agent with query decomposition and consistency votingLink
Patho-AgenticRAGPreprintTask decomposition and multi-turn search for pathology textbooksLink
HEARACM MMW 2025Closed-loop multi-agent reasoning with VLM-based document parsingLink
SLEUTHPreprintCoarse-to-fine agent filtering and distilling salient evidenceLink

Datasets & Benchmarks

Document Understanding Datasets

Dataset#Queries#Docs/ImagesContentPaper
TabFQuAD210210 (I)TableLink
PlotQA28.9M224K (I)ChartLink
DocVQA50K12,767 (I)Text, Table, ChartLink
VisualMRC30,56210,197 (I)Text, Table, ChartLink
TAT-DQA16,5582,758 (D)Text, Table, ChartLink
InfoVQA30K5.4K (I)Text, Table, ChartLink
ChartQA23.1K17.1K (I)ChartLink
ScienceQA21K7,803 (I)Text, Table, ChartLink
DUDE41,4914,974 (D)Text, Table, ChartLink
SlideVQA52K14.5K (I)SlideLink
ArXivQA100K16.6K (D)Text, Table, ChartLink
MMLongBench-Doc1,062130 (D)Text, Table, Chart, SlideLink
PaperTab393307 (D)Text, TableLink
FetaTab1,023878 (D)TableLink
SPIQA27K25.5K (D)Table, ChartLink
LongDocURL2,325396 (D)Text, Table, ChartLink

Multimodal RAG Benchmarks

Benchmark#Queries#Docs/ImagesContentIntroduced ByPaper
ViDoRe3.8K8.3K (D)Text, Table, ChartColPaliLink
VisR-Bench471226 (D)Text, Table, Chart, SlideVisR-BenchLink
M3DocVQA2,4413,368 (D)Text, Table, ChartM3DocRAGLink
VisDoMBench2,2711,277 (D)Text, Table, Chart, SlideVisDoMRAGLink
ViDoSeek1,142300 (D)Text, Table, ChartViDoRAGLink
OpenDocVQA206K43K (I)Text, Table, ChartVDocRAGLink
UniDoc-Bench1.6K70K (I)Text, Table, ChartUniDocLink
BBox-DocVQA32K4.4K (D)Text, Table, ChartBBox-DocVQALink

Citation

If you find this survey useful, please cite our paper:

@article{gao2025scaling,
      title={Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding},
      author={Sensen Gao and Shanshan Zhao and Xu Jiang and Lunhao Duan and Yong Xien Chng and Qing-Guo Chen and Weihua Luo and Kaifu Zhang and Jia-Wang Bian and Mingming Gong},
      year={2025},
      eprint={2510.15253},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.15253},
}

Contact

For questions or suggestions, feel free to open an issue or contact Sensen Gao.

Contributors

SensenGao

7 commits

SensenGao/Multimodal-RAG-Survey-For-Document

[ACL2026 Main] Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding

21

7 commits

updated Jun 6, 2026

See the code

README

Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding

ACL 2026 Main Conference

arXiv Project Page GitHub Stars

Sensen Gao1*, Shanshan Zhao2†, Xu Jiang3, Lunhao Duan4*, Yong Xien Chng3*,
Qing-Guo Chen2, Weihua Luo2, Kaifu Zhang2, Jia-Wang Bian1, Mingming Gong1,5†

1MBZUAI   2Alibaba International Digital Commerce Group   3Tsinghua University   4Wuhan University   5University of Melbourne

* Work done during an internship at Alibaba International Digital Commerce Group. † Corresponding authors.


Multimodal RAG for Document Understanding

This repository maintains a curated list of methods, datasets, and benchmarks for Multimodal Retrieval-Augmented Generation (RAG) in Document Understanding, based on our ACL 2026 survey. We will keep updating this list. Feel free to open an issue or PR if we missed any relevant work!

Updates

  • Ongoing - We keep this list actively maintained! If you have a recent paper, or notice any relevant work we missed, feel free to contact us or open an issue/PR — we'll include it in a future update.
  • 2026.04 - Paper accepted at ACL 2026 Main Conference!
  • 2025.10 - Paper available on arXiv.

Table of Contents

Overview

A taxonomy of multimodal RAG for document understanding: retrieval domain (open vs. closed), retrieval modality (image vs. image+text), retrieval granularity (page- vs. element-level), and hybrid graph-/agent-based enhancements.

Methods

Open-Domain Methods

MethodVenueModalityGranularityTrainingPaper
DSEEMNLP 2024ImagePageYesLink
ColPaliICLR 2025ImagePageYesLink
ColQwen2ICLR 2025ImagePageYesLink
VisRAGICLR 2025ImagePageYesLink
M3DocRAGPreprintImagePageNoLink
VisDoMRAGNAACL 2025Image+TextPageNoLink
GMECVPR 2025Image+TextPageYesLink
ViDoRAGEMNLP 2025Image+TextPageNoLink
HM-RAGACM MM 2025Image+TextPageNoLink
VDocRAGCVPR 2025ImagePageYesLink
VRAG-RLPreprintImageElementYesLink
CoRe-MMRAGACL 2025Image+TextPageYesLink
Light-ColPaliACL 2025ImagePageYesLink
MM-R5PreprintImagePageYesLink
SimpleDocPreprintImage+TextPageNoLink
DocVQA-RAPICIC 2025ImageElementNoLink
RL-QRPreprintImagePageYesLink
Patho-AgenticRAGPreprintImagePageYesLink
M2IO-R1PreprintImage+TextPageYesLink
mKG-RAGPreprintImage+TextElementYesLink
DB3Team-RAGPreprintImage+TextPageYesLink
PREMIREMNLP 2025Image+TextElementNoLink
CMRAGPreprintImage+TextPageNoLink
MoLoRAGEMNLP 2025ImagePageYesLink
SERVALPreprintImagePageNoLink
MetaEmbedPreprintImagePageYesLink
DocPrunerPreprintImagePageYesLink
RECONPreprintImage+TextElementNoLink
LAD-RAGPreprintImage+TextElementNoLink
HEAVENPreprintImagePageNoLink
MARAACM MM 2025ImageElementYesLink
HPC-ColPaliPreprintImagePageYesLink
RegionRAGPreprintImageElementYesLink
IndustryRAGEMNLP Industry 2025ImagePageNoLink
COLMATEEMNLP Industry 2025ImagePageYesLink
LILaCEMNLP 2025ImageElementNoLink
HKRAGPreprintImageElementYesLink
SLEUTHPreprintImagePageNoLink
SnappyPreprintImageElementNoLink

Closed-Domain Methods

MethodVenueModalityGranularityTrainingPaper
CREAMACM MM 2024Image+TextPageYesLink
SV-RAGICLR 2025ImagePageYesLink
FRAGPreprintImagePageNoLink
MG-RAGPreprintImage+TextElementNoLink
VisChunkPreprintImage+TextPageNoLink
MMRAG-DocQAPreprintImage+TextElementNoLink
ReDocRAGICDAR WML 2025ImagePageYesLink
DREAMACM MM 2025ImagePageYesLink
HEARACM MMW 2025Image+TextPageNoLink

Graph-based Methods

MethodVenueKey IdeaPaper
HM-RAGACM MM 2025Hierarchical multi-agent framework with graph databases for structured relation captureLink
mKG-RAGPreprintMultimodal knowledge graphs aligning entities across vision and textLink
DB3Team-RAGPreprintImage-indexed knowledge graphs for domain-specific retrievalLink
MoLoRAGEMNLP 2025Page graphs encoding logical connections via graph traversalLink
RECONPreprintGlobal multimodal document graph linking intra-page and inter-page relationsLink
LAD-RAGPreprintLayout-aware component graphs with dynamic traversalLink
LILaCEMNLP 2025Layered component graph with late interaction subgraph retrievalLink

Agent-based Methods

MethodVenueKey IdeaPaper
ViDoRAGEMNLP 2025Iterative agent workflow with exploration, summarization, and reflectionLink
HM-RAGACM MM 2025Hierarchical multi-agent with query decomposition and consistency votingLink
Patho-AgenticRAGPreprintTask decomposition and multi-turn search for pathology textbooksLink
HEARACM MMW 2025Closed-loop multi-agent reasoning with VLM-based document parsingLink
SLEUTHPreprintCoarse-to-fine agent filtering and distilling salient evidenceLink

Datasets & Benchmarks

Document Understanding Datasets

Dataset#Queries#Docs/ImagesContentPaper
TabFQuAD210210 (I)TableLink
PlotQA28.9M224K (I)ChartLink
DocVQA50K12,767 (I)Text, Table, ChartLink
VisualMRC30,56210,197 (I)Text, Table, ChartLink
TAT-DQA16,5582,758 (D)Text, Table, ChartLink
InfoVQA30K5.4K (I)Text, Table, ChartLink
ChartQA23.1K17.1K (I)ChartLink
ScienceQA21K7,803 (I)Text, Table, ChartLink
DUDE41,4914,974 (D)Text, Table, ChartLink
SlideVQA52K14.5K (I)SlideLink
ArXivQA100K16.6K (D)Text, Table, ChartLink
MMLongBench-Doc1,062130 (D)Text, Table, Chart, SlideLink
PaperTab393307 (D)Text, TableLink
FetaTab1,023878 (D)TableLink
SPIQA27K25.5K (D)Table, ChartLink
LongDocURL2,325396 (D)Text, Table, ChartLink

Multimodal RAG Benchmarks

Benchmark#Queries#Docs/ImagesContentIntroduced ByPaper
ViDoRe3.8K8.3K (D)Text, Table, ChartColPaliLink
VisR-Bench471226 (D)Text, Table, Chart, SlideVisR-BenchLink
M3DocVQA2,4413,368 (D)Text, Table, ChartM3DocRAGLink
VisDoMBench2,2711,277 (D)Text, Table, Chart, SlideVisDoMRAGLink
ViDoSeek1,142300 (D)Text, Table, ChartViDoRAGLink
OpenDocVQA206K43K (I)Text, Table, ChartVDocRAGLink
UniDoc-Bench1.6K70K (I)Text, Table, ChartUniDocLink
BBox-DocVQA32K4.4K (D)Text, Table, ChartBBox-DocVQALink

Citation

If you find this survey useful, please cite our paper:

@article{gao2025scaling,
      title={Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding},
      author={Sensen Gao and Shanshan Zhao and Xu Jiang and Lunhao Duan and Yong Xien Chng and Qing-Guo Chen and Weihua Luo and Kaifu Zhang and Jia-Wang Bian and Mingming Gong},
      year={2025},
      eprint={2510.15253},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.15253},
}

Contact

For questions or suggestions, feel free to open an issue or contact Sensen Gao.

Contributors

SensenGao

7 commits