ChunmingHe/awesome-multimodal-large-language-models-in-low-level-vision

39 commits

updated Jul 1, 2026

README

Awesome Multimodal Large Language Models In Low-level Vision

🔥A curated list of awesome Multimodal Large Language Models(MLLMs) & Vision-Language Models(VLMs) in low-level vision.🔥

Please feel free to offer your suggestions in the Issues and pull requests to add links.

[ Last updated at 2026/07/01 ]

Awesome Multimodal Large Language Models In Low-level Vision

Latest Works Recommended

Diffusion Models in Low-Level Vision: A Survey
Chunming He, Yuqi Shen, Chengyu Fang, Fengyang Xiao, Longxiang Tang, Yulun Zhang, Wangmeng Zuo, Zhenhua Guo, Xiu Li
TPAMI. [Paper]

Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model
Chunming He, Chengyu Fang, Yulun Zhang, Kai Li, Longxiang Tang, Chenyu You, Fengyang Xiao, Zhenhua Guo, Xiu Li
ICLR 2025, Spotlight. [Paper] [Github]
Jan. 2025

Awesome Papers

1. Direct VLM Adaptation for Low-Level Vision

1.1 Visual Encoder Adaptation: Handling Details

1.1.1 Resolution Scaling

Title	Venue	Date	Code
Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAG	GRSM	2024-11	Github
FeatSharp: Your Vision Model Features, Sharper	ICML	2025-02	Github
GenDR: Lightning Generative Detail Restorator	arXiv	2025-03	Github
Ultra High-Resolution Image Inpainting with Patch-Based Content Consistency Adapter	arXiv	2025-10	Github
Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation	arXiv	2026-02	-

1.1.2 Feature Fusion

Title	Venue	Date	Code
Exploiting Diffusion Prior for Real-World Image Super-Resolution	IJCV	2023-05	Github
MoVA: Adapting Mixture of Vision Experts to Multimodal Context	NeurIPS	2024-04	Github
CLIP-aware Domain-Adaptive Super-Resolution	MMS	2025-05	-
RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration	arXiv	2025-09	Github
Vision-Language Alignment from Compressed Image Representations using 2D Gaussian Splatting	arXiv	2025-09	-
LP-LLM: End-to-End Real-World Degraded License Plate Text Recognition via Large Multimodal Models	arXiv	2026-01	-
Rethinking Cross-Domain Evaluation for Face Forgery Detection with Semantic Fine-grained Alignment and Mixture-of-Experts	arXiv	2026-04	Github

1.2 Language Branch Adaptation: Bridging Modalities

1.2.1 Prompt Learning Strategies

Title	Venue	Date	Code
Iterative Prompt Learning for Unsupervised Backlit Image Enhancement	ICCV	2023-03	Github
Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration	CVPR	2023-12	-
RESTORE: Towards Feature Shift for Vision-Language Prompt Learning	arXiv	2024-03	Github
FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training	arXiv	2024-11	Github
RAP-SR: RestorAtion Prior Enhancement in Diffusion Models for Realistic Image Super-Resolution	AAAI	2024-12	Github
Causality-guided Prompt Learning for Vision-language Models via Visual Granulation	ICCV	2025-09	Github
Generalizable Face Forgery Detection via Separable Prompt Learning	arXiv	2026-04	Github

1.2.2 Instruction Tuning for Restoration

Title	Venue	Date	Code
Q-Instruct: Improving Low-Level Visual Abilities for Multi-Modality Foundation Models	CVPR	2023-11	Github
Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision	arXiv	2025-04	-

1.3 Output Head Adaptation: From Tokens to Pixels

1.3.1 Tokenizer–Decoder Framework

Title	Venue	Date	Code
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	CVPR	2023-11	HuggingFace
LISA: Reasoning Segmentation via Large Language Model	CVPR	2023-08	Github
Beyond Text: Frozen Large Language Models in Visual Signal Comprehension	CVPR	2024-03	Github
Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models	ICCV	2025-03	Github
SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generation	arXiv	2025-03	-
Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Training	ICCV	2025-07	Github
Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?	ICML	2026-06	Github

1.4 Parameter-Efficient Fine-Tuning in Restoration

1.4.1 LoRA & Adapter Integration

Title	Venue	Date	Code
Explicit Visual Prompting for Low-Level Structure Segmentations	CVPR	2023-03	Github
One-Step Effective Diffusion Network for Real-World Image Super-Resolution	NeurIPS	2024-06	Github
LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration	arXiv	2024-10	Github
TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution	CVPR	2024-11	Github
Acquire and then Adapt: Squeezing out Text-to-Image Model for Image Restoration	CVPR	2025-04	-
Demystifying the Visual Quality Paradox in Multimodal Large Language Models	arXiv	2025-06	-
RDDM: Practicing RAW Domain Diffusion Model for Real-world Image Restoration	arXiv	2025-08	-
BIR-Adapter: A Low-Complexity Diffusion Model Adapter for Blind Image Restoration	arXiv	2025-09	-
One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution	arXiv	2025-11	Github

1.4.2 Freezing Strategies

Title	Venue	Date	Code
Grounding Language Models to Images for Multimodal Inputs and Outputs :contentReference[oaicite:0]{index=0}	ICML	2023-01	Github
Prismer: A Vision-Language Model with Multi-Task Experts	TMLR	2023-03	Github
LM4LV: A Frozen Large Language Model for Low-level Vision Tasks	arXiv	2024-05	Github
Frozen Transformers in Language Models Are Effective Visual Encoder Layers	ICLR	2023-10	Github

2. VLM as Auxiliary for Low-Level Vision

2.1 VLM as Semantic Provider: Text-Guided Restoration

2.1.1 Text-Conditioned Injection

Title	Venue	Date	Code
TextIR: A Simple Framework for Text-based Editable Image Restoration	TVCG	2023-02	Github
Pixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization	ECCV	2023-08	Github
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution	CVPR	2023-11	Github
Textual Prompt Guided Image Restoration	EAAI	2023-12	Github
Low-light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion	TOMM	2024-01	Github
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild	CVPR	2024-01	Github
Image Fusion via Vision-Language Model	ICML	2024-02	Github
Boosting Image Restoration via Priors from Pre-trained Models	CVPR	2024-03	Github
MMGInpainting: Multi-Modality Guided Image Inpainting Based on Diffusion Models	TMM	2024-03	Github
Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion	CVPR	2024-03	Github
XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution	ECCV	2024-03	Github
Photo-Realistic Image Restoration in the Wild with Controlled Vision-Language Models	CVPRW	2024-04	Github
Unsupervised Image Prior via Prompt Learning and CLIP Semantic Guidance for Low-Light Image Enhancement	CVPRW	2024-05	-
SSP-IR: Semantic and Structure Priors for Diffusion-based Realistic Image Restoration	TCSVT	2024-07	Github
Beyond Pixels: Text Enhances Generalization in Real-World Image Restoration	arXiv	2024-12	-
CLIP-SR: Collaborative Linguistic and Image Processing for Super-Resolution	TMM	2024-12	Github
MGFusion: a multimodal large language model-guided information perception for infrared and visible image fusion	Front. Neurorobot.	2024-12	-
Textual-Visual Interaction for Enhanced Single Image Deraining using Adapter-Tuned VLMs	arXiv	2024-12	Github
Unified Image Restoration and Enhancement: Degradation Calibrated Cycle Reconstruction Diffusion Model	arXiv	2024-12	Github
Multi-Text Guidance Is Important: Multi-Modality Image Fusion via Large Generative Vision-Language Model	IJCV	2025-02	Github
MegaSR: Mining Customized Semantics and Expressive Guidance for Image Super-Resolution	arXiv	2025-03	Github
The Power of Context: How Multimodality Improves Image Super-Resolution	CVPR	2025-03	-
Coarse-to-fine text injecting for realistic image super-resolution	Neurocom	2025-02	-
DeepSPG: Exploring Deep Semantic Prior Guidance for Low-light Image Enhancement with Multimodal Learning	arXiv	2025-04	Github
CTD-inpainting: Towards the Coherence of Text-driven Inpainting with Blended Diffusion	Information Fusion	2025-04	-
Exploiting Diffusion Prior for Real-World Image Dehazing with Unpaired Training	AAAI	2025-03	Github
A Preliminary Study for GPT-4o on Image Restoration	arXiv	2025-05	Github
Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignment	NeurIPS	2025-05	Github
Text-Aware Image Restoration with Diffusion Models	arXiv	2025-06	Github
LD-RPS: Zero-Shot Unified Image Restoration via Latent Diffusion Recurrent Posterior Sampling	ICCV	2025-07	Github
LLaVA-based semantic feature modulation diffusion model for underwater image enhancement	InfFus	2025-07	-
Harnessing Diffusion-Yielded Score Priors for Image Restoration	SIGGRAPH Asia	2025-07	Github
Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancement	arXiv	2025-07	Github
HVI-CIDNet+: Beyond Extreme Darkness for Low-Light Image Enhancement	arXiv	2025-07	Github
ModalFormer: Multimodal Transformer for Low-Light Image Enhancement	arXiv	2025-07	Github
AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception	arXiv	2025-08	Github
MambaTrans: Multimodal Fusion Image Translation via Large Language Model Priors for Downstream Visual Tasks	arXiv	2025-08	-
LucidFlux: Caption-Free Universal Image Restoration via a Large-Scale Diffusion Transformer	arXiv	2025-09	Github
Vision-Language Model Guided Image Restoration	arXiv	2025-12	-
Zero-Shot Image Super-Resolution Using Prompt-Driven Vision-Language Foundation Models Without Task-Specific Fine-Tuning	arXiv	2025-09	-
Extreme Blind Image Restoration via Prompt-Conditioned Information Bottleneck	arXiv	2025-10	-
A structural information-guided cross-modal method for damaged inscription inpainting via vision-language models	npj Heritage Science	2025-09	Github
GLYPH-SR: Can We Achieve Both High-Quality Image Super-Resolution and High-Fidelity Text Recovery via VLM-Guided Latent Diffusion Model?	arXiv	2025-10	-
Unified Diffusion Transformer for High-fidelity Text-Aware Image Restoration	arXiv	2025-12	Github
SCENE: Semantic-aware Codec Enhancement with Neural Embeddings	ICASSP	2026-01	-
DACESR: Degradation-Aware Conditional Embedding for Real-World Image Super-Resolution	TIP	2026-02	Github
QUSR: Quality-Aware and Uncertainty-Guided Image Super-Resolution Diffusion Model	ICASSP	2026-03	Github
Enhancing Single-Image Facial Demorphing using Multimodal Large Language Models	arXiv	2026-05	-
Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts	arXiv	2026-05	-

2.1.2 Language-Driven Manipulation

Title	Venue	Date	Code
InstructPix2Pix: Learning to Follow Image Editing Instructions	CVPR	2022-11	Github
StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model	ICCV	2023-03	Github
L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors	NeurIPS	2023-05	Github
L-CoIns: Language-Based Colorization With Instance Awareness	CVPR	2023-06	Github
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior	ECCV	2023-08	Github
LLMGA: Multimodal Large Language Model based Generation Assistant	ECCV	2023-11	Github
SPIRE: Semantic Prompt-Driven Image Restoration	ECCV	2023-12	-
InstructIR: High-Quality Image Restoration Following Human Instructions	ECCV	2024-01	Github
Text-guided Explorable Image Super-resolution	CVPR	2024-03	Github
Paint by Inpaint: Learning to Add Image Objects by Removing Them First	CVPR	2024-04	Github
Brush2Prompt: Contextual Prompt Generator for Object Inpainting	CVPR	2024-06	-
Range-Null Latent Prior-guided Consistency Model for Low Light Image Enhancement	arXiv	2024-09	-
PainterNet: Adaptive Image Inpainting with Actual-Token Attention and Diverse Mask Control	arXiv	2024-12	-
Instruct-CLIP: Improving Instruction-Guided Image Editing with Automated Data Refinement Using Contrastive Learning	CVPR	2025-03	Github
Instruct2See: Learning to Remove Any Obstructions Across Distributions	ICML	2025-05	Project
MIND-Edit: MLLM Insight-Driven Editing via Language-Vision Projection	arXiv	2025-05	-
Fidelity-Preserving Enhancement of Ptychography with Foundational Text-to-Image Models	arXiv	2025-09	Github
FOCUS: Unified Vision-Language Modeling for Interactive Editing Driven by Referential Segmentation	NeurIPS	2025-06	-
LaTo: Landmark-Tokenized Diffusion Transformer for Fine-Grained Human Face Editing	arXiv	2025-09	-
Query-Kontext: An Unified Multimodal Model for Image Generation and Editing	arXiv	2025-09	-
Visual Autoregressive Modeling for Instruction-Guided Image Editing	arXiv	2025-08	Github
Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editing	arXiv	2025-10	Github
Region in Context: Text-Conditioned Image Editing with Human-Like Semantic Reasoning	arXiv	2025-10	Github
UniSER: A Foundation Model for Unified Soft Effects Removal	arXiv	2025-11	-
PhotoFramer: Multi-modal Image Composition Instruction	CVPR	2025-11	Github
Generative Editing in the Joint Vision-Language Space for Zero-Shot Composed Image Retrieval	arXiv	2025-12	-
How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing	arXiv	2026-02	Github
MCIE: Multimodal LLM-Driven Complex Instruction Image Editing with Spatial Guidance	AAAI	2026-02	-
EditCaption: Human-Aligned Instruction Synthesis for Image Editing via Supervised Fine-Tuning and Direct Preference Optimization	arXiv	2026-04	-
ScribbleEdit: Synthetic Data for Image Editing with Scribbles and Text	arXiv	2026-05	-
Multimodal Image Colorization: Quantifying the Impact of Text-Conditioned Guidance on Grayscale-to-Color Translation	arXiv	2026-06	-
SeamEdit: A Black-Box VLM-Agnostic Pipeline for Large-Image Semantic Editing	arXiv	2026-06	Github
TeleStyle V2: Beyond Content-Preserving Style Transfer with Self-Distillation and Distribution-Matching-Distillation	arXiv	2026-06	Github
When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing	arXiv	2026-06	-

2.1.3 Subject-Aware Restoration

Title	Venue	Date	Code
Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model	ICLR	2024-10	-
LLV-FSR: Exploiting Large Language-Vision Prior for Face Super-resolution	arXiv	2024-11	-
InstructRestore: Region-Customized Image Restoration with Human Instructions	arXiv	2025-03	Github
PromptLNet: Region-Adaptive Aesthetic Enhancement via Prompt Guidance in Low-Light Enhancement Net	arXiv	2025-03	-
TSCnet: A Text-driven Semantic-level Controllable Framework for Customized Low-Light Image Enhancement	Neurocomputing	2025-03	Project
EAM: Enhancing Anything with Diffusion Transformers for Blind Super-Resolution	arXiv	2025-05	-
Empowering Semantic-Sensitive Underwater Image Enhancement with VLM	AAAI	2026-03	-
JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation	arXiv	2026-05	Github

2.2 VLM as Degradation Interpreter: Visual-Prompting & Context

2.2.1 Degradation Classification

Title	Venue	Date	Code
Exploring the Application of Large-Scale Pre-Trained Models on Adverse Weather Removal	TIP	2023-06	-
LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network	CVPR	2023-07	Github
AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion	ECCV	2023-10	Github
Unifying Image Processing as Visual Prompting Question Answering	ICML	2023-10	Github
Controlling Vision-Language Models for Multi-Task Image Restoration	ICLR	2024-02	Github
DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution	arXiv	2024-06	Github
DAP-LED: Learning Degradation-Aware Priors with CLIP for Joint Low-light Enhancement and Deblurring	ICRA	2024-09	-
Multi-modal degradation feature learning for unified image restoration based on contrastive learning	Neurocomputing	2024-11	-
All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model	TMM	2024-11	-
AllRestorer: All-in-One Transformer for Image Restoration under Composite Degradations	arXiv	2024-11	-
LMHaze: Intensity-aware Image Dehazing with a Large-scale Multi-intensity Real Haze Dataset	MMAsia	2024-12	-
Adaptive Fuzzy Degradation Perception Based on CLIP Prior for All-in-One Image Restoration	TFS	2024-12	Github
Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks	CVPR	2025-03	Github
DA2Diff: Exploring Degradation-aware Adaptive Diffusion Priors for All-in-One Weather Restoration	arXiv	2025-04	-
VL-UR: Vision-Language-guided Universal Restoration of Images Degraded by Adverse Weather Conditions	ICME	2025-04	-
Controlling vision-language model for enhancing image restoration	IVC	2025-04	-
CLIP-driven rain perception: Adaptive deraining with pattern-aware network routing and mask-guided cross-attention	PR	2025-06	-
Degradation-Aware Image Enhancement via Vision-Language Classification	arXiv	2025-06	-
M2Restore: Mixture-of-Experts-based Mamba-CNN Fusion Framework for All-in-One Image Restoration	TIP	2025-06	Github
Real-world super-resolution with VLM-based degradation prior learning	Sci Rep	2025-08	-
Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution	arXiv	2025-11	-
Zero-Reference Joint Low-Light Enhancement and Deblurring via Visual Autoregressive Modeling with VLM-Derived Modulation	AAAI	2025-11	Github
CLIP-Guided Unsupervised Semantic-Aware Exposure Correction	ICASSP	2026-01	-
Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignment	arXiv	2026-02	-
TextShield-R1: Reinforced Reasoning for Tampered Text Detection	AAAI	2026-02	Github
Understanding Degradation with Vision Language Model	arXiv	2026-02	-
Generalizable Detection of AI Generated Images with Large Models and Fuzzy Decision Tree	arXiv	2026-03	-
DistortBench: Benchmarking Vision Language Models on Image Distortion Identification	arXiv	2026-04	-

2.2.2 Description-based Restoration

Title	Venue	Date	Code
Image Super-Resolution with Text Prompt Diffusion	arXiv	2023-11	Github
Language-driven All-in-one Adverse Weather Removal	CVPR	2023-12	Github
LLMRA: Multi-modal Large Language Model based Restoration Assistant	arXiv	2024-01	-
Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration	arXiv	2024-07	Github
Training-Free Large Model Priors for Multiple-in-One Image Restoration	arXiv	2024-07	-
UniProcessor: A Text-induced Unified Low-level Image Processor	ECCV	2024-07	Github
AWRaCLe: All-Weather Image Restoration using Visual In-Context Learning	AAAI	2024-08	Github
DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation	NeurIPS	2024-10	Github
Leveraging vision-language prompts for real-world image restoration and enhancement	CVIU	2024-11	-
Leveraging Content and Context Cues for Low-Light Image Enhancement	TMM	2024-12	Github
RamIR: Reasoning and action prompting with Mamba for all-in-one image restoration	-	2025-01	-
ControlFusion: A Controllable Image Fusion Network with Language-Vision Degradation Prompts	arXiv	2025-03	Github
Prompt to Restore, Restore to Prompt: Cyclic Prompting for Universal Adverse Weather Removal	TIP	2025-03	Github
UniCoRN: Latent Diffusion-based Unified Controllable Image Restoration Network across Multiple Degradations	WACV	2025-03	Project
CLIP-RestoreX: Restore Image Structure and Perception in Exposure Correction	AAAI	2025-04	Github
Dual Prompting Image Restoration with Diffusion Transformers	CVPR	2025-04	Github
A single image deraining algorithm guided by text generation based on depth information conditions	ASOC	2025-07	Github
Adaptive multi-modal prompting for universal image restoration amidst diverse degradations	TVC	2025-05	Github
RAGSR: Regional Attention Guided Diffusion for Image Super-Resolution	arXiv	2025-08	-
Dual-Domain Perspective on Degradation-Aware Fusion: A VLM-Guided Robust Infrared and Visible Image Fusion Framework	arXiv	2025-09	-
Real-world super-resolution with VLM-based degradation prior learning	Scientific Reports	2025-08	-
Coupled Degradation Modeling and Fusion: A VLM-Guided Degradation-Coupled Network for Degradation-Aware Infrared and Visible Image Fusion	arXiv	2025-10	Github
Coupled Degradation Modeling and Fusion: A VLM-Guided Degradation-Coupled Network for Degradation-Aware Infrared and Visible Image Fusion	arXiv	2025-10	Github
MdaIF: Robust One-Stop Multi-Degradation-Aware Image Fusion with Language-Driven Semantics	AAAI	2025-11	Github
Multimodal image fusion network with prior-guided dynamic degradation removal for extreme environment perception	Sci Rep	2025-10	Github
Nested Unfolding Network for Real-World Concealed Object Segmentation	arXiv	2025-11	-
VL-UR: Vision-Language-guided Universal Restoration of Images Degraded by Adverse Weather Conditions	arXiv	2025-04	-
VLM-Augmented Degradation Modeling for Image Restoration Under Adverse Weather Conditions	arXiv	2025-11	-

2.3 VLM as Quality Evaluator: Perception and Assessment

2.3.1 No-Reference Quality Assessment

Title	Venue	Date	Code
Exploring CLIP for Assessing the Look and Feel of Images	AAAI	2022-07	Github
Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective	CVPR	2023-03	Github
VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining	CVPR	2023-03	Github
DegAE: A New Pretraining Paradigm for Low-Level Vision	CVPR	2023-06	Github
Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models	ECCV	2023-12	Github
Quality-Aware Image-Text Alignment for Opinion-Unaware Image Quality Assessment (QualiCLIP)	arXiv	2024-03	Github
Descriptive Image Quality Assessment in the Wild	arXiv	2024-05	Github
UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment	arXiv	2024-06	Github
CLIP-AGIQA: Boosting the Performance of AI-Generated Image Quality Assessment with CLIP	ICPR	2024-08	Github
Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessment	arXiv	2024-10	Github
Large Language Models for Lossless Image Compression: Next-Pixel Prediction in Language Space is All You Need	NeurIPS	2024-11	Github
Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models	AAAI	2024-11	Github
Low-Light Image Enhancement via Generative Perceptual Priors	AAAI	2024-12	Github
Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution	CVPR	2025-01	Github
CLIP-DQA: Blindly Evaluating Dehazed Images from Global and Local Perspectives Using CLIP	ISCAS	2025-02	Github
DVLTA-VQA: Decoupled Vision-Language Modeling with Text-Guided Adaptation for Blind Video Quality Assessment	arXiv	2025-04	-
Q-Adapt: Adapting LMM for Visual Quality Assessment with Progressive Instruction Tuning	arXiv	2025-04	Github
Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration	TIP	2024-08	Github
GIE-Bench: Towards Grounded Evaluation for Text-Guided Image Editing	arXiv	2025-05	Github
ImgEdit: A Unified Image Editing Dataset and Benchmark	NeurIPS	2025-05	Github
What Changed? Detecting and Evaluating Instruction-Guided Image Edits with Multimodal Large Language Models	ICCV	2025-05	Github
BPCLIP: A Bottom-up Image Quality Assessment from Distortion to Semantics Based on CLIP	ICME	2025-06	-
Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment	arXiv	2025-06	Github
Leveraging Vision-Language Models to Select Trustworthy Super-Resolution Samples Generated by Diffusion Models	TCSVT	2025-06	-
VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning	arXiv	2025-06	-
DehazeMamba: large multi-modal model guided single image dehazing via mamba	VisIntell	2025-07	-
Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution	arXiv	2025-07	-
Q-CLIP: Unleashing the Power of Vision-Language Models for Video Quality Assessment through Unified Cross-Modal Adaptation	arXiv	2025-08	-
Fine-grained Image Quality Assessment for Perceptual Image Restoration	AAAI	2025-08	Github
Refine-IQA: Multi-Stage Reinforcement Finetuning for Perceptual Image Quality Assessment	AAAI	2025-08	Github
Segmenting and Understanding: Region-aware Semantic Attention for Fine-grained Image Quality Assessment with Large Language Models	arXiv	2025-08	-
AgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessment	arXiv	2025-09	-
Describe-to-Score: Text-Guided Efficient Image Complexity Assessment	arXiv	2025-09	Github
Parameter-Efficient Adaptation of mPLUG-Owl2 via Pixel-Level Visual Prompts for NR-IQA	arXiv	2025-09	Github
Revisiting Vision–Language Foundations for No-Reference Image Quality Assessment	WACV	2025-09	-
From Filters to VLMs: Benchmarking Defogging Methods through Object Detection and Segmentation Performance	arXiv	2025-10	Github
Revisiting MLLM Based Image Quality Assessment: Errors and Remedy	AAAI	2025-11	Github
ClearAIR: A Human-Visual-Perception-Inspired All-in-One Image Restoration	AAAI	2026-01	Github
Enhancing Image Quality Assessment Ability of LMMs via Retrieval-Augmented Generation	arXiv	2026-01	-
QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding	arXiv	2026-01	Github
Q-Bench-Portrait: Benchmarking Multimodal Large Language Models on Portrait Image Quality Perception	arXiv	2026-01	-
VideoAesBench: Benchmarking the Video Aesthetics Perception Capabilities of Large Multimodal Models	arXiv	2026-01	Github
Q-Hawkeye: Reliable Visual Policy Optimization for Image Quality Assessment	arXiv	2026-01	-
ELIQ: A Label-Free Framework for Quality Assessment of Evolving AI-Generated Images	arXiv	2026-02	-
Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis	arXiv	2026-02	Github
Synthetic Image Detection with CLIP: Understanding and Assessing Predictive Cues	arXiv	2026-02	Github
CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructions	arXiv	2026-03	Github
Evaluating Image Editing with LLMs: A Comprehensive Benchmark and Intermediate-Layer Probing Approach	arXiv	2026-03	-
ME-IQA: Memory-Enhanced Image Quality Assessment via Re-Ranking	arXiv	2026-03	-
Q-Tacit: Image Quality Assessment via Latent Visual Reasoning	arXiv	2026-03	Github
R4-CGQA: Retrieval-based Vision Language Models for Computer Graphics Image Quality Assessment	CVPR	2026-03	Github
Vision-Language Models vs Human: Perceptual Image Quality Assessment	arXiv	2026-03	-
EvaNet: Towards More Efficient and Consistent Infrared and Visible Image Fusion Assessment	TPAMI	2026-04	Github
Multi-Granularity Reasoning for Image Quality Assessment via Attribute-Aware Reinforcement Learning to Rank	arXiv	2026-04	-
NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1)	CVPRW	2026-04	Github
Redefining Quality Criteria and Distance-Aware Score Modeling for Image Editing Assessment	arXiv	2026-04	-
UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs	arXiv	2026-04	Github
Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment	arXiv	2026-05	-
Employing Vision-Language Models for Face Image Quality Assessment	FG	2026-05	Github
UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs	arXiv	2026-04	Github
DAL-PCQA: Enabling Distortion-Level and Language-Driven Reasoning for Point Cloud Quality Assessment	arXiv	2026-06	Github
Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment	arXiv	2026-06	Github
LEIQ-Assessor: Multi-dimensional Quality Assessment of Low-light Enhanced Images via Multi-task Learning	arXiv	2026-06	Github
LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models	arXiv	2026-06	Github
Tool-IQA: Augmenting Image Quality Assessment with Simple Tools	arXiv	2026-06	Github

2.3.2 Semantic Consistency Loss

Title	Venue	Date	Code
Cycle Contrastive Adversarial Learning for Unsupervised image Deraining	arXiv	2024-07	Github
Contourlet Residual for Prompt Learning Enhanced Infrared Image Super-Resolution	ECCV	2024-09	Github
Underwater Diffusion Attention Network with Contrastive Language-Image Joint Learning for Underwater Image Enhancement	arXiv	2025-05	-
Unveiling the Underwater World: CLIP Perception Model-Guided Underwater Image Enhancement	PR	2025-06	-
One-Step Diffusion-based Real-World Image Super-Resolution with Visual Perception Distillation	arXiv	2025-06	-
From Enhancement to Understanding: Build a Generalized Bridge for Low-light Vision via Semantically Consistent Unsupervised Fine-tuning	ICCV	2025-07	Github
WeatherCycle: Unpaired Multi-Weather Restoration via Color Space Decoupled Cycle Learning	arXiv	2025-09	-
Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback	arXiv	2026-06	Github

2.3.3 Feedback Loops

Title	Venue	Date	Code
HazeCLIP: Towards Language Guided Real-World Image Dehazing	ICASSP	2024-07	Github
Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models	ECCV	2024-09	Github
DSPO: Direct Semantic Preference Optimization for Real-World Image Super-Resolution	arXiv	2025-04	-
SnowMaster: Comprehensive Real-world Image Desnowing via MLLM with Multi-Model Feedback Optimization	CVPR	2025-06	Github
EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing	arXiv	2025-09	Github
Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking	arXiv	2025-09	-
Learning an Image Editing Model without Image Editing Pairs	arXiv	2025-10	-
Test-Time Preference Optimization for Image Restoration	AAAI	2025-11	Github
UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in Reinforcement Learning	arXiv	2025-11	Github
UARE: Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancement	arXiv	2025-12	Github
Zoom-IQA: Image Quality Assessment with Reliable Region-Aware Reasoning	arXiv	2026-01	-
CoCoEdit: Content-Consistent Image Editing via Region Regularized Reinforcement Learning	IJCAI	2026-02	Github
EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing	arXiv	2026-03	Github
HP-Edit: A Human-Preference Post-Training Framework for Image Editing	CVPR	2026-04	-
Q-DeepSight: Incentivizing Thinking with Images for Image Quality Assessment and Refinement	arXiv	2026-04	-
GGT-100K: Generative Ground Truth for Generalizable Real-World Image Restoration	arXiv	2026-05	Github
Stable-Layers: Fine-Tuning Image Layer Decomposition Models with VLM-Scored Reinforcement Learning	arXiv	2026-05	Project
White-Balance First, Adjust Later: Cross-Camera Color Constancy via Vision-Language Evaluation	CVPR	2026-05	Github

2.4 VLM as Intelligent Controller: Agent-Based Frameworks

2.4.1 Motivation as Intelligent Controller

2.4.2 Tool Usage & Orchestration

Title	Venue	Date	Code
LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing	arXiv	2023-11	Github
Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Models	IJCAI	2024-01	-
Image Inpainting Models are Effective Tools for Instruction-guided Image Editing	CVPRW	2024-07	Github
RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models	NeurIPS	2024-07	-
VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing	NeurIPS	2024-10	Github
LLMCO4MR: LLMs-Aided Neural Combinatorial Optimization for Ancient Manuscript Restoration from Fragments with Case Studies on Dunhuang	ECCV	2024-11	-
I Dream My Painting: Connecting MLLMs and Diffusion Models via Prompt Generation for Text-Guided Multi-Mask Inpainting	WACV	2024-11	Github
BrushEdit: All-In-One Image Inpainting and Editing	NeurIPS	2024-12	Github
Hybrid Agents for Image Restoration	arXiv	2025-03	-
Multi-Agent Image Restoration	arXiv	2025-03	-
JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration	CVPR	2025-04	Github
Q-Agent: Quality-Driven Chain-of-Thought Image Restoration Agent through Robust Multimodal Large Language Model	arXiv	2025-04	-
LEGO: LLM-enhanced genetic optimization for underwater robot image restoration	PR	2025-05	Github
Light as Deception: GPT-driven Natural Relighting Against Vision-Language Pre-training Models	arXiv	2025-05	-
MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills	TOG	2025-05	Github
PhotoArtAgent: Intelligent Photo Retouching with Language Model-Based Artist Agents	arXiv	2025-05	-
JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent	NeurIPS	2025-06	Github
EdiVal-Agent: An Object-Centric Framework for Automated, Scalable, Fine-Grained Evaluation of Multi-Turn Editing	arXiv	2025-09	Github
InstructVTON: Optimal Auto-Masking and Natural-Language-Guided Interactive Style Control for Inpainting-Based Virtual Try-On	arXiv	2025-09	-
Prompt-Driven Image Analysis with Multimodal Generative AI: Detection, Segmentation, Inpainting, and Interpretation	arXiv	2025-09	-
MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration	JSTSP	2025-10	Github
From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustration	arXiv	2025-10	Github
Image-POSER: Reflective RL for Multi-Expert Image Generation and Editing	arXiv	2025-11	-
T2T-VICL: Unlocking the Boundaries of Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs	arXiv	2025-11	-
RePlan: Reasoning-guided Region Planning for Complex Instruction-based Image Editing	arXiv	2025-12	Github
ForgeryVCR: Visual-Centric Reasoning via Efficient Forensic Tools in MLLMs for Image Forgery Detection and Localization	arXiv	2026-02	Project
Interaction-Consistent Object Removal via MLLM-Based Reasoning	arXiv	2026-02	-
MSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing	arXiv	2026-03	-
PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning	arXiv	2026-03	Project
TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration	arXiv	2026-03	-
GenSmoke-GS: A Multi-Stage Method for Novel View Synthesis from Smoke-Degraded Images Using a Generative Model	CVPRW	2026-04	Github
DiTTo: Scalable Order-aware All-in-One Image Restoration Agent	arXiv	2026-05	Project
From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editing	arXiv	2026-05	Project
IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment	CVPR	2026-06	Github
Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution	arXiv	2026-06	-

Title	Venue	Date	Code
Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement	arXiv	2023-11	-
Anywhere: A Multi-Agent Framework for User-Guided, Reliable, and Diverse Foreground-Conditioned Image Generation	AAAI	2024-04	Github
EditScribe: Non-Visual Image Editing with Natural Language Verification Loops	ASSETS	2024-08	-
An Intelligent Agentic System for Complex Image Restoration Problems	ICLR	2024-10	Github
CURVE: CLIP-Utilized Reinforcement Learning for Visual Image Enhancement via Simple Image Processing	ICIP	2025-05	-
RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought	arXiv	2025-06	Github
4KAgent: Agentic Any Image to 4K Super-Resolution	NeurIPS	2025-07	Github
Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing	arXiv	2025-10	Github
Dynamic VLM-Guided Negative Prompting for Diffusion Models	NeurIPS	2025-10	-
From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustration	arXiv	2025-10	Github
Q-Probe: Scaling Image Quality Assessment to High Resolution via Context-Aware Agentic Probing	arXiv	2026-01	-
Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions	arXiv	2026-04	-
EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinement	NeurIPS	2026-05	Github

3. Extended Applications

3.1 Medical Image Processing

3.1.1 Biomedical VLMs

Title	Venue	Date	Code
HuatuoGPT-Vision: Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale	EMNLP	2024-06	Github
MLeVLM: Improve Multi-level Progressive Capabilities based on Multimodal Large Language Model for Medical Visual Question Answering	ACL	2024-08	Github
PET Image Denoising via Text-Guided Diffusion: Integrating Anatomical Priors through Text Prompts	arXiv	2025-02	-
AgentPolyp: Accurate Polyp Segmentation via Image Enhancement Agent	SPL	2025-04	-
Text controllable PET denoising	SPIE	2026-01	-
FluoCLIP: Stain-Aware Focus Quality Assessment in Fluorescence Microscopy	arXiv	2026-02	Project
MedQ-UNI: Toward Unified Medical Image Quality Assessment and Restoration via Vision-Language Modeling	arXiv	2026-03	-

3.1.2 CT and MRI

Title	Venue	Date	Code
Controllable Text-to-Image Synthesis for Multi-Modality MR Images	WACV	2024-01	-
Low-dose CT Denoising with Language-engaged Dual-space Alignment	arXiv	2024-03	Github
Dual-Domain CLIP-Assisted Residual Optimization Perception Model for Metal Artifact Reduction	TRPMS	2024-08	-
A-IDE: Agent-Integrated Denoising Experts	arXiv	2025-03	-
Patient-Level Anatomy Meets Scanning-Level Physics: Personalized Federated Low-Dose CT Denoising Empowered by Large Language Model	CVPR	2025-03	Github
TDMF: Text-Guided Denoising and Interactive Medical Image Fusion	ICASSP	2025-04	-
Taming Stable Diffusion for Computed Tomography Blind Super-Resolution	MICCAI WS	2025-06	Github
LangMamba: A Language-driven Mamba Framework for Low-dose CT Denoising with Vision-language Models	TRPMS	2025-07	Github
Prompt-Conditioned FiLM and Multi-Scale Fusion on MedSigLIP for Low-Dose CT Quality Assessment	arXiv	2025-11	Github
Vision–Language Controlled Deep Unfolding for Joint Medical Image Restoration and Segmentation	arXiv	2026-01	-
CT-DegradBench: A Physics-Informed Benchmark for CT Degradation Detection and Severity Estimation	arXiv	2026-05	Github

3.2 Remote Sensing Data Processing

3.2.1 Spatial-domain Tasks

Title	Venue	Date	Code
Zooming Out on Zooming In: Advancing Super-Resolution for Remote Sensing	arXiv	2023-11	Github
Large Foundation Model Empowered Discriminative Underwater Image Enhancement	IEEE	2025-01	Gitee
Semantic-Aware Guidance for Blind Super-Resolution of Remote Sensing Images	GRSL	2025-01	-
SeG-SR: Integrating Semantic Knowledge into Remote Sensing Image Super-Resolution via Vision-Language Model	TGRS	2025-05	Github
SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models	arXiv	2026-05	Github

3.2.2 Spectral-domain Tasks

Title	Venue	Date	Code
PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation	arXiv	2024-11	Github
Leveraging Language-Aligned Visual Knowledge for Remote Sensing Image Spectral Super-Resolution	WHISPERS	2024-12	Github
MP-HSIR: A Multi-Prompt Framework for Universal Hyperspectral Image Restoration	ICCV	2025-03	Github

3.3 Other Extended Applications

3.3.1 CAD

Title	Venue	Date	Code
Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models	CVPR	2024-11	Github
ChatCAD: An MLLM-Guided Framework for Zero-shot CAD Drawing Restoration	ICASSP	2025-04	-
ImageRAG: Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAG	IEEE	2024-11	Github
Semantic Document Derendering: SVG Reconstruction via Vision-Language Modeling	AAAI	2025-11	Github
ReCAD: Reinforcement Learning Enhanced Parametric CAD Model Generation with Vision-Language Models	AAAI	2025-12	-

3.3.2 Video Processing Tasks

Title	Venue	Date	Code
Towards Language-Driven Video Inpainting via Multimodal Large Language Models	CVPR	2024-01	Github
Triplane-Smoothed Video Dehazing with CLIP-Enhanced Generalization	IJCV	2024-08	-
Universal Video Face Restoration Method Based on Vision-Language Model	ACML	2024-09	-
StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification	arXiv	2024-11	Github
Towards General-Purpose Video Reconstruction through Synergy of Grid-Splicing Diffusion and Large Language Models	TCSVT	2025-03	-
Grounding Degradations in Natural Language for All-In-One Video Restoration	arXiv	2025-07	-
When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs	arXiv	2025-09	-
Edit-Your-Interest: Efficient Video Editing via Feature Most-Similar Propagation	arXiv	2025-10	-
Zero-Shot Video Deraining with Video Diffusion Models	WACV	2025-11	Github
ReViSE: Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning	arXiv	2025-12	Github
S3-CLIP: Video Super Resolution for Person-ReID	WACV	2026-01	Github
Video-Rate Streaming Stylization on a Vision-Aware MLLM-Conditioned Edit Diffusion: Asymmetric Batched Inference on a Distilled UNet + MLLM Text Encoder	arXiv	2026-06	Github

3.3.3 3D Processing Tasks

Title	Venue	Date	Code
RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction	ICRA	2023-07	Github
DepthLM: Metric Depth From Vision Language Models	arXiv	2025-09	Github
SRSplat: Feed-Forward Super-Resolution Gaussian Splatting from Sparse Multi-View Images	AAAI	2025-11	Github
GS-Light: Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relighting	arXiv	2025-11	-
Lang3D-XL: Language Embedded 3D Gaussians for Large-scale Scenes	SIGGRAPH Asia	2025-12	Github

Datasets

Task	Dataset	Size	Repo / Download	Description
SR	DIV2K	900/100	Download	High-resolution natural images for super-resolution with synthetic degradations.
SR	RealSR	595	Repo	Real-captured paired low-resolution and high-resolution images using focal-length changes.
SR	DRealSR	2,507	Repo	Real-world super-resolution pairs collected across indoor and outdoor scenes.
LLIE	LOLv1	585/15	Download	Paired low-light and normal-light real images for low-light enhancement.
LLIE	MIT-Adobe FiveK	25,000	Download	Expert-retouched photo pairs for illumination and tone related enhancement tasks.
LLIE	LOLv2	1,589/200	Repo	Combined real and synthetic paired data for low-light enhancement.
Dehaze	RESIDE	13,000/990	Repo	Dehazing benchmark including synthetic and real-world subsets.
Dehaze	NH-Haze	55	Download	Real paired outdoor images with non-homogeneous haze.
Dehaze	Haze-4K	4,000	Repo	Synthetic 4K dehazing pairs with auxiliary physical annotations.
Inpainting	CelebA	200,000	Download	Large-scale face dataset used for face-centric restoration and inpainting.
Inpainting	CelebA-HQ	30,000	Repo	High-quality face images derived from CelebA for high-fidelity restoration.
Inpainting, Derain	MSCOCO	328,124	Download	Large-scale natural images with segmentation annotations for generic restoration and editing.
Derain	Rain100H	1,800/100	Download	Synthetic heavy-rain streak pairs for supervised deraining.
Derain	RainDrop	861/239	Repo	Real paired images with adherent raindrops for deraining and restoration.
Deblur	GoPro	2,103/1,111	Download	Motion blur benchmark synthesized from high-frame-rate video frames.
Deblur	RealBlur	3,758/980	Repo	Real paired short-exposure and long-exposure images for motion deblurring.
Deblur	HIDE	8,422	Repo	Human-centered motion deblurring dataset with complex dynamic motion.
CT denoising	2016 NIH-AAPM-Mayo	5,936	Download	Low-dose CT dataset used for dose reduction and denoising research.
CT denoising	Mayo-2016	4,800/1,136	Download	Challenge dataset used for low-dose CT denoising and reconstruction.
CT denoising	Mayo-2020	2,400/580	Download	Preprocessed low-dose CT dataset for training and evaluation.

Title	Venue	Date	Code	Note
Diffusion Models in Low-Level Vision: A Survey	TPAMI	2024-06	Repo	Diffusion-based methods for low-level vision, including task formulations, architectures, and training design patterns.
A Survey on All-in-One Image Restoration: Taxonomy, Methods, and Future Directions	arXiv	2024-10	Repo	All-in-one image restoration models covering multi-task, multi-degradation, unified training, and evaluation protocols.
Instruction Guided Editing Controls for Images and Multimedia: A Survey in the Era of Large Language Models	arXiv	2024-11	Repo	Instruction-driven image and multimedia editing, covering control types, model families, and evaluation for edit fidelity and consistency.
Vision-Language Models for Vision Tasks: A Survey	TPAMI	2023-04	Repo	VLM methodology for vision tasks, including pretraining, transfer learning, distillation, datasets, and benchmarks.
A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges	CVPRW	2025-01	Repo	LVLM design and evaluation, focusing on alignment, benchmark suites, failure modes, and measurement practices.
A Survey on Multimodal Large Language Models	NSR	2023-06	Repo	Core MLLM architectures, training data, instruction tuning, evaluation, and common issues such as hallucination.
Vision Language Models: A Survey of 26K Papers	arXiv	2025-10	-	Trend analysis and taxonomy at scale, summarizing research directions from a large paper collection.
Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions	arXiv	2024-02	-	Broad overview of VLM model families, training paradigms, and benchmarks, with an emphasis on capabilities and limitations.
Vision Encoders in Vision-Language Models: A Survey	Tech Report	2025-12	-	Vision encoder design choices for VLMs, including architecture families, scaling trends, and practical tradeoffs.

Other Awesome VLM Papers

Title	Venue	Date	Code
MMAIF: Multi-task and Multi-degradation All-in-One for Image Fusion with Language Guidance	arXiv	2025-03	Github
Adding Conditional Control to Text-to-Image Diffusion Models	ICCV	2023-02	Github
GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions	IJCV	2023-05	Github
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding	CVPR	2023-11	Github
IMProv: Inpainting-based Multimodal Prompting for Computer Vision Tasks	TMLR	2023-12	Github
Generative Multi-Modal Knowledge Retrieval with Large Language Models	AAAI	2024-01	Github
CoLLaVO: Crayon Large Language and Vision mOdel	arXiv	2024-02	Github
Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond	ACL	2024-02	Github
MMCTAgent: Multi-modal Critical Thinking Agent Framework for Complex Visual Reasoning	NeurIPSW	2024-05	Github
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding	NeurIPS	2024-06	Github
Multi-Expert Adaptive Selection: Task-Balancing for All-in-One Image Restoration	arXiv	2024-07	Github
Vision Language Models Are Blind: Failing to Translate Detailed Visual Features into Words	ACCV	2024-07	Github
Learning A Low-Level Vision Generalist via Visual Task Prompt	ACM MM	2024-08	Github
On Domain-Adaptive Post-Training for Multimodal Large Language Models	EMNLP	2024-11	Github
ForgeryGPT: Multimodal Large Language Model For Explainable Image Forgery Detection and Localization	arXiv	2024-10	-
Scene Text Detection in Foggy Weather Utilizing Knowledge Distillation of Diffusion Models	SPL	2025-02	-
Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent	arXiv	2025-03	Github
Why Compress What You Can Generate? When GPT-4o Generation Ushers in Image Compression Fields	ICCVW	2025-04	-
OmniFuse: Composite Degradation-Robust Image Fusion with Language-Driven Semantics	TPAMI	2025-05	Github
Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model	arXiv	2025-05	-
Vision-Language Model Priors-Driven State Space Model for Infrared-Visible Image Fusion	SPL	2025-06	-
UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration	arXiv	2025-07	-
Follow-Your-Instruction: A Comprehensive MLLM Agent for World Data Synthesis	arXiv	2025-08	-
Dataset Distillation for Super-Resolution without Class Labels and Pre-trained Models	arXiv	2025-09	Github
Evaluating Robustness of Vision-Language Models Under Noisy Conditions	arXiv	2025-09	-
VLM-Guided Inpainting for Anomaly Detection	Journal of Multimedia Information System	2025-09	-
Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation	arXiv	2025-10	Github
Generative Image Restoration and Super-Resolution using Physics-Informed Synthetic Data for Scanning Tunneling Microscopy	arXiv	2025-10	Github
DiffSeg30k: A Multi-Turn Diffusion Editing Benchmark for Localized AIGC Detection	arXiv	2025-11	HuggingFace
FOCA: Frequency-Oriented Cross-Domain Forgery Detection, Localization and Explanation via Multi-Modal Large Language Model	ICASSP	2026-02	-
Universal Anti-forensics Attack against Image Forgery Detection via Multi-modal Guidance	arXiv	2026-02	-
Generalizable Detection of AI Generated Images with Large Models and Fuzzy Decision Tree	arXiv	2026-03	-
When Detectors Forget Forensics: Blocking Semantic Shortcuts for Generalizable AI-Generated Image Detection	arXiv	2026-03	-

Contributors

19 commits

10 commits

5 commits

5 commits

Similar repos

YingqingHe/Awesome-LLMs-meet-Multimodal-Generation

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

552

friedrichor/Awesome-Multimodal-Papers

A curated list of awesome Multimodal studies.

346

Purshow/Awesome-Unified-Multimodal

📖 This is a repository for organizing papers, codes, and other resources related to unified…

366

uncbiag/Awesome-Foundation-Models

A curated list of foundation models for vision and language tasks

1,177

ATH-MaaS/Awesome-Unified-Multimodal-Models

Awesome Unified Multimodal Models

1,321

aimerou/awesome-ai-papers

A curated list of the most impressive AI papers

1,310

harrytea/Awesome-Document-Understanding

Document Artifical Intelligence

202

zhengli97/Awesome-Prompt-Adapter-Learning-for-VLMs

A curated list of awesome prompt/adapter learning methods for vision-language models like CLIP.

796

ChunmingHe/awesome-multimodal-large-language-models-in-low-level-vision

39 commits

updated Jul 1, 2026

See the code

README

Awesome Multimodal Large Language Models In Low-level Vision

🔥A curated list of awesome Multimodal Large Language Models(MLLMs) & Vision-Language Models(VLMs) in low-level vision.🔥

Please feel free to offer your suggestions in the Issues and pull requests to add links.

[ Last updated at 2026/07/01 ]

Awesome Multimodal Large Language Models In Low-level Vision

Title	Venue	Date	Code
Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAG	GRSM	2024-11	Github
FeatSharp: Your Vision Model Features, Sharper	ICML	2025-02	Github
GenDR: Lightning Generative Detail Restorator	arXiv	2025-03	Github
Ultra High-Resolution Image Inpainting with Patch-Based Content Consistency Adapter	arXiv	2025-10	Github
Global-Local Dual Perception for MLLMs in High-Resolution Text-Rich Image Translation	arXiv	2026-02	-

1.1.2 Feature Fusion

Title	Venue	Date	Code
Exploiting Diffusion Prior for Real-World Image Super-Resolution	IJCV	2023-05	Github
MoVA: Adapting Mixture of Vision Experts to Multimodal Context	NeurIPS	2024-04	Github
CLIP-aware Domain-Adaptive Super-Resolution	MMS	2025-05	-
RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration	arXiv	2025-09	Github
Vision-Language Alignment from Compressed Image Representations using 2D Gaussian Splatting	arXiv	2025-09	-
LP-LLM: End-to-End Real-World Degraded License Plate Text Recognition via Large Multimodal Models	arXiv	2026-01	-
Rethinking Cross-Domain Evaluation for Face Forgery Detection with Semantic Fine-grained Alignment and Mixture-of-Experts	arXiv	2026-04	Github

1.2 Language Branch Adaptation: Bridging Modalities

1.2.1 Prompt Learning Strategies

Title	Venue	Date	Code
Iterative Prompt Learning for Unsupervised Backlit Image Enhancement	ICCV	2023-03	Github
Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration	CVPR	2023-12	-
RESTORE: Towards Feature Shift for Vision-Language Prompt Learning	arXiv	2024-03	Github
FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training	arXiv	2024-11	Github
RAP-SR: RestorAtion Prior Enhancement in Diffusion Models for Realistic Image Super-Resolution	AAAI	2024-12	Github
Causality-guided Prompt Learning for Vision-language Models via Visual Granulation	ICCV	2025-09	Github
Generalizable Face Forgery Detection via Separable Prompt Learning	arXiv	2026-04	Github

1.2.2 Instruction Tuning for Restoration

Title	Venue	Date	Code
Q-Instruct: Improving Low-Level Visual Abilities for Multi-Modality Foundation Models	CVPR	2023-11	Github
Lumina-OmniLV: A Unified Multimodal Framework for General Low-Level Vision	arXiv	2025-04	-

1.3 Output Head Adaptation: From Tokens to Pixels

1.3.1 Tokenizer–Decoder Framework

Title	Venue	Date	Code
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks	CVPR	2023-11	HuggingFace
LISA: Reasoning Segmentation via Large Language Model	CVPR	2023-08	Github
Beyond Text: Frozen Large Language Models in Visual Signal Comprehension	CVPR	2024-03	Github
Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models	ICCV	2025-03	Github
SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generation	arXiv	2025-03	-
Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Training	ICCV	2025-07	Github
Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?	ICML	2026-06	Github

1.4 Parameter-Efficient Fine-Tuning in Restoration

1.4.1 LoRA & Adapter Integration

Title	Venue	Date	Code
Explicit Visual Prompting for Low-Level Structure Segmentations	CVPR	2023-03	Github
One-Step Effective Diffusion Network for Real-World Image Super-Resolution	NeurIPS	2024-06	Github
LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration	arXiv	2024-10	Github
TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution	CVPR	2024-11	Github
Acquire and then Adapt: Squeezing out Text-to-Image Model for Image Restoration	CVPR	2025-04	-
Demystifying the Visual Quality Paradox in Multimodal Large Language Models	arXiv	2025-06	-
RDDM: Practicing RAW Domain Diffusion Model for Real-world Image Restoration	arXiv	2025-08	-
BIR-Adapter: A Low-Complexity Diffusion Model Adapter for Blind Image Restoration	arXiv	2025-09	-
One-Step Diffusion Transformer for Controllable Real-World Image Super-Resolution	arXiv	2025-11	Github

1.4.2 Freezing Strategies

Title	Venue	Date	Code
Grounding Language Models to Images for Multimodal Inputs and Outputs :contentReference[oaicite:0]{index=0}	ICML	2023-01	Github
Prismer: A Vision-Language Model with Multi-Task Experts	TMLR	2023-03	Github
LM4LV: A Frozen Large Language Model for Low-level Vision Tasks	arXiv	2024-05	Github
Frozen Transformers in Language Models Are Effective Visual Encoder Layers	ICLR	2023-10	Github

Title	Venue	Date	Code
TextIR: A Simple Framework for Text-based Editable Image Restoration	TVCG	2023-02	Github
Pixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization	ECCV	2023-08	Github
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution	CVPR	2023-11	Github
Textual Prompt Guided Image Restoration	EAAI	2023-12	Github
Low-light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion	TOMM	2024-01	Github
Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild	CVPR	2024-01	Github
Image Fusion via Vision-Language Model	ICML	2024-02	Github
Boosting Image Restoration via Priors from Pre-trained Models	CVPR	2024-03	Github
MMGInpainting: Multi-Modality Guided Image Inpainting Based on Diffusion Models	TMM	2024-03	Github
Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion	CVPR	2024-03	Github
XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution	ECCV	2024-03	Github
Photo-Realistic Image Restoration in the Wild with Controlled Vision-Language Models	CVPRW	2024-04	Github
Unsupervised Image Prior via Prompt Learning and CLIP Semantic Guidance for Low-Light Image Enhancement	CVPRW	2024-05	-
SSP-IR: Semantic and Structure Priors for Diffusion-based Realistic Image Restoration	TCSVT	2024-07	Github
Beyond Pixels: Text Enhances Generalization in Real-World Image Restoration	arXiv	2024-12	-
CLIP-SR: Collaborative Linguistic and Image Processing for Super-Resolution	TMM	2024-12	Github
MGFusion: a multimodal large language model-guided information perception for infrared and visible image fusion	Front. Neurorobot.	2024-12	-
Textual-Visual Interaction for Enhanced Single Image Deraining using Adapter-Tuned VLMs	arXiv	2024-12	Github
Unified Image Restoration and Enhancement: Degradation Calibrated Cycle Reconstruction Diffusion Model	arXiv	2024-12	Github
Multi-Text Guidance Is Important: Multi-Modality Image Fusion via Large Generative Vision-Language Model	IJCV	2025-02	Github
MegaSR: Mining Customized Semantics and Expressive Guidance for Image Super-Resolution	arXiv	2025-03	Github
The Power of Context: How Multimodality Improves Image Super-Resolution	CVPR	2025-03	-
Coarse-to-fine text injecting for realistic image super-resolution	Neurocom	2025-02	-
DeepSPG: Exploring Deep Semantic Prior Guidance for Low-light Image Enhancement with Multimodal Learning	arXiv	2025-04	Github
CTD-inpainting: Towards the Coherence of Text-driven Inpainting with Blended Diffusion	Information Fusion	2025-04	-
Exploiting Diffusion Prior for Real-World Image Dehazing with Unpaired Training	AAAI	2025-03	Github
A Preliminary Study for GPT-4o on Image Restoration	arXiv	2025-05	Github
Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and Preference Alignment	NeurIPS	2025-05	Github
Text-Aware Image Restoration with Diffusion Models	arXiv	2025-06	Github
LD-RPS: Zero-Shot Unified Image Restoration via Latent Diffusion Recurrent Posterior Sampling	ICCV	2025-07	Github
LLaVA-based semantic feature modulation diffusion model for underwater image enhancement	InfFus	2025-07	-
Harnessing Diffusion-Yielded Score Priors for Image Restoration	SIGGRAPH Asia	2025-07	Github
Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancement	arXiv	2025-07	Github
HVI-CIDNet+: Beyond Extreme Darkness for Low-Light Image Enhancement	arXiv	2025-07	Github
ModalFormer: Multimodal Transformer for Low-Light Image Enhancement	arXiv	2025-07	Github
AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception	arXiv	2025-08	Github
MambaTrans: Multimodal Fusion Image Translation via Large Language Model Priors for Downstream Visual Tasks	arXiv	2025-08	-
LucidFlux: Caption-Free Universal Image Restoration via a Large-Scale Diffusion Transformer	arXiv	2025-09	Github
Vision-Language Model Guided Image Restoration	arXiv	2025-12	-
Zero-Shot Image Super-Resolution Using Prompt-Driven Vision-Language Foundation Models Without Task-Specific Fine-Tuning	arXiv	2025-09	-
Extreme Blind Image Restoration via Prompt-Conditioned Information Bottleneck	arXiv	2025-10	-
A structural information-guided cross-modal method for damaged inscription inpainting via vision-language models	npj Heritage Science	2025-09	Github
GLYPH-SR: Can We Achieve Both High-Quality Image Super-Resolution and High-Fidelity Text Recovery via VLM-Guided Latent Diffusion Model?	arXiv	2025-10	-
Unified Diffusion Transformer for High-fidelity Text-Aware Image Restoration	arXiv	2025-12	Github
SCENE: Semantic-aware Codec Enhancement with Neural Embeddings	ICASSP	2026-01	-
DACESR: Degradation-Aware Conditional Embedding for Real-World Image Super-Resolution	TIP	2026-02	Github
QUSR: Quality-Aware and Uncertainty-Guided Image Super-Resolution Diffusion Model	ICASSP	2026-03	Github
Enhancing Single-Image Facial Demorphing using Multimodal Large Language Models	arXiv	2026-05	-
Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts	arXiv	2026-05	-

2.1.2 Language-Driven Manipulation

Title	Venue	Date	Code
InstructPix2Pix: Learning to Follow Image Editing Instructions	CVPR	2022-11	Github
StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model	ICCV	2023-03	Github
L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors	NeurIPS	2023-05	Github
L-CoIns: Language-Based Colorization With Instance Awareness	CVPR	2023-06	Github
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior	ECCV	2023-08	Github
LLMGA: Multimodal Large Language Model based Generation Assistant	ECCV	2023-11	Github
SPIRE: Semantic Prompt-Driven Image Restoration	ECCV	2023-12	-
InstructIR: High-Quality Image Restoration Following Human Instructions	ECCV	2024-01	Github
Text-guided Explorable Image Super-resolution	CVPR	2024-03	Github
Paint by Inpaint: Learning to Add Image Objects by Removing Them First	CVPR	2024-04	Github
Brush2Prompt: Contextual Prompt Generator for Object Inpainting	CVPR	2024-06	-
Range-Null Latent Prior-guided Consistency Model for Low Light Image Enhancement	arXiv	2024-09	-
PainterNet: Adaptive Image Inpainting with Actual-Token Attention and Diverse Mask Control	arXiv	2024-12	-
Instruct-CLIP: Improving Instruction-Guided Image Editing with Automated Data Refinement Using Contrastive Learning	CVPR	2025-03	Github
Instruct2See: Learning to Remove Any Obstructions Across Distributions	ICML	2025-05	Project
MIND-Edit: MLLM Insight-Driven Editing via Language-Vision Projection	arXiv	2025-05	-
Fidelity-Preserving Enhancement of Ptychography with Foundational Text-to-Image Models	arXiv	2025-09	Github
FOCUS: Unified Vision-Language Modeling for Interactive Editing Driven by Referential Segmentation	NeurIPS	2025-06	-
LaTo: Landmark-Tokenized Diffusion Transformer for Fine-Grained Human Face Editing	arXiv	2025-09	-
Query-Kontext: An Unified Multimodal Model for Image Generation and Editing	arXiv	2025-09	-
Visual Autoregressive Modeling for Instruction-Guided Image Editing	arXiv	2025-08	Github
Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editing	arXiv	2025-10	Github
Region in Context: Text-Conditioned Image Editing with Human-Like Semantic Reasoning	arXiv	2025-10	Github
UniSER: A Foundation Model for Unified Soft Effects Removal	arXiv	2025-11	-
PhotoFramer: Multi-modal Image Composition Instruction	CVPR	2025-11	Github
Generative Editing in the Joint Vision-Language Space for Zero-Shot Composed Image Retrieval	arXiv	2025-12	-
How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing	arXiv	2026-02	Github
MCIE: Multimodal LLM-Driven Complex Instruction Image Editing with Spatial Guidance	AAAI	2026-02	-
EditCaption: Human-Aligned Instruction Synthesis for Image Editing via Supervised Fine-Tuning and Direct Preference Optimization	arXiv	2026-04	-
ScribbleEdit: Synthetic Data for Image Editing with Scribbles and Text	arXiv	2026-05	-
Multimodal Image Colorization: Quantifying the Impact of Text-Conditioned Guidance on Grayscale-to-Color Translation	arXiv	2026-06	-
SeamEdit: A Black-Box VLM-Agnostic Pipeline for Large-Image Semantic Editing	arXiv	2026-06	Github
TeleStyle V2: Beyond Content-Preserving Style Transfer with Self-Distillation and Distribution-Matching-Distillation	arXiv	2026-06	Github
When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing	arXiv	2026-06	-

2.1.3 Subject-Aware Restoration

Title	Venue	Date	Code
Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model	ICLR	2024-10	-
LLV-FSR: Exploiting Large Language-Vision Prior for Face Super-resolution	arXiv	2024-11	-
InstructRestore: Region-Customized Image Restoration with Human Instructions	arXiv	2025-03	Github
PromptLNet: Region-Adaptive Aesthetic Enhancement via Prompt Guidance in Low-Light Enhancement Net	arXiv	2025-03	-
TSCnet: A Text-driven Semantic-level Controllable Framework for Customized Low-Light Image Enhancement	Neurocomputing	2025-03	Project
EAM: Enhancing Anything with Diffusion Transformers for Blind Super-Resolution	arXiv	2025-05	-
Empowering Semantic-Sensitive Underwater Image Enhancement with VLM	AAAI	2026-03	-
JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation	arXiv	2026-05	Github

2.2 VLM as Degradation Interpreter: Visual-Prompting & Context

2.2.1 Degradation Classification

Title	Venue	Date	Code
Exploring the Application of Large-Scale Pre-Trained Models on Adverse Weather Removal	TIP	2023-06	-
LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network	CVPR	2023-07	Github
AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion	ECCV	2023-10	Github
Unifying Image Processing as Visual Prompting Question Answering	ICML	2023-10	Github
Controlling Vision-Language Models for Multi-Task Image Restoration	ICLR	2024-02	Github
DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution	arXiv	2024-06	Github
DAP-LED: Learning Degradation-Aware Priors with CLIP for Joint Low-light Enhancement and Deblurring	ICRA	2024-09	-
Multi-modal degradation feature learning for unified image restoration based on contrastive learning	Neurocomputing	2024-11	-
All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model	TMM	2024-11	-
AllRestorer: All-in-One Transformer for Image Restoration under Composite Degradations	arXiv	2024-11	-
LMHaze: Intensity-aware Image Dehazing with a Large-scale Multi-intensity Real Haze Dataset	MMAsia	2024-12	-
Adaptive Fuzzy Degradation Perception Based on CLIP Prior for All-in-One Image Restoration	TFS	2024-12	Github
Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks	CVPR	2025-03	Github
DA2Diff: Exploring Degradation-aware Adaptive Diffusion Priors for All-in-One Weather Restoration	arXiv	2025-04	-
VL-UR: Vision-Language-guided Universal Restoration of Images Degraded by Adverse Weather Conditions	ICME	2025-04	-
Controlling vision-language model for enhancing image restoration	IVC	2025-04	-
CLIP-driven rain perception: Adaptive deraining with pattern-aware network routing and mask-guided cross-attention	PR	2025-06	-
Degradation-Aware Image Enhancement via Vision-Language Classification	arXiv	2025-06	-
M2Restore: Mixture-of-Experts-based Mamba-CNN Fusion Framework for All-in-One Image Restoration	TIP	2025-06	Github
Real-world super-resolution with VLM-based degradation prior learning	Sci Rep	2025-08	-
Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution	arXiv	2025-11	-
Zero-Reference Joint Low-Light Enhancement and Deblurring via Visual Autoregressive Modeling with VLM-Derived Modulation	AAAI	2025-11	Github
CLIP-Guided Unsupervised Semantic-Aware Exposure Correction	ICASSP	2026-01	-
Detecting Deepfakes with Multivariate Soft Blending and CLIP-based Image-Text Alignment	arXiv	2026-02	-
TextShield-R1: Reinforced Reasoning for Tampered Text Detection	AAAI	2026-02	Github
Understanding Degradation with Vision Language Model	arXiv	2026-02	-
Generalizable Detection of AI Generated Images with Large Models and Fuzzy Decision Tree	arXiv	2026-03	-
DistortBench: Benchmarking Vision Language Models on Image Distortion Identification	arXiv	2026-04	-

2.2.2 Description-based Restoration

Title	Venue	Date	Code
Image Super-Resolution with Text Prompt Diffusion	arXiv	2023-11	Github
Language-driven All-in-one Adverse Weather Removal	CVPR	2023-12	Github
LLMRA: Multi-modal Large Language Model based Restoration Assistant	arXiv	2024-01	-
Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration	arXiv	2024-07	Github
Training-Free Large Model Priors for Multiple-in-One Image Restoration	arXiv	2024-07	-
UniProcessor: A Text-induced Unified Low-level Image Processor	ECCV	2024-07	Github
AWRaCLe: All-Weather Image Restoration using Visual In-Context Learning	AAAI	2024-08	Github
DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation	NeurIPS	2024-10	Github
Leveraging vision-language prompts for real-world image restoration and enhancement	CVIU	2024-11	-
Leveraging Content and Context Cues for Low-Light Image Enhancement	TMM	2024-12	Github
RamIR: Reasoning and action prompting with Mamba for all-in-one image restoration	-	2025-01	-
ControlFusion: A Controllable Image Fusion Network with Language-Vision Degradation Prompts	arXiv	2025-03	Github
Prompt to Restore, Restore to Prompt: Cyclic Prompting for Universal Adverse Weather Removal	TIP	2025-03	Github
UniCoRN: Latent Diffusion-based Unified Controllable Image Restoration Network across Multiple Degradations	WACV	2025-03	Project
CLIP-RestoreX: Restore Image Structure and Perception in Exposure Correction	AAAI	2025-04	Github
Dual Prompting Image Restoration with Diffusion Transformers	CVPR	2025-04	Github
A single image deraining algorithm guided by text generation based on depth information conditions	ASOC	2025-07	Github
Adaptive multi-modal prompting for universal image restoration amidst diverse degradations	TVC	2025-05	Github
RAGSR: Regional Attention Guided Diffusion for Image Super-Resolution	arXiv	2025-08	-
Dual-Domain Perspective on Degradation-Aware Fusion: A VLM-Guided Robust Infrared and Visible Image Fusion Framework	arXiv	2025-09	-
Real-world super-resolution with VLM-based degradation prior learning	Scientific Reports	2025-08	-
Coupled Degradation Modeling and Fusion: A VLM-Guided Degradation-Coupled Network for Degradation-Aware Infrared and Visible Image Fusion	arXiv	2025-10	Github
Coupled Degradation Modeling and Fusion: A VLM-Guided Degradation-Coupled Network for Degradation-Aware Infrared and Visible Image Fusion	arXiv	2025-10	Github
MdaIF: Robust One-Stop Multi-Degradation-Aware Image Fusion with Language-Driven Semantics	AAAI	2025-11	Github
Multimodal image fusion network with prior-guided dynamic degradation removal for extreme environment perception	Sci Rep	2025-10	Github
Nested Unfolding Network for Real-World Concealed Object Segmentation	arXiv	2025-11	-
VL-UR: Vision-Language-guided Universal Restoration of Images Degraded by Adverse Weather Conditions	arXiv	2025-04	-
VLM-Augmented Degradation Modeling for Image Restoration Under Adverse Weather Conditions	arXiv	2025-11	-

2.3 VLM as Quality Evaluator: Perception and Assessment

2.3.1 No-Reference Quality Assessment

Title	Venue	Date	Code
Exploring CLIP for Assessing the Look and Feel of Images	AAAI	2022-07	Github
Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective	CVPR	2023-03	Github
VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining	CVPR	2023-03	Github
DegAE: A New Pretraining Paradigm for Low-Level Vision	CVPR	2023-06	Github
Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models	ECCV	2023-12	Github
Quality-Aware Image-Text Alignment for Opinion-Unaware Image Quality Assessment (QualiCLIP)	arXiv	2024-03	Github
Descriptive Image Quality Assessment in the Wild	arXiv	2024-05	Github
UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment	arXiv	2024-06	Github
CLIP-AGIQA: Boosting the Performance of AI-Generated Image Quality Assessment with CLIP	ICPR	2024-08	Github
Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessment	arXiv	2024-10	Github
Large Language Models for Lossless Image Compression: Next-Pixel Prediction in Language Space is All You Need	NeurIPS	2024-11	Github
Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models	AAAI	2024-11	Github
Low-Light Image Enhancement via Generative Perceptual Priors	AAAI	2024-12	Github
Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution	CVPR	2025-01	Github
CLIP-DQA: Blindly Evaluating Dehazed Images from Global and Local Perspectives Using CLIP	ISCAS	2025-02	Github
DVLTA-VQA: Decoupled Vision-Language Modeling with Text-Guided Adaptation for Blind Video Quality Assessment	arXiv	2025-04	-
Q-Adapt: Adapting LMM for Visual Quality Assessment with Progressive Instruction Tuning	arXiv	2025-04	Github
Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration	TIP	2024-08	Github
GIE-Bench: Towards Grounded Evaluation for Text-Guided Image Editing	arXiv	2025-05	Github
ImgEdit: A Unified Image Editing Dataset and Benchmark	NeurIPS	2025-05	Github
What Changed? Detecting and Evaluating Instruction-Guided Image Edits with Multimodal Large Language Models	ICCV	2025-05	Github
BPCLIP: A Bottom-up Image Quality Assessment from Distortion to Semantics Based on CLIP	ICME	2025-06	-
Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment	arXiv	2025-06	Github
Leveraging Vision-Language Models to Select Trustworthy Super-Resolution Samples Generated by Diffusion Models	TCSVT	2025-06	-
VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning	arXiv	2025-06	-
DehazeMamba: large multi-modal model guided single image dehazing via mamba	VisIntell	2025-07	-
Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution	arXiv	2025-07	-
Q-CLIP: Unleashing the Power of Vision-Language Models for Video Quality Assessment through Unified Cross-Modal Adaptation	arXiv	2025-08	-
Fine-grained Image Quality Assessment for Perceptual Image Restoration	AAAI	2025-08	Github
Refine-IQA: Multi-Stage Reinforcement Finetuning for Perceptual Image Quality Assessment	AAAI	2025-08	Github
Segmenting and Understanding: Region-aware Semantic Attention for Fine-grained Image Quality Assessment with Large Language Models	arXiv	2025-08	-
AgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessment	arXiv	2025-09	-
Describe-to-Score: Text-Guided Efficient Image Complexity Assessment	arXiv	2025-09	Github
Parameter-Efficient Adaptation of mPLUG-Owl2 via Pixel-Level Visual Prompts for NR-IQA	arXiv	2025-09	Github
Revisiting Vision–Language Foundations for No-Reference Image Quality Assessment	WACV	2025-09	-
From Filters to VLMs: Benchmarking Defogging Methods through Object Detection and Segmentation Performance	arXiv	2025-10	Github
Revisiting MLLM Based Image Quality Assessment: Errors and Remedy	AAAI	2025-11	Github
ClearAIR: A Human-Visual-Perception-Inspired All-in-One Image Restoration	AAAI	2026-01	Github
Enhancing Image Quality Assessment Ability of LMMs via Retrieval-Augmented Generation	arXiv	2026-01	-
QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding	arXiv	2026-01	Github
Q-Bench-Portrait: Benchmarking Multimodal Large Language Models on Portrait Image Quality Perception	arXiv	2026-01	-
VideoAesBench: Benchmarking the Video Aesthetics Perception Capabilities of Large Multimodal Models	arXiv	2026-01	Github
Q-Hawkeye: Reliable Visual Policy Optimization for Image Quality Assessment	arXiv	2026-01	-
ELIQ: A Label-Free Framework for Quality Assessment of Evolving AI-Generated Images	arXiv	2026-02	-
Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis	arXiv	2026-02	Github
Synthetic Image Detection with CLIP: Understanding and Assessing Predictive Cues	arXiv	2026-02	Github
CREval: An Automated Interpretable Evaluation for Creative Image Manipulation under Complex Instructions	arXiv	2026-03	Github
Evaluating Image Editing with LLMs: A Comprehensive Benchmark and Intermediate-Layer Probing Approach	arXiv	2026-03	-
ME-IQA: Memory-Enhanced Image Quality Assessment via Re-Ranking	arXiv	2026-03	-
Q-Tacit: Image Quality Assessment via Latent Visual Reasoning	arXiv	2026-03	Github
R4-CGQA: Retrieval-based Vision Language Models for Computer Graphics Image Quality Assessment	CVPR	2026-03	Github
Vision-Language Models vs Human: Perceptual Image Quality Assessment	arXiv	2026-03	-
EvaNet: Towards More Efficient and Consistent Infrared and Visible Image Fusion Assessment	TPAMI	2026-04	Github
Multi-Granularity Reasoning for Image Quality Assessment via Attribute-Aware Reinforcement Learning to Rank	arXiv	2026-04	-
NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1)	CVPRW	2026-04	Github
Redefining Quality Criteria and Distance-Aware Score Modeling for Image Editing Assessment	arXiv	2026-04	-
UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs	arXiv	2026-04	Github
Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment	arXiv	2026-05	-
Employing Vision-Language Models for Face Image Quality Assessment	FG	2026-05	Github
UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs	arXiv	2026-04	Github
DAL-PCQA: Enabling Distortion-Level and Language-Driven Reasoning for Point Cloud Quality Assessment	arXiv	2026-06	Github
Decoupling Semantics from Distortions: Multi-Scale Two-Stream Vision-Language Alignment for AI-Generated Image Quality Assessment	arXiv	2026-06	Github
LEIQ-Assessor: Multi-dimensional Quality Assessment of Low-light Enhanced Images via Multi-task Learning	arXiv	2026-06	Github
LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models	arXiv	2026-06	Github
Tool-IQA: Augmenting Image Quality Assessment with Simple Tools	arXiv	2026-06	Github

2.3.2 Semantic Consistency Loss

Title	Venue	Date	Code
Cycle Contrastive Adversarial Learning for Unsupervised image Deraining	arXiv	2024-07	Github
Contourlet Residual for Prompt Learning Enhanced Infrared Image Super-Resolution	ECCV	2024-09	Github
Underwater Diffusion Attention Network with Contrastive Language-Image Joint Learning for Underwater Image Enhancement	arXiv	2025-05	-
Unveiling the Underwater World: CLIP Perception Model-Guided Underwater Image Enhancement	PR	2025-06	-
One-Step Diffusion-based Real-World Image Super-Resolution with Visual Perception Distillation	arXiv	2025-06	-
From Enhancement to Understanding: Build a Generalized Bridge for Low-light Vision via Semantically Consistent Unsupervised Fine-tuning	ICCV	2025-07	Github
WeatherCycle: Unpaired Multi-Weather Restoration via Color Space Decoupled Cycle Learning	arXiv	2025-09	-
Where, What, Why, and Importance: Structured Defect Grounding for Text-to-Image Feedback	arXiv	2026-06	Github

2.3.3 Feedback Loops

Title	Venue	Date	Code
HazeCLIP: Towards Language Guided Real-World Image Dehazing	ICASSP	2024-07	Github
Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models	ECCV	2024-09	Github
DSPO: Direct Semantic Preference Optimization for Real-World Image Super-Resolution	arXiv	2025-04	-
SnowMaster: Comprehensive Real-world Image Desnowing via MLLM with Multi-Model Feedback Optimization	CVPR	2025-06	Github
EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing	arXiv	2025-09	Github
Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking	arXiv	2025-09	-
Learning an Image Editing Model without Image Editing Pairs	arXiv	2025-10	-
Test-Time Preference Optimization for Image Restoration	AAAI	2025-11	Github
UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in Reinforcement Learning	arXiv	2025-11	Github
UARE: Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancement	arXiv	2025-12	Github
Zoom-IQA: Image Quality Assessment with Reliable Region-Aware Reasoning	arXiv	2026-01	-
CoCoEdit: Content-Consistent Image Editing via Region Regularized Reinforcement Learning	IJCAI	2026-02	Github
EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing	arXiv	2026-03	Github
HP-Edit: A Human-Preference Post-Training Framework for Image Editing	CVPR	2026-04	-
Q-DeepSight: Incentivizing Thinking with Images for Image Quality Assessment and Refinement	arXiv	2026-04	-
GGT-100K: Generative Ground Truth for Generalizable Real-World Image Restoration	arXiv	2026-05	Github
Stable-Layers: Fine-Tuning Image Layer Decomposition Models with VLM-Scored Reinforcement Learning	arXiv	2026-05	Project
White-Balance First, Adjust Later: Cross-Camera Color Constancy via Vision-Language Evaluation	CVPR	2026-05	Github

Title	Venue	Date	Code
LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing	arXiv	2023-11	Github
Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Models	IJCAI	2024-01	-
Image Inpainting Models are Effective Tools for Instruction-guided Image Editing	CVPRW	2024-07	Github
RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models	NeurIPS	2024-07	-
VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing	NeurIPS	2024-10	Github
LLMCO4MR: LLMs-Aided Neural Combinatorial Optimization for Ancient Manuscript Restoration from Fragments with Case Studies on Dunhuang	ECCV	2024-11	-
I Dream My Painting: Connecting MLLMs and Diffusion Models via Prompt Generation for Text-Guided Multi-Mask Inpainting	WACV	2024-11	Github
BrushEdit: All-In-One Image Inpainting and Editing	NeurIPS	2024-12	Github
Hybrid Agents for Image Restoration	arXiv	2025-03	-
Multi-Agent Image Restoration	arXiv	2025-03	-
JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration	CVPR	2025-04	Github
Q-Agent: Quality-Driven Chain-of-Thought Image Restoration Agent through Robust Multimodal Large Language Model	arXiv	2025-04	-
LEGO: LLM-enhanced genetic optimization for underwater robot image restoration	PR	2025-05	Github
Light as Deception: GPT-driven Natural Relighting Against Vision-Language Pre-training Models	arXiv	2025-05	-
MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills	TOG	2025-05	Github
PhotoArtAgent: Intelligent Photo Retouching with Language Model-Based Artist Agents	arXiv	2025-05	-
JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent	NeurIPS	2025-06	Github
EdiVal-Agent: An Object-Centric Framework for Automated, Scalable, Fine-Grained Evaluation of Multi-Turn Editing	arXiv	2025-09	Github
InstructVTON: Optimal Auto-Masking and Natural-Language-Guided Interactive Style Control for Inpainting-Based Virtual Try-On	arXiv	2025-09	-
Prompt-Driven Image Analysis with Multimodal Generative AI: Detection, Segmentation, Inpainting, and Interpretation	arXiv	2025-09	-
MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration	JSTSP	2025-10	Github
From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustration	arXiv	2025-10	Github
Image-POSER: Reflective RL for Multi-Expert Image Generation and Editing	arXiv	2025-11	-
T2T-VICL: Unlocking the Boundaries of Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs	arXiv	2025-11	-
RePlan: Reasoning-guided Region Planning for Complex Instruction-based Image Editing	arXiv	2025-12	Github
ForgeryVCR: Visual-Centric Reasoning via Efficient Forensic Tools in MLLMs for Image Forgery Detection and Localization	arXiv	2026-02	Project
Interaction-Consistent Object Removal via MLLM-Based Reasoning	arXiv	2026-02	-
MSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing	arXiv	2026-03	-
PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning	arXiv	2026-03	Project
TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration	arXiv	2026-03	-
GenSmoke-GS: A Multi-Stage Method for Novel View Synthesis from Smoke-Degraded Images Using a Generative Model	CVPRW	2026-04	Github
DiTTo: Scalable Order-aware All-in-One Image Restoration Agent	arXiv	2026-05	Project
From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editing	arXiv	2026-05	Project
IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment	CVPR	2026-06	Github
Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution	arXiv	2026-06	-

Title	Venue	Date	Code
Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement	arXiv	2023-11	-
Anywhere: A Multi-Agent Framework for User-Guided, Reliable, and Diverse Foreground-Conditioned Image Generation	AAAI	2024-04	Github
EditScribe: Non-Visual Image Editing with Natural Language Verification Loops	ASSETS	2024-08	-
An Intelligent Agentic System for Complex Image Restoration Problems	ICLR	2024-10	Github
CURVE: CLIP-Utilized Reinforcement Learning for Visual Image Enhancement via Simple Image Processing	ICIP	2025-05	-
RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought	arXiv	2025-06	Github
4KAgent: Agentic Any Image to 4K Super-Resolution	NeurIPS	2025-07	Github
Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing	arXiv	2025-10	Github
Dynamic VLM-Guided Negative Prompting for Diffusion Models	NeurIPS	2025-10	-
From Pixels to Paths: A Multi-Agent Framework for Editable Scientific Illustration	arXiv	2025-10	Github
Q-Probe: Scaling Image Quality Assessment to High Resolution via Context-Aware Agentic Probing	arXiv	2026-01	-
Making Image Editing Easier via Adaptive Task Reformulation with Agentic Executions	arXiv	2026-04	-
EditRefiner: A Human-Aligned Agentic Framework for Image Editing Refinement	NeurIPS	2026-05	Github

Title	Venue	Date	Code
HuatuoGPT-Vision: Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale	EMNLP	2024-06	Github
MLeVLM: Improve Multi-level Progressive Capabilities based on Multimodal Large Language Model for Medical Visual Question Answering	ACL	2024-08	Github
PET Image Denoising via Text-Guided Diffusion: Integrating Anatomical Priors through Text Prompts	arXiv	2025-02	-
AgentPolyp: Accurate Polyp Segmentation via Image Enhancement Agent	SPL	2025-04	-
Text controllable PET denoising	SPIE	2026-01	-
FluoCLIP: Stain-Aware Focus Quality Assessment in Fluorescence Microscopy	arXiv	2026-02	Project
MedQ-UNI: Toward Unified Medical Image Quality Assessment and Restoration via Vision-Language Modeling	arXiv	2026-03	-

3.1.2 CT and MRI

Title	Venue	Date	Code
Controllable Text-to-Image Synthesis for Multi-Modality MR Images	WACV	2024-01	-
Low-dose CT Denoising with Language-engaged Dual-space Alignment	arXiv	2024-03	Github
Dual-Domain CLIP-Assisted Residual Optimization Perception Model for Metal Artifact Reduction	TRPMS	2024-08	-
A-IDE: Agent-Integrated Denoising Experts	arXiv	2025-03	-
Patient-Level Anatomy Meets Scanning-Level Physics: Personalized Federated Low-Dose CT Denoising Empowered by Large Language Model	CVPR	2025-03	Github
TDMF: Text-Guided Denoising and Interactive Medical Image Fusion	ICASSP	2025-04	-
Taming Stable Diffusion for Computed Tomography Blind Super-Resolution	MICCAI WS	2025-06	Github
LangMamba: A Language-driven Mamba Framework for Low-dose CT Denoising with Vision-language Models	TRPMS	2025-07	Github
Prompt-Conditioned FiLM and Multi-Scale Fusion on MedSigLIP for Low-Dose CT Quality Assessment	arXiv	2025-11	Github
Vision–Language Controlled Deep Unfolding for Joint Medical Image Restoration and Segmentation	arXiv	2026-01	-
CT-DegradBench: A Physics-Informed Benchmark for CT Degradation Detection and Severity Estimation	arXiv	2026-05	Github

3.2 Remote Sensing Data Processing

3.2.1 Spatial-domain Tasks

Title	Venue	Date	Code
Zooming Out on Zooming In: Advancing Super-Resolution for Remote Sensing	arXiv	2023-11	Github
Large Foundation Model Empowered Discriminative Underwater Image Enhancement	IEEE	2025-01	Gitee
Semantic-Aware Guidance for Blind Super-Resolution of Remote Sensing Images	GRSL	2025-01	-
SeG-SR: Integrating Semantic Knowledge into Remote Sensing Image Super-Resolution via Vision-Language Model	TGRS	2025-05	Github
SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models	arXiv	2026-05	Github

3.2.2 Spectral-domain Tasks

Title	Venue	Date	Code
PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation	arXiv	2024-11	Github
Leveraging Language-Aligned Visual Knowledge for Remote Sensing Image Spectral Super-Resolution	WHISPERS	2024-12	Github
MP-HSIR: A Multi-Prompt Framework for Universal Hyperspectral Image Restoration	ICCV	2025-03	Github

3.3 Other Extended Applications

3.3.1 CAD

Title	Venue	Date	Code
Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models	CVPR	2024-11	Github
ChatCAD: An MLLM-Guided Framework for Zero-shot CAD Drawing Restoration	ICASSP	2025-04	-
ImageRAG: Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAG	IEEE	2024-11	Github
Semantic Document Derendering: SVG Reconstruction via Vision-Language Modeling	AAAI	2025-11	Github
ReCAD: Reinforcement Learning Enhanced Parametric CAD Model Generation with Vision-Language Models	AAAI	2025-12	-

3.3.2 Video Processing Tasks

Title	Venue	Date	Code
Towards Language-Driven Video Inpainting via Multimodal Large Language Models	CVPR	2024-01	Github
Triplane-Smoothed Video Dehazing with CLIP-Enhanced Generalization	IJCV	2024-08	-
Universal Video Face Restoration Method Based on Vision-Language Model	ACML	2024-09	-
StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification	arXiv	2024-11	Github
Towards General-Purpose Video Reconstruction through Synergy of Grid-Splicing Diffusion and Large Language Models	TCSVT	2025-03	-
Grounding Degradations in Natural Language for All-In-One Video Restoration	arXiv	2025-07	-
When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs	arXiv	2025-09	-
Edit-Your-Interest: Efficient Video Editing via Feature Most-Similar Propagation	arXiv	2025-10	-
Zero-Shot Video Deraining with Video Diffusion Models	WACV	2025-11	Github
ReViSE: Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning	arXiv	2025-12	Github
S3-CLIP: Video Super Resolution for Person-ReID	WACV	2026-01	Github
Video-Rate Streaming Stylization on a Vision-Aware MLLM-Conditioned Edit Diffusion: Asymmetric Batched Inference on a Distilled UNet + MLLM Text Encoder	arXiv	2026-06	Github

3.3.3 3D Processing Tasks

Title	Venue	Date	Code
RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction	ICRA	2023-07	Github
DepthLM: Metric Depth From Vision Language Models	arXiv	2025-09	Github
SRSplat: Feed-Forward Super-Resolution Gaussian Splatting from Sparse Multi-View Images	AAAI	2025-11	Github
GS-Light: Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relighting	arXiv	2025-11	-
Lang3D-XL: Language Embedded 3D Gaussians for Large-scale Scenes	SIGGRAPH Asia	2025-12	Github

Datasets

Task	Dataset	Size	Repo / Download	Description
SR	DIV2K	900/100	Download	High-resolution natural images for super-resolution with synthetic degradations.
SR	RealSR	595	Repo	Real-captured paired low-resolution and high-resolution images using focal-length changes.
SR	DRealSR	2,507	Repo	Real-world super-resolution pairs collected across indoor and outdoor scenes.
LLIE	LOLv1	585/15	Download	Paired low-light and normal-light real images for low-light enhancement.
LLIE	MIT-Adobe FiveK	25,000	Download	Expert-retouched photo pairs for illumination and tone related enhancement tasks.
LLIE	LOLv2	1,589/200	Repo	Combined real and synthetic paired data for low-light enhancement.
Dehaze	RESIDE	13,000/990	Repo	Dehazing benchmark including synthetic and real-world subsets.
Dehaze	NH-Haze	55	Download	Real paired outdoor images with non-homogeneous haze.
Dehaze	Haze-4K	4,000	Repo	Synthetic 4K dehazing pairs with auxiliary physical annotations.
Inpainting	CelebA	200,000	Download	Large-scale face dataset used for face-centric restoration and inpainting.
Inpainting	CelebA-HQ	30,000	Repo	High-quality face images derived from CelebA for high-fidelity restoration.
Inpainting, Derain	MSCOCO	328,124	Download	Large-scale natural images with segmentation annotations for generic restoration and editing.
Derain	Rain100H	1,800/100	Download	Synthetic heavy-rain streak pairs for supervised deraining.
Derain	RainDrop	861/239	Repo	Real paired images with adherent raindrops for deraining and restoration.
Deblur	GoPro	2,103/1,111	Download	Motion blur benchmark synthesized from high-frame-rate video frames.
Deblur	RealBlur	3,758/980	Repo	Real paired short-exposure and long-exposure images for motion deblurring.
Deblur	HIDE	8,422	Repo	Human-centered motion deblurring dataset with complex dynamic motion.
CT denoising	2016 NIH-AAPM-Mayo	5,936	Download	Low-dose CT dataset used for dose reduction and denoising research.
CT denoising	Mayo-2016	4,800/1,136	Download	Challenge dataset used for low-dose CT denoising and reconstruction.
CT denoising	Mayo-2020	2,400/580	Download	Preprocessed low-dose CT dataset for training and evaluation.

Title	Venue	Date	Code	Note
Diffusion Models in Low-Level Vision: A Survey	TPAMI	2024-06	Repo	Diffusion-based methods for low-level vision, including task formulations, architectures, and training design patterns.
A Survey on All-in-One Image Restoration: Taxonomy, Methods, and Future Directions	arXiv	2024-10	Repo	All-in-one image restoration models covering multi-task, multi-degradation, unified training, and evaluation protocols.
Instruction Guided Editing Controls for Images and Multimedia: A Survey in the Era of Large Language Models	arXiv	2024-11	Repo	Instruction-driven image and multimedia editing, covering control types, model families, and evaluation for edit fidelity and consistency.
Vision-Language Models for Vision Tasks: A Survey	TPAMI	2023-04	Repo	VLM methodology for vision tasks, including pretraining, transfer learning, distillation, datasets, and benchmarks.
A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges	CVPRW	2025-01	Repo	LVLM design and evaluation, focusing on alignment, benchmark suites, failure modes, and measurement practices.
A Survey on Multimodal Large Language Models	NSR	2023-06	Repo	Core MLLM architectures, training data, instruction tuning, evaluation, and common issues such as hallucination.
Vision Language Models: A Survey of 26K Papers	arXiv	2025-10	-	Trend analysis and taxonomy at scale, summarizing research directions from a large paper collection.
Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions	arXiv	2024-02	-	Broad overview of VLM model families, training paradigms, and benchmarks, with an emphasis on capabilities and limitations.
Vision Encoders in Vision-Language Models: A Survey	Tech Report	2025-12	-	Vision encoder design choices for VLMs, including architecture families, scaling trends, and practical tradeoffs.

Other Awesome VLM Papers

Title	Venue	Date	Code
MMAIF: Multi-task and Multi-degradation All-in-One for Image Fusion with Language Guidance	arXiv	2025-03	Github
Adding Conditional Control to Text-to-Image Diffusion Models	ICCV	2023-02	Github
GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions	IJCV	2023-05	Github
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding	CVPR	2023-11	Github
IMProv: Inpainting-based Multimodal Prompting for Computer Vision Tasks	TMLR	2023-12	Github
Generative Multi-Modal Knowledge Retrieval with Large Language Models	AAAI	2024-01	Github
CoLLaVO: Crayon Large Language and Vision mOdel	arXiv	2024-02	Github
Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond	ACL	2024-02	Github
MMCTAgent: Multi-modal Critical Thinking Agent Framework for Complex Visual Reasoning	NeurIPSW	2024-05	Github
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding	NeurIPS	2024-06	Github
Multi-Expert Adaptive Selection: Task-Balancing for All-in-One Image Restoration	arXiv	2024-07	Github
Vision Language Models Are Blind: Failing to Translate Detailed Visual Features into Words	ACCV	2024-07	Github
Learning A Low-Level Vision Generalist via Visual Task Prompt	ACM MM	2024-08	Github
On Domain-Adaptive Post-Training for Multimodal Large Language Models	EMNLP	2024-11	Github
ForgeryGPT: Multimodal Large Language Model For Explainable Image Forgery Detection and Localization	arXiv	2024-10	-
Scene Text Detection in Foggy Weather Utilizing Knowledge Distillation of Diffusion Models	SPL	2025-02	-
Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent	arXiv	2025-03	Github
Why Compress What You Can Generate? When GPT-4o Generation Ushers in Image Compression Fields	ICCVW	2025-04	-
OmniFuse: Composite Degradation-Robust Image Fusion with Language-Driven Semantics	TPAMI	2025-05	Github
Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model	arXiv	2025-05	-
Vision-Language Model Priors-Driven State Space Model for Infrared-Visible Image Fusion	SPL	2025-06	-
UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration	arXiv	2025-07	-
Follow-Your-Instruction: A Comprehensive MLLM Agent for World Data Synthesis	arXiv	2025-08	-
Dataset Distillation for Super-Resolution without Class Labels and Pre-trained Models	arXiv	2025-09	Github
Evaluating Robustness of Vision-Language Models Under Noisy Conditions	arXiv	2025-09	-
VLM-Guided Inpainting for Anomaly Detection	Journal of Multimedia Information System	2025-09	-
Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation	arXiv	2025-10	Github
Generative Image Restoration and Super-Resolution using Physics-Informed Synthetic Data for Scanning Tunneling Microscopy	arXiv	2025-10	Github
DiffSeg30k: A Multi-Turn Diffusion Editing Benchmark for Localized AIGC Detection	arXiv	2025-11	HuggingFace
FOCA: Frequency-Oriented Cross-Domain Forgery Detection, Localization and Explanation via Multi-Modal Large Language Model	ICASSP	2026-02	-
Universal Anti-forensics Attack against Image Forgery Detection via Multi-modal Guidance	arXiv	2026-02	-
Generalizable Detection of AI Generated Images with Large Models and Fuzzy Decision Tree	arXiv	2026-03	-
When Detectors Forget Forensics: Blocking Semantic Shortcuts for Generalizable AI-Generated Image Detection	arXiv	2026-03	-

Similar repos

YingqingHe/Awesome-LLMs-meet-Multimodal-Generation

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

552

friedrichor/Awesome-Multimodal-Papers

A curated list of awesome Multimodal studies.

346

Purshow/Awesome-Unified-Multimodal

📖 This is a repository for organizing papers, codes, and other resources related to unified…

366

uncbiag/Awesome-Foundation-Models

A curated list of foundation models for vision and language tasks

1,177

ATH-MaaS/Awesome-Unified-Multimodal-Models

Awesome Unified Multimodal Models

1,321

aimerou/awesome-ai-papers

A curated list of the most impressive AI papers

1,310

harrytea/Awesome-Document-Understanding

Document Artifical Intelligence

202

zhengli97/Awesome-Prompt-Adapter-Learning-for-VLMs

A curated list of awesome prompt/adapter learning methods for vision-language models like CLIP.

796

Contributors

19 commits

10 commits

5 commits

5 commits

ChunmingHe/awesome-multimodal-large-language-models-in-low-level-vision

README

Awesome Multimodal Large Language Models In Low-level Vision

Contents

Contributors

Similar repos

ChunmingHe/awesome-multimodal-large-language-models-in-low-level-vision

README

Awesome Multimodal Large Language Models In Low-level Vision

Contents

Similar repos

Contributors