Awesome Agentic Visual Generation

Agentic Visual Generation: From Generative Models to Agentic Control
Yinming Huang^1,2,*, Shuyuan Tu^1,*, Xi Yan^1,*, Jiahao Zhan⁴, Zihan Yang¹, Zhen Xing³, Hui Zhang¹, Tiehua Zhang⁵, Yu-Gang Jiang¹, Zuxuan Wu^1,2,†
[¹Fudan University; ²Shanghai Innovative Institute; ³Wan Team, Alibaba Tongyi Lab; ⁴CUHK, MMLab; ⁵School of Computer Science and Technology, Tongji University]
[^*Equal contribution; ^†Corresponding author]

❤️ If you find our work useful, please consider giving a star ⭐ to this GitHub repository ❤️.

News

• [2026-09-09]: 🔥 The paper is now available on arXiv.

Figure 1. Visual overview of controller decision-making scope.
Vector PDF · arXiv paper

Pull requests are very welcome! Please help us add new papers, official resources, or corrections.

A curated and taxonomy-driven collection of 313 reviewed records spanning the L0 boundary and four controller-capability levels. The repository covers image generation and editing, video generation and editing, slide and user-interface generation, 3D scene construction, and world models.

The primary organization follows one rule: agenticity is determined by the deepest point in a generation trajectory at which the controller can causally change a future generation decision. Modality and mechanism are orthogonal tags. This prevents tool use, multi-agent design, memory, or reinforcement learning from being treated as agenticity levels by themselves.

Contents

• Paper on arXiv
• Scope and inclusion rule
• Controller-capability taxonomy
• Full classification standard
• Structured corpus
• L0: Fixed Support
• L1: Conditioning Control
• L2: Execution Control
• L3: Outcome-Adaptive Control
• L4: Experience-Adaptive Control
• Evaluation, Benchmarks, and Reward Models
• Contact
• Community

Scope and inclusion rule

An agentic visual generation system contains a visual generator or editor and a controller that makes generation-level decisions. In most current systems, an LLM, VLM, or MLLM is the controller and the visual generator is one of its tools. The controller may also be hybrid or internalized in a unified model, but generation capability alone does not establish agenticity.

We classify a system by the highest controller capability demonstrated by the complete method:

• The action type does not determine the level. A prompt rewrite before generation is L1, while a prompt rewrite caused by inspection of a generated image is L3.
• Tool use describes the action space. Multi-agent design describes the topology. Reinforcement learning describes a training method. None of them alone determines the level.
• A paper appears once in L0-L4 according to its maximum demonstrated level. The Path column records the lower-level capabilities that it also contains.
• L0 records may use agentic machinery during data construction or optimization, but the deployed visual system still follows a fixed rule. Stand-alone evaluators, reward models, benchmarks, and additional foundational components are listed separately.

Controller-capability taxonomy

The controller-capability levels form a progression above the L0 boundary:

conditions  ->  execution  ->  current trajectory  ->  future trajectories
    L1              L2                 L3                      L4

Modality tags used below are Image, Editing, Video, Slide, UI, 3D, and World.

The decisive boundary is decision-making scope, not architectural complexity. Multi-agent role decomposition and internal search can remain L1 when they only construct a specification. L2 requires the controller to invoke actual visual generation, editing, rendering, or artifact-mutating operations. L3 additionally requires a causal outcome-to-action link in the current trajectory, and L4 requires a persistent update across independent tasks. See TAXONOMY.md for the decision procedure, counterexamples, and evidence policy.

Figure 2. Representative-paper roadmap across the L1-L4 controller levels.
Vector PDF

Figure 7. From L0 Fixed Support to L2 Execution Control.
Vector PDF

L0: Fixed Support

L0 Fixed Support marks the inclusion boundary rather than a peer agent level. The organizing axis is support function: generation or retrieval, data or training, evaluation, and the boundary test for fixed pipelines.

Generation and Retrieval Components

Foundational generators, editors, and fixed retrievers are cataloged under Supporting Components: the L0 Boundary.

Data Construction and Training Infrastructure

Evaluators and Benchmarks

Standalone evaluators, reward models, and benchmarks are cataloged under Evaluation, Benchmarks, and Reward Models.

Boundary Test for L0 Fixed Support

Back to top

L1: Conditioning Control

L1 controllers construct the generator-facing specification before execution. The categories follow the primary controlled specification: textual prompts, spatial and geometric structure, retrieved evidence, temporal and camera controls, or structured content.

Textual Prompt Specifications

Spatial and Geometric Specifications

Retrieved Evidence Specifications

Temporal and Camera Specifications

Structured Content Specifications

Back to top

L2: Execution Control

L2 controllers select and invoke actual visual operations before observing their outcomes. The categories follow the primary executable object: model and tool operations, image and structured-graphic operations, video and audiovisual operations, document and interface operations, or 3D, CAD, and world operations.

Model and Tool Operations

Image and Structured-Graphic Operations

Video and Audiovisual Operations

Document and Interface Operations

3D, CAD, and World Operations

Back to top

L3: Outcome-Adaptive Control

L3 controllers use an observed result to change a later generation action within the current task. The categories follow the decisive feedback source: perceptual outcomes, structured and execution state, physical constraints, or explicit human review.

Perceptual Outcome Feedback

Structured and Execution Feedback

Physical and Constraint Feedback

Human Review Feedback

Back to top

L4: Experience-Adaptive Control

L4 controllers retain completed-task experience that changes decisions on later independent tasks. The categories follow the persistent carrier: capability and tool profiles, episodic and user memory, reusable procedures and skills, executable workflows and harnesses, or policy and model updates.

Capability and Tool Profiles

Episodic and User Memory

Reusable Procedures and Skills

Executable Workflows and Harnesses

Policy and Model Updates

Back to top

Evaluation, Benchmarks, and Reward Models

These resources evaluate outputs, trajectories, controllers, or supporting signals. A stand-alone evaluator is not assigned an agenticity level. When an agent uses its feedback to choose a new generation action, the complete system may qualify as L3 or L4.

Figure 9. Causal evaluation of agentic visual generation.
Vector PDF

Agent and trajectory evaluation

Output benchmarks and evaluators

Slide and user-interface benchmarks

Reward models, verifiers, and preference data

Back to top

Supporting Components: the L0 Boundary

L0 is an inclusion boundary, not an agent category. The following systems are important generators, editors, retrieval modules, or optimization methods, but their fixed execution rules do not give a controller any generation-level decisions by themselves.

Back to top

Community

Welcome to join our community to discuss agentic visual generation:

• WeChat community — open the QR code and scan it with WeChat.
• Discord community — join the discussion on Discord.

Contact

If you have any suggestions or find this repo helpful, feel free to contact us.

Email: yinminghuang1828@gmail.com, francisshuyuan@gmail.com.

If you find our work useful, please consider giving a star ⭐ to this github repository and citing it ❤️:

@article{huang2026agenticvisualgeneration,
  title={Agentic Visual Generation: From Generative Models to Agentic Control},
  author={Huang, Yinming and Tu, Shuyuan and Yan, Xi and Zhan, Jiahao and Yang, Zihan and Xing, Zhen and Zhang, Hui and Zhang, Tiehua and Jiang, Yu-Gang and Wu, Zuxuan},
  journal={arXiv preprint arXiv:2609.06758},
  year={2026}
}