KnowGen Benchmark

Project Page | Paper | Code

This repository contains the KnowGen benchmark data for Gen-Searcher: Reinforcing Agentic Search for Image Generation.

👀 Intro

We introduce Gen-Searcher, as the first attempt to train a multimodal deep research agent for image generation that requires complex real-world knowledge. Gen-Searcher can search the web, browse evidence, reason over multiple sources, and search visual references before generation, enabling more accurate and up-to-date image synthesis in real-world scenarios.

We build two dedicated training datasets Gen-Searcher-SFT-10k, Gen-Searcher-RL-6k and one new benchmark KnowGen for search-grounded image generation.

Gen-Searcher achieves significant improvements, delivering 15+ point gains on the KnowGen and WISE benchmarks. It also demonstrates strong transferability to various image generators.

All code, models, data, and benchmark are fully released.

🔍 KnowGen-Bench

Our KnowGen bench covers around 20 diverse categories in real-world scenarios.

🏆 Performance

Our method delivers consistent gains across backbones, improving Qwen-Image by around 16 points on KnowGen. It also shows strong transferability, generalizing to Seedream 4.5 and Nano Banana Pro with no additional training, yielding about 16-point and 3-point improvements, respectively.

📐 KnowGen Bench Evaluation

To evaluate your model on the KnowGen benchmark, you can use the evaluation scripts provided in the GitHub repository:

cd KnowGen_Eval
bash gpt_eval_knowgen.sh

Ensure that your results are organized in the following format for evaluation:

  [
    {
      "id": 3260,
      "success": true,
      "prompt": "xxxxx",
      "meta": { "category": "Biology", "difficulty": "easy" },
      "output_path": "./images/output_3260.png",
      "gt_image": "./gt_image/answer_3260.png"
    }
  ]

For ground truth images, you may download gt_image_part1.zip and unzip it, or directly download the gt_image folder.

Citation

If you find this work or dataset helpful, please consider citing:

@article{feng2026gen,
  title={Gen-Searcher: Reinforcing Agentic Search for Image Generation},
  author={Feng, Kaituo and Zhang, Manyuan and Chen, Shuang and Lin, Yunlong and Fan, Kaixuan and Jiang, Yilei and Li, Hongyu and Zheng, Dian and Wang, Chenyang and Yue, Xiangyu},
  journal={arXiv preprint arXiv:2603.28767},
  year={2026}
}