Unison is a comprehensive benchmark comprising 2,169 high-quality unified task samples, designed to evaluate joint understanding and generation in unified multimodal models.

Code: github.com/FudanCVL/Unison — if you use this benchmark or find it helpful, please give it a star!

Tasks

Layout

├── Internal_Consistency/          # IC
│   ├── prompts.txt                # 556 generation prompts
│   ├── questions.json             # yes/no VQA keyed by prompt_<N>
│   └── images/                    # 556 reference images
├── Und_Guided_Gen/                # UGG
│   ├── UGG.json                   # 543 grounded generation items
│   └── images/                    # 539 source images
├── Gen_Guided_Und/                # GGU (three spatial sub-categories)
│   ├── 2D_Spatial/                # 182 2D spatial items
│   ├── 3D_Spatial/                # 180 3D spatial items
│   └── Complex_Relation/          # 180 relation items
└── Mutual_Enhancement/            # ME
    ├── ME.json                    # 264 rows, 528 samples
    └── images/                    # 264 source images

Field notes

• IC — questions.json is keyed prompt_<N> matching the 0-indexed line N in prompts.txt; each value maps question numbers to yes/no questions about the generated image.
• UGG — image_path is relative to Und_Guided_Gen/ (e.g. images/1.jpg); bbox/mask give the grounded edit region, operation the edit type.
• GGU — image_path is relative to each sub-category dir: 2D_Spatial/matrices/..., 3D_Spatial/cubes/.... Complex_Relation items carry no source image and are reconstructed from their description / validation questions.
• ME — image_path is a bare filename resolving under Mutual_Enhancement/images/.