README
Unison-Judge is a fine-tuned Qwen3-VL-8B vision-language model that serves as the local automatic judge for the Unison benchmark. It scores UMMs' outputs across all four unified tasks (IC, UGG, GGU and ME) without requiring a hosted API.
Code: github.com/FudanCVL/Unison — if you use this model or find it helpful, please give it a star!
Judge Consistency Data
The Judge_Consistency/ directory contains 231 evaluation cases used to assess the scoring consistency of Unison-Judge across all four tasks.
| Field | Description |
|---|---|
id | Item identifier |
task | One of IC, UGG, GGU, ME |
family | Question type |
model | The UMM whose output is being evaluated |
questions | List of sub-questions, each with the model's answer and judge-assigned score |
images | Reference image(s) and the model-generated image |
Task distribution: IC (57), ME (62), GGU (56), UGG (56)
Models covered: BAGEL-7B-MoT, OmniGen2, SEED-X-17B, UniWorld-V1
Contributors
16 commits
Similar repos
README
Unison-Judge is a fine-tuned Qwen3-VL-8B vision-language model that serves as the local automatic judge for the Unison benchmark. It scores UMMs' outputs across all four unified tasks (IC, UGG, GGU and ME) without requiring a hosted API.
Code: github.com/FudanCVL/Unison — if you use this model or find it helpful, please give it a star!
Judge Consistency Data
The Judge_Consistency/ directory contains 231 evaluation cases used to assess the scoring consistency of Unison-Judge across all four tasks.
| Field | Description |
|---|---|
id | Item identifier |
task | One of IC, UGG, GGU, ME |
family | Question type |
model | The UMM whose output is being evaluated |
questions | List of sub-questions, each with the model's answer and judge-assigned score |
images | Reference image(s) and the model-generated image |
Task distribution: IC (57), ME (62), GGU (56), UGG (56)
Models covered: BAGEL-7B-MoT, OmniGen2, SEED-X-17B, UniWorld-V1
Similar repos
Linked in READMEs
Contributors
16 commits