Dataset Card for PosterRewardBench

Table of Contents

• Dataset Description
  • Dataset Summary
  • Supported Tasks and Leaderboards
  • Languages
• Dataset Structure
  • Data Instances
  • Data Fields
  • Data Splits
• Dataset Creation
  • Curation Rationale
  • Source Data
  • Annotations
  • Personal and Sensitive Information
• Considerations for Using the Data
  • Social Impact of Dataset
  • Discussion of Biases
  • Other Known Limitations
• Additional Information
  • Dataset Curators
  • Licensing Information
  • Citation Information
  • Contributions

Dataset Description

• Homepage: https://alexlai2860.github.io/PosterReward/
• Repository: https://github.com/creative-graphic-design/huggingface-datasets/tree/main/datasets/PosterRewardBench
• Hugging Face Dataset: https://huggingface.co/datasets/creative-graphic-design/PosterRewardBench
• Original Code and Data: https://github.com/MeiGen-AI/PosterReward
• Paper (CVPR 2026 / arXiv): https://arxiv.org/abs/2603.29855
• Source Model Repository: https://huggingface.co/MeiGen-AI/PosterReward_v1
• Leaderboard: Not available in the original release.
• Point of Contact: https://github.com/MeiGen-AI/PosterReward/issues

Dataset Summary

PosterRewardBench is the preference benchmark released with PosterReward: Unlocking Accurate Evaluation for High-Quality Graphic Design Generation. It is designed to evaluate reward models for poster assessment. Each row contains a poster prompt, one preferred generated poster image, and one rejected generated poster image.

This Hugging Face loader exposes the public benchmark as two configurations:

• basic: PosterRewardBench-Basic, generated by Flux, Flux-Krea, and SD3.5-L, with larger quality variation.
• advanced: PosterRewardBench-Advanced, generated by Seedream-3.0, Seedream-4.0, and Qwen-Image-Lightning, with higher overall quality and smaller quality gaps.

The original paper evaluates poster quality across five dimensions: foundational visual quality, AI artifacts, textual accuracy, prompt fidelity, and aesthetic value.

Supported Tasks and Leaderboards

This dataset is intended for reward-model evaluation and preference modeling over poster image pairs. A model can score both images conditioned on the same prompt, then predict the higher-quality poster by selecting the image with the higher score. Accuracy over pairwise preferences is the primary usage pattern in the original PosterRewardBench evaluation.

No active public leaderboard is bundled with this Hugging Face dataset. For exact reproduction of the paper's model evaluation pipeline, use the upstream PosterReward repository.

Languages

Prompts are primarily in English (en). The Advanced subset also includes prompts containing Chinese text (zh) for posters with rendered Chinese copy.

Dataset Structure

Data Instances

Each row corresponds to one preference pair from the upstream PRB_basic_relative.json or PRB_advanced_relative.json file.

{
  "id": "basic-00000",
  "prompt": "This poster features a close-up of a soldier...",
  "chosen_image": "<image>",
  "rejected_image": "<image>",
  "chosen_image_path": ".../PRB_basic_images/prompt_088_img02_stable-diffusion-3.5-large_chosen.png",
  "rejected_image_path": ".../PRB_basic_images/prompt_088_img05_stable-diffusion-3.5-large_reject.png",
  "messages": [
    {
      "role": "user",
      "content": "This poster features a close-up of a soldier..."
    },
    {
      "role": "assistant",
      "content": ""
    }
  ],
  "rejected_messages": [
    {
      "role": "user",
      "content": "This poster features a close-up of a soldier..."
    },
    {
      "role": "assistant",
      "content": ""
    }
  ]
}

Data Fields

• id (string): Stable row identifier generated by this loader as {config}-{index:05d}.
• prompt (string): Poster generation prompt from messages[0].content.
• chosen_image (Image): Preferred generated poster image.
• rejected_image (Image): Rejected generated poster image.
• chosen_image_path (string): Local resolved path for the preferred image.
• rejected_image_path (string): Local resolved path for the rejected image.
• messages (list): Original chosen-side chat-style message payload with role and content.
• rejected_messages (list): Original rejected-side chat-style message payload with role and content.

Data Splits

Both configurations expose a single test split because PosterRewardBench is an evaluation benchmark and the upstream release does not provide training data partitions.

Dataset Creation

Curation Rationale

PosterRewardBench was created to evaluate reward models on poster generation quality, where general-purpose reward models may miss typography, layout, prompt fidelity, and poster-specific aesthetic criteria.

Source Data

The upstream PosterReward repository provides the JSON metadata files. The benchmark images are hosted in the MeiGen-AI/PosterReward_v1 Hugging Face model repository as PRB_basic_images.tar.gz and PRB_advanced_images.tar.gz.

Annotations

Each row contains a chosen image and a rejected image for the same poster prompt. The original paper describes PosterReward data construction as using multi-MLLM consensus for poster preference data.

Personal and Sensitive Information

The dataset consists of machine-generated poster images and text prompts. The dataset card does not identify personal information in the released benchmark. Some prompts or generated posters may depict people, fictional characters, brands, events, or culturally specific text because poster generation tasks often include such visual concepts.

Considerations for Using the Data

Social Impact of Dataset

PosterRewardBench can support better evaluation of poster generation systems, especially for text rendering, layout, and design quality. It should be used as an evaluation benchmark rather than as a complete proxy for human design judgment.

Discussion of Biases

The benchmark reflects the prompt distribution, source generation models, and preference construction process used by the original authors. Models evaluated on this dataset may overfit to the visual styles and failure modes represented by the Basic and Advanced subsets.

Other Known Limitations

The benchmark contains pairwise preferences but does not expose separate scalar ratings for each of the five PosterReward evaluation dimensions. It also does not provide official train, validation, and test splits in the released JSON files.

Additional Information

Dataset Curators

The dataset was created by the PosterReward authors. This Hugging Face dataset loader was added in the creative-graphic-design/huggingface-datasets repository.

Licensing Information

The original model card is marked Apache-2.0, but a dataset-specific license for PosterRewardBench contents was not found in the source repository at the time this loader was created. This dataset card therefore marks the dataset content license as unknown.

Citation Information

@misc{lai2026posterreward,
  title={PosterReward: Unlocking Accurate Evaluation for High-Quality Graphic Design Generation},
  author={Lai, Jianyu and Chen, Sixiang and Gao, Jialin and Shi, Hengyu and Liu, Zhongying and Zhai, Fuxiang and Luo, Junfeng and Wei, Xiaoming and Wang, Lujia and Zhu, Lei},
  year={2026},
  eprint={2603.29855},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2603.29855}
}

Contributions

Thanks to MeiGen-AI and the PosterReward authors for creating and releasing this benchmark.