.vertical-container { display: flex; flex-direction: column; gap: 60px; } .horizontal-container { display: flex; flex-direction: row; justify-content: center; gap: 60px; } .image-container img { max-height: 250px; /* Set the desired height */ margin:0; object-fit: contain; /* Ensures the aspect ratio is maintained */ width: auto; /* Adjust width automatically based on height */ box-sizing: content-box; } .image-container img.big { max-height: 350px; /* Set the desired height */ } .image-container { display: flex; /* Aligns images side by side */ justify-content: space-around; /* Space them evenly */ align-items: center; /* Align them vertically */ gap: .5rem } .container { width: 90%; margin: 0 auto; } .text-center { text-align: center; } .score-amount { margin: 0; margin-top: 10px; } .score-percentage {Score: font-size: 12px; font-weight: semi-bold; }

Rapidata OpenAI 4o Preference

This T2I dataset contains over 200'000 human responses from over ~45,000 individual annotators, collected in less than half a day using the Rapidata Python API, accessible to anyone and ideal for large scale evaluation. Evaluating OpenAI 4o (version from 26.3.2025) across three categories: preference, coherence, and alignment.

Explore our latest model rankings on our website.

If you get value from this dataset and would like to see more in the future, please consider liking it ❤️

Overview

The evaluation consists of 1v1 comparisons between OpenAI 4o (version from 26.3.2025) and 12 other models: Ideogram V2, Recraft V2, Lumina-15-2-25, Frames-23-1-25, Imagen-3, Flux-1.1-pro, Flux-1-pro, DALL-E 3, Midjourney-5.2, Stable Diffusion 3, Aurora, and Janus-7b.

Below, you'll find key visualizations that highlight how these models compare in terms of prompt alignment and coherence, where OpenAI 4o (version from 26.3.2025) significantly outperforms the other models.

Master of Absurd Prompts

The benchmark intentially includes a range of absurd or conflicting prompts that aim to target situations or scenes that are very unlikely to occur in the training data such as 'A Chair on a cat' or 'Car is bigger than the airplane.'. Most other models struggle to adhere to these prompts consistently, but the 4o image generation model appears to be significantly ahead of the competition in this regard.

That being said, some of the 'absurd' prompts are still not fully solved.

Alignment

The alignment score quantifies how well an video matches its prompt. Users were asked: "Which image matches the description better?".

Coherence

The coherence score measures whether the generated video is logically consistent and free from artifacts or visual glitches. Without seeing the original prompt, users were asked: "Which image has more glitches and is more likely to be AI generated?"

Preference

The preference score reflects how visually appealing participants found each image, independent of the prompt. Users were asked: "Which image do you prefer?"

About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit rapidata.ai to learn more about how we're revolutionizing human feedback collection for AI development.