README
EditReward Curated 100
100 high-quality, richly annotated image editing preference examples curated from TIGER-Lab/EditReward-Data.
Each example contains a source image, two edited outputs (left/right) from different models, human preference labels, and detailed multi-dimensional annotations.
Quick Start
from datasets import load_dataset
ds = load_dataset("AgPerry/EditReward-Data-100", split="train")
# Access an example
example = ds[0]
print(f"Instruction: {example['instruction']}")
print(f"Human preference: {example['vote_type']}")
print(f"Category: {example['category']}")
# View images
example['source_image'].show() # Original image
example['left_image'].show() # Edit A
example['right_image'].show() # Edit B
Dataset Summary
| Property | Value |
|---|---|
| Total examples | 100 |
| Quality tiers | Gold 75, Silver 25 |
| Vote distribution | Left 48, Tie 3, Right 49 |
| Edit categories | 11 types |
| Unique instructions | 46 |
| Annotation dimensions | 10 per example |
| Cross-model verified | Yes (4 independent models, ~2,500 API calls) |
| Source | TIGER-Lab/EditReward-Data (170K+ pairs) |
Data Fields
Images (3 per example)
source_image— Original source imageleft_image— Edit A (left output)right_image— Edit B (right output)
Metadata & Annotations
| Field | Type | Description |
|---|---|---|
dev_id | string | Unique identifier |
instruction | string | Editing instruction given to both models |
category | string | Edit category (11 types) |
left_model / right_model | string | Models that produced each edit |
left_overall_score / right_overall_score | float | Human overall quality scores (1-8 scale) |
left_instruction_following_score / right_instruction_following_score | float | Instruction following score (1-4 scale) |
left_quality_aesthetic_score / right_quality_aesthetic_score | float | Visual quality & aesthetics score (1-4 scale) |
vote_type | string | Human preference: leftvote / rightvote / tie |
reasoning_chain_A / reasoning_chain_B | string | Step-by-step quality analysis per edit |
failure_modes_A / failure_modes_B | list[string] | Identified failure modes (11-type taxonomy) |
difficulty | string | easy / medium / hard |
preference_justification | string | Reasoning behind human preference |
useful_tools | list[string] | Recommended analysis tools for this edit type |
key_evaluation_insight | string | Decisive quality difference |
quality_tier | string | gold / silver |
Category Distribution
| Category | Count |
|---|---|
| other | 21 |
| color_texture | 20 |
| object_replacement | 15 |
| object_removal | 8 |
| object_addition | 7 |
| attribute_modification | 7 |
| text_editing | 5 |
| style_transfer | 5 |
| weather_lighting_season | 4 |
| action_pose | 4 |
| background | 4 |
License
MIT License. Source data from TIGER-Lab/EditReward-Data.
Contributors
AgPerry
10 commits
Similar repos
README
EditReward Curated 100
100 high-quality, richly annotated image editing preference examples curated from TIGER-Lab/EditReward-Data.
Each example contains a source image, two edited outputs (left/right) from different models, human preference labels, and detailed multi-dimensional annotations.
Quick Start
from datasets import load_dataset
ds = load_dataset("AgPerry/EditReward-Data-100", split="train")
# Access an example
example = ds[0]
print(f"Instruction: {example['instruction']}")
print(f"Human preference: {example['vote_type']}")
print(f"Category: {example['category']}")
# View images
example['source_image'].show() # Original image
example['left_image'].show() # Edit A
example['right_image'].show() # Edit B
Dataset Summary
| Property | Value |
|---|---|
| Total examples | 100 |
| Quality tiers | Gold 75, Silver 25 |
| Vote distribution | Left 48, Tie 3, Right 49 |
| Edit categories | 11 types |
| Unique instructions | 46 |
| Annotation dimensions | 10 per example |
| Cross-model verified | Yes (4 independent models, ~2,500 API calls) |
| Source | TIGER-Lab/EditReward-Data (170K+ pairs) |
Data Fields
Images (3 per example)
source_image— Original source imageleft_image— Edit A (left output)right_image— Edit B (right output)
Metadata & Annotations
| Field | Type | Description |
|---|---|---|
dev_id | string | Unique identifier |
instruction | string | Editing instruction given to both models |
category | string | Edit category (11 types) |
left_model / right_model | string | Models that produced each edit |
left_overall_score / right_overall_score | float | Human overall quality scores (1-8 scale) |
left_instruction_following_score / right_instruction_following_score | float | Instruction following score (1-4 scale) |
left_quality_aesthetic_score / right_quality_aesthetic_score | float | Visual quality & aesthetics score (1-4 scale) |
vote_type | string | Human preference: leftvote / rightvote / tie |
reasoning_chain_A / reasoning_chain_B | string | Step-by-step quality analysis per edit |
failure_modes_A / failure_modes_B | list[string] | Identified failure modes (11-type taxonomy) |
difficulty | string | easy / medium / hard |
preference_justification | string | Reasoning behind human preference |
useful_tools | list[string] | Recommended analysis tools for this edit type |
key_evaluation_insight | string | Decisive quality difference |
quality_tier | string | gold / silver |
Category Distribution
| Category | Count |
|---|---|
| other | 21 |
| color_texture | 20 |
| object_replacement | 15 |
| object_removal | 8 |
| object_addition | 7 |
| attribute_modification | 7 |
| text_editing | 5 |
| style_transfer | 5 |
| weather_lighting_season | 4 |
| action_pose | 4 |
| background | 4 |
License
MIT License. Source data from TIGER-Lab/EditReward-Data.
Similar repos
Linked in READMEs
Contributors
AgPerry
10 commits