We acknowledge the data contribution and support from

📖 Introduction

This is the official implementation for the paper: EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing. In this paper, we introduce EditReward, a human-aligned reward model powered by a high-quality dataset for instruction-guided image editing. We first construct EditReward-Data, a large-scale, high-fidelity preference dataset for instruction-guided image editing. It comprises over 200K manually annotated preference pairs, covering a diverse range of edits produced by seven state-of-the-art models across twelve distinct sources. Every preference annotation in EditReward-Data was curated by trained annotators following a rigorous and standardized protocol, ensuring high alignment with considered human judgment and minimizing label noise. Using this dataset, we train the reward model EditReward to score instruction-guided image edits. To rigorously assess EditReward and future models, we also introduce EditReward-Bench a new benchmark built upon our high-quality annotations, which includes more difficult multi-way preference prediction.

📰 News

• [2026-02-06] 🔥 We started maintaining a list of Awesome Works using EditReward!
• [2026-01-27] 🔥 Add training & inference support for Qwen3-VL Series!
• [2026-01-26] 🔥 Our paper has been accepted by ICLR 2026!
• [2025-10-29] 🔥 Release the training guideline of EditReward, see Training Insctruction!
• [2025-10-14] 🔥 Release the evaluation code and guideline of EditReward-Bench, see Evaluate Insctruction!
• [2025-10-10] 🔥 Release our evaluation benchmark EditReward-Bench, Welcome to use!
• [2025-10-08] 🔥 Release our training dataset EditReward-Data, Welcome to use!
• [2025-10-03] 🔥 Release inference code and pretrained model.
• [2025-10-01] 🎉 We initialize the official repo of EditReward.

🚧 TODO List

• Release inference code and pretrained model
• Release evaluation benchmark
• Release training code
• Release training dataset

📄 Table of Contents

• 🛠️ Installation
• 👨‍🏫 Get Started
• 🏋️ Training
• 📊 Benchmark
• 🖊️ Citation
• 🤝 Acknowledgement
• ✨ Awesome Works using EditReward
• 🎫 License

---

🚀 Quick Start

EditReward is a VLM-based reward model trained on EditReward-Data that demonstrates superior alignment with human preferences.

💻 Installation

# Install locally for development or training.
git clone https://github.com/TIGER-AI-Lab/EditReward.git
cd EditReward

conda create -n edit_reward python=3.10 -y
conda activate edit_reward

# Install PyTorch first (CUDA 12.4 example).
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124

# Install the remaining runtime dependencies.
pip install -U datasets pillow openai megfile sentencepiece deepspeed fire omegaconf \
  matplotlib peft trl==0.8.6 tensorboard scipy transformers==4.57.0 accelerate \
  requests packaging pandas opencv-python einops timm huggingface_hub qwen-vl-utils \
  av regex safetensors tqdm

# Optional but recommended.
pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.2.post1/flash_attn-2.7.2.post1+cu12torch2.5cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

# Install EditReward as an editable package.
pip install -e .

🚀 Usage

Basic Command

import torch
from EditReward import EditRewardInferencer
from EditReward.inference_vl_edit import EditRewardVLInferencer

# ------------------------------------------------------------------------------
# Example script for evaluating edited images with EditReward
# ------------------------------------------------------------------------------

# Package-relative config paths work after `pip install -e .`.
# You can also pass an absolute path to a local YAML config.
CHECKPOINT_PATH = "your/local/path/to/checkpoint"
CONFIG_PATH = "config/EditReward-MiMo-VL-7B-SFT-2508.yaml"

# Initialize reward model
inferencer = EditRewardInferencer(
    config_path=CONFIG_PATH,
    checkpoint_path=CHECKPOINT_PATH,
    device="cuda",        # or "cpu"
    reward_dim="overall_detail",    # choose reward dimension if applicable
    rm_head_type="ranknet_multi_head"
)

# (Optional) Unified inferencer for Qwen2.5-VL / Qwen3-VL:
# Just switch CONFIG_PATH to either:
# - "config/EditReward-Qwen2.5-7B-VL.yaml"
# - "config/EditReward-Qwen3-VL.yaml"
# inferencer = EditRewardVLInferencer(
#     config_path=CONFIG_PATH,
#     checkpoint_path=CHECKPOINT_PATH,
#     device="cuda",
#     reward_dim="overall_detail",
#     rm_head_type="ranknet_multi_head",
# )

# Example input data -----------------------------------------------------------
# image_src = [
#     "../assets/examples/source_img_1.png",
#     "../assets/examples/source_img_1.png",
# ]

# image_paths = [
#     "../assets/examples/target_img_1.png",
#     "../assets/examples/target_img_2.png",
# ]
image_src = [
    "your/local/path/to/source_image_1.jpg",
    "your/local/path/to/source_image_2.jpg",
]

image_paths = [
    "your/local/path/to/edited_image_1.jpg",
    "your/local/path/to/edited_image_2.jpg",
]

# example instruction: "Add a green bowl on the branch"
# prompts = [
#     "Add a green bowl on the branch",
#     "Add a green bowl on the branch"
# ]
prompts = [
    "your first editing instruction",
    "your second editing instruction"
]

# ------------------------------------------------------------------------------
# Main evaluation modes
# ------------------------------------------------------------------------------
if __name__ == "__main__":
    mode = "pairwise_inference"  # or "single_inference"

    if mode == "pairwise_inference":
        # ----------------------------------------------------------
        # Pairwise comparison: compares two edited images side-by-side
        # ----------------------------------------------------------
        with torch.no_grad():
          rewards = inferencer.reward(
              prompts=prompts,
              image_src=image_src,
              image_paths=image_paths
          )
        scores = [reward[0].item() for reward in rewards]
        print(f"[Pairwise Inference] Image scores: {scores}")

    elif mode == "single_inference":
        # ----------------------------------------------------------
        # Single image scoring: evaluates one edited image at a time
        # ----------------------------------------------------------
        with torch.no_grad():
          rewards = inferencer.reward(
              prompts=[prompts[0]],
              image_src=[image_src[0]],
              image_paths=[image_paths[0]]
          )
        print(f"[Single Inference] Image 1 score: {[reward[0].item() for reward in rewards]}")
        
        with torch.no_grad():
          rewards = inferencer.reward(
              prompts=[prompts[0]],
              image_src=[image_src[0]],
              image_paths=[image_paths[1]]
          )
        print(f"[Single Inference] Image 2 score: {[reward[0].item() for reward in rewards]}")

---

📁 Dataset

EditReward-Data

Download EditReward

huggingface-cli download --repo-type dataset TIGER-Lab/EditReward-Data --local-dir /your-local-dataset-path

🏋️ Training

🤖 Model Support

• Qwen2.5-VL Series
• MiMo-VL Series
• Qwen3-VL Series

🚀 Training Command

To train EditReward model, follow the detail instruction in Training Insctruction

Unified training entry (Qwen2.5-VL / Qwen3-VL)

We provide a unified training entry that automatically selects the correct model/collator based on model_name_or_path:

# Qwen2.5-VL
python EditReward/train_qwen_vl_edit.py --config EditReward/config/EditReward-Qwen2.5-7B-VL.yaml

# Qwen3-VL
python EditReward/train_qwen_vl_edit.py --config EditReward/config/EditReward-Qwen3-VL.yaml

---

📊 Benchmark

To evaluate EditReward preference accuracy, follow the detail instruction in Evaluate Insctruction

Experimental Results: Alignment with Humans

Method	GenAI-Bench	AURORA-Bench	ImagenHub	EditReward-Bench (Overall)
Random	25.90	33.43	--	13.84
Human-to-Human	--	--	41.84	--
Proprietary Models
GPT-4o	53.54	50.81	38.21	28.31
GPT-5	59.61	47.27	40.85	37.81
Gemini-2.0-Flash	53.32	44.31	23.69	33.47
Gemini-2.5-Flash	57.01	47.63	41.62	38.02
Open-Source VLMs
Qwen2.5-VL-3B-Inst	42.76	30.69	-2.54	26.86
Qwen2.5-VL-7B-Inst	40.48	38.62	18.59	29.75
Qwen2.5-VL-32B-Inst	39.28	37.06	26.87	28.72
MiMo-VL-7B-SFT-2508	57.89	30.43	22.14	31.19
ADIEE	59.96	55.56	34.50	--
Reward Models (Ours)
EditReward (on Qwen2.5-VL-7B)	63.97	59.50	36.18	36.78
EditReward (on MiMo-VL-7B)	65.72	63.62	35.20	38.42

---

EditReward-Bench Results

Method	EditReward-Bench (K=2)	EditReward-Bench (K=3)	EditReward-Bench (K=4)	EditReward-Bench (Overall)
Random	25.81	11.33	1.35	13.84
Human-to-Human	--	--	--	--
Proprietary Models
GPT-4o	45.69	27.33	7.31	28.31
GPT-5	57.53	38.51	12.84	37.81
Gemini-2.0-Flash	52.43	33.33	13.51	33.47
Gemini-2.5-Flash	58.61	39.86	12.16	38.02
Open-Source VLMs
Qwen2.5-VL-3B-Inst	51.07	20.27	2.71	26.86
Qwen2.5-VL-7B-Inst	52.69	24.67	3.38	29.75
Qwen2.5-VL-32B-Inst	50.54	25.27	4.05	28.72
MiMo-VL-7B-SFT-2508	49.46	30.41	9.46	31.19
ADIEE	--	--	--	--
Reward Models (Ours)
EditReward (on Qwen2.5-VL-7B)	56.99	36.00	10.81	36.78
EditReward (on MiMo-VL-7B)	56.45	42.67	11.49	38.42

---

📚 Citation

Please kindly cite our paper if you use our code, data, models or results:

@article{wu2025editreward,
  title={EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing},
  author={Wu, Keming and Jiang, Sicong and Ku, Max and Nie, Ping and Liu, Minghao and Chen, Wenhu},
  journal={arXiv preprint arXiv:2509.26346},
  year={2025}
}

---

🙏 Acknowledgements

We would like to thank the HPSv3, VideoAlign and GenAI-Bench codebase for providing valuable references.

---

✨ Awesome Works using EditReward

😊 TIGER-AI-Lab, RewardHarness: Self-Evolving Agentic Post-Training.

😊 Reve, CUHK, PromptRL: Prompt Matters in RL for Flow-Based Image Generation.

😊 Adobe, HKU, Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing.

😊 Meta, Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image.

😊 Google DeepMind, CUHK, Image Diffusion Preview with Consistency Solver.

---

⭐ Star History 🔝

💬 Support

For questions and support:

• Issues: GitHub Issues
• Email: wukeming0608@gmail.com & wenhuchen@uwaterloo.ca