Image Editing As Programs with Diffusion Models

Image Editing As Programs with Diffusion Models
Yujia Hu, Songhua Liu, ZhenXiong Tan, Xingyi Yang, and Xinchao Wang
xML Lab, National University of Singapore

We propose IEAP! IEAP is an image editing framework that decomposes complex instructions into composable atomic operations, enabling high-precision handling of both structurally consistent and inconsistent edits, as well as compositional multi-step instructions.

Pipeline:

🔧 Installation

1. First install python and pytorch:

conda create -n ieap python=3.10
conda activate ieap
conda install pytorch==2.3.1 torchvision==0.18.1 pytorch-cuda=12.1 cuda -c pytorch  -c "nvidia/label/cuda-12.1.0" -c "nvidia/label/cuda-12.1.1"

2. Install other dependencies:

pip install -r requirements.txt

🖼️ Run

Start with:

python main.py image_path editing_instructions

For example:

python main.py "assets/a12.jpg" "Change the action of the woman to running and minify the woman."

If you have already had a json file for editing:

python main_json.py image_path json_path

For example:

python main_json.py "assets/a12.jpg" "instructions.json"

🔦 ToDo List

• More functions and demos.

🔗 Related Projects

We sincerely thank the excellent open-source projects: OminiControl and Sa2VA.

Citation

@article{hu2025ieap,
  title={Image Editing As Programs with Diffusion Models},
  author={Hu, Yujia and Liu, Songhua and Tan, Zhenxiong and Yang, Xingyi and Wang, Xinchao},
  journal={NeurIPS},
  year={2025}
}