ICEdit

Image editing using instruction-based diffusion models with efficient LoRA fine-tuning.

Installation

Environment Setup

conda create -n icedit python=3.10
conda activate icedit
pip install -r requirements.txt
pip install -U huggingface_hub

Download Pretrained Weights

• Flux.1-fill-dev
• ICEdit-normal-LoRA

Usage

Basic Inference

python scripts/inference.py --image assets/girl.png \
                            --instruction "Make her hair dark green and her clothes checked." \
                            --seed 304897401

Fisher Data Generation

Generate paired safe/unsafe image datasets for training or evaluation using various diffusion models.

Basic Usage

python scripts/generate_fisher_data_N_flux.py --model flux \
                                              --numbers_per_class 5 \
                                              --device "0,1" \
                                              --output_path ./data/

Supported Models

• flux: FLUX.1-dev (default text-to-image + FLUX.1-Fill-dev for inpainting)
• sdxl: Stable Diffusion XL 1.0 + inpainting model
• sd21: Stable Diffusion 2.1 + inpainting model
• sd15: Stable Diffusion 1.5 + inpainting model
• sd14: Stable Diffusion 1.4 + inpainting model

Parameters

• --model: Model type (flux/sdxl/sd21/sd15/sd14)
• --numbers_per_class: Number of images per prompt (default: 1)
• --device: GPU devices to use, comma-separated (default: "0,1")
• --output_path: Output directory for generated images (default: ./data/)

Output Structure

The script generates three directories:

• safe_data{N}: Original images (nude content)
• unsafe_data{N}: Edited images (clothed versions)
• safe_unsafe_comparison{N}: Side-by-side comparisons

Example Commands

# Generate 10 images per class using FLUX on GPUs 0,1
python scripts/generate_fisher_data_N_flux.py --model flux --numbers_per_class 10 --device "0,1"

License

This project cannot be used for commercial purposes. See LICENSE for details.

Citation

@misc{zhang2025ICEdit,
      title={In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer}, 
      author={Zechuan Zhang and Ji Xie and Yu Lu and Zongxin Yang and Yi Yang},
      year={2025},
      eprint={2504.20690},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2504.20690}, 
}