Erasing with Precision: Evaluating Specific Concept Erasure from Text-to-Image Generative Models

Official implementation of the paper, Erasing with Precision: Evaluating Specific Concept Erasure from Text-to-Image Generative Models.

You can evaluate many concept erasure methods for text-to-image generative models (such as Stbale Diffusion and FLUX) by using this implementation.

The hash of our paper is a3fc6d5a3ec5794ace407c3a2229bc92d5cffb01, so you can reproduce

git clone https://github.com/fmp453/erase-eval.git
git checkout a3fc6d5a3ec5794ace407c3a2229bc92d5cffb01

Install (our experiments)

1. base environments

   docker pull pytorch/pytorch:2.3.1-cuda11.8-cudnn8-devel
   

2. other dependencies (in the container)

   apt update && apt upgrade
   apt install git
   apt install wget
   pip install -r requirements.txt
   wget -P train_methods https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
   wget -P train_methods https://huggingface.co/lkeab/hq-sam/resolve/main/sam_hq_vit_h.pth
   

Install (in general)

apt update && apt upgrade
apt install git
apt install wget 
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip install  -r requirements.txt
wget -P train_methods https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
wget -P train_methods https://huggingface.co/lkeab/hq-sam/resolve/main/sam_hq_vit_h.pth

Erasing

python main.py --mode train --method esd --concepts "English springer" --device "0,1"

For Ablating Concepts, make .csv file contains the over 200 prompts including guided concept.

python main.py --mode train --method ac --concepts "English springer" --device "0,1" --ac_prompt_path dog.csv

For more arguments can be shown in utils.py.

Inference

python main.py --mode infer --method esd --prompt "a photo of English springer" --erased_model $MODEL_DIR

Evaluation

Before the evaluation, you need to get openai api key and huggingface token.

export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
export HF_TOKEN="YOUR_HUGGINGFACE_TOKEN"

python eval.py --method esd --protocol 1 --device 0

Additional Methods

This repository provides the concept erasure methods listed in Acknowledgement. If you want to try other methods,

1. create train_METHODNAME.py in train_methods directory using diffusers.
2. create infer_METHODNAME.py in infer_methods directory using diffusers.
3. add name and other hyperparameters of your methods in Arguments.method of utils.py.

Citation

Our paper can be cited as follows.

@misc{fuchi2025erasingprecisionevaluatingspecific,
      title={Erasing with Precision: Evaluating Specific Concept Erasure from Text-to-Image Generative Models}, 
      author={Masane Fuchi and Tomohiro Takagi},
      year={2025},
      eprint={2502.13989},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2502.13989}, 
}

Acknowledgement

We reimplemented methods listed below using 🤗 Diffusers

• Erased Stable Diffusion (ESD)
• Ablating Concepts (AC)
• Unified Concept Editing (UCE)
• Semi-Permeable Membrane (SPM)
• Safe self-Distillation Diffusion (SDD)
• Mass Concept Erasure (MACE)
• Reliable Concept Erasing via Lightweight Erasers (Receler)
• Erasing-Adversarial-Preservation (EAP)
• LocoEdit
• SalUn
• Forget-Me-Not (FMN)
• DoCo
• Adaptive Guided Erasure (AGE)
• Adaptive Value Decomposer (AdaVD)
• Gated Low-rank adaptation for Concept Erasure (GLoCE)
• Anti-Editing Concept Erasure in Text-to-Image Models (ACE)
• Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gate (CPE)
• A Two-Stage Framework for Adversarially Robust Concept Erasing from Text-to-Image Diffusion Models (STEREO)
• Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Model (R.A.C.E)
• Automatically guides deNoising Trajectories (ANT)
• EraseFlow
• Minimalist Concept Erasure in Generative Models (MCE)
• Localized Concept Erasure in Text-to-Image Diffusion Models via High-Level Representation Misdirection (HiRM)
• Scalable, Precise, and Efficient Concept Erasure for Diffusion Models (SPEED)
• Co-occurring Associated REtained concepts in Diffusion Unlearning (CARE)

File Stracture is followed by TabSyn.

MSCOCO provided by https://huggingface.co/datasets/shunk031/MSCOCO

CMMD PyTorch Source Code: https://github.com/sayakpaul/cmmd-pytorch