thunxxx/MLLM-Jailbreak-evaluation-MMJ-Bench

Python

29 commits

updated Mar 30, 2025

README

MMJ-Bench

MMJ-Bench is a comprehensive benchmark designed to systematically evaluate existing multi-modal jailbreak attacks and defenses in a unified manner. We present jailbreak attack and defense techniques evaluated in our paper in the following tables.

Multi-model jailbreak attack

Method	Source	Key Properties
FigStep	FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts	Generation-based
MM-SafetyBench	MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models	Generation-based
VisualAdv	Visual Adversarial Examples Jailbreak Aligned Large Language Models	Optimization-based
ImgJP	Jailbreaking Attack against Multimodal Large Language Model	Optimization-based
AttackVLM	On Evaluating Adversarial Robustness of Large Vision-Language Models	Geneation-based
Hades	Images are achilles’ heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models	Generation-based
FC-Attack	FC-Attack: Jailbreaking Large Vision-Language Models via Auto-Generated Flowcharts	Generation-based
BEST-OF-N	BEST-OF-N JAILBREAKING	Generation-based
Arondight	Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts	Generation-based
ImgTrojan	ImgTrojan: Jailbreaking Vision-Language Models with ONE Image	Optimization-based
VisualGCG	White-box Multimodal Jailbreaks Against Large Vision-Language Models	Optimization-based
IDEATOR	IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves	Generation-based
Align is not Enough	Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models	Optimization-based
JPPN	Utilizing Jailbreak Probability to Attack and Safeguard Multimodal LLMs	Optimization-based
SI-Attack	Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency	Generation-based
MIRAGE	MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks	Generation-based
CS-DJ	Distraction is All You Need for Multimodal Large Language Model Jailbreaking	Optimization-based

Our evaluation results of MLLMs jailbreak attacks across six models are as follows:

ASR

Method	Source	Key Properties
VLGuard	Safety fine-tuning at (almost) no cost: A baseline for vision large language models	Proactive defense
Adashield	Adashield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting	Reactice defense
JailGuard	JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks	Reactive defense
CIDER	Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models	Reactive defense
BlueSuffix	BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks	Reactive defense
ProEAT	Adversarial Training for Multimodal Large Language Models against Jailbreak Attacks	Proactive defense
ShieldLearner	ShieldLearner: A New Paradigm for Jailbreak Attack Defense in LLMs	Reactive defense
HiddenDetect	HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States	Reactive defense
Tit-for-Tat	Tit-for-Tat: Safeguarding Large Vision-Language Models Against Jailbreak Attacks via Adversarial Defense	Reactive defense
TAIJI	TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models	Reactive defense

Usage

Installation

conda create -n MMJ-Bench python=3.10
conda create MMJ-Bench
pip install -r requirements.txt
python -m spacy download en_core_web_sm

Step 1 - Generate Test Cases

In the first step, jailbreak attack techniques are used to generate test cases with generate_test_cases.py.

./scripts/generate_test_cases.sh $method_name $behaviors_path $save_dir

Step 2 - Generate Completions

After generating test cases ， we can generate completions for a target model with or without defense techniques.

Without defense methods:

./scripts/generate_completions.sh $model_name $behaviors_path $test_cases_path $save_path $max_new_tokens $incremental_update

With defense methods.

./scripts/generate_completions_defense.sh $attack_type $target_model $defense_type

Step 3 - Evaluate Completions

After generate completions from a target_model from Step 2, We will utilize the classifier provided by HarmBench to label whether each completion is an example of its corresponding behavior.

./scripts/evaluate_completions.sh $cls_path $behaviors_path $completions_path $save_path

Acknowledgement and citation

We thank the following open-source reposities.

[1]  https://github.com/centerforaisafety/HarmBench
[2]  https://github.com/thuccslab/figstep
[3]  https://github.com/isXinLiu/MM-SafetyBench
[4]  https://github.com/Unispac/Visual-Adversarial-Examples-Jailbreak-Large-Language-Models
[5]  https://github.com/abc03570128/Jailbreaking-Attack-against-Multimodal-Large-Language-Model
[6]  https://github.com/yunqing-me/AttackVLM
[7]  https://github.com/AoiDragon/HADES
[8]  https://github.com/shiningrain/JailGuard
[9]  https://github.com/SaFoLab-WISC/AdaShield

Similar repos

liuxuannan/Awesome-Multimodal-Jailbreak

A Survey on Jailbreak Attacks and Defenses against Multimodal Generative Models

335

jiaxiaojunQAQ/OmniSafeBench-MM

A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack–Defense Evaluation

EddyLuo1232/JailBreakV_28K

[COLM 2024] JailBreakV-28K: A comprehensive benchmark designed to evaluate the transferability of…

SaFo-Lab/JailBreakV_28K

[COLM 2024] JailBreakV-28K: A comprehensive benchmark designed to evaluate the transferability of…

JailbreakV-28K/JailBreakV-28k

Dataset

⛓‍💥 JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models…

isXinLiu/Awesome-MLLM-Safety

Accepted by IJCAI-24 Survey Track

235

EddyLuo/JailBreakV_28K

Dataset

⛓‍💥 JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models…

Jinxiaolong1129/JailbreakDiffusionBench

thunxxx/MLLM-Jailbreak-evaluation-MMJ-Bench

Python

29 commits

updated Mar 30, 2025

See the code

README

MMJ-Bench

Multi-model jailbreak attack

Method	Source	Key Properties
FigStep	FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts	Generation-based
MM-SafetyBench	MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models	Generation-based
VisualAdv	Visual Adversarial Examples Jailbreak Aligned Large Language Models	Optimization-based
ImgJP	Jailbreaking Attack against Multimodal Large Language Model	Optimization-based
AttackVLM	On Evaluating Adversarial Robustness of Large Vision-Language Models	Geneation-based
Hades	Images are achilles’ heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models	Generation-based
FC-Attack	FC-Attack: Jailbreaking Large Vision-Language Models via Auto-Generated Flowcharts	Generation-based
BEST-OF-N	BEST-OF-N JAILBREAKING	Generation-based
Arondight	Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts	Generation-based
ImgTrojan	ImgTrojan: Jailbreaking Vision-Language Models with ONE Image	Optimization-based
VisualGCG	White-box Multimodal Jailbreaks Against Large Vision-Language Models	Optimization-based
IDEATOR	IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves	Generation-based
Align is not Enough	Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models	Optimization-based
JPPN	Utilizing Jailbreak Probability to Attack and Safeguard Multimodal LLMs	Optimization-based
SI-Attack	Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency	Generation-based
MIRAGE	MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks	Generation-based
CS-DJ	Distraction is All You Need for Multimodal Large Language Model Jailbreaking	Optimization-based

Our evaluation results of MLLMs jailbreak attacks across six models are as follows:

ASR

Method	Source	Key Properties
VLGuard	Safety fine-tuning at (almost) no cost: A baseline for vision large language models	Proactive defense
Adashield	Adashield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting	Reactice defense
JailGuard	JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks	Reactive defense
CIDER	Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Models	Reactive defense
BlueSuffix	BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks	Reactive defense
ProEAT	Adversarial Training for Multimodal Large Language Models against Jailbreak Attacks	Proactive defense
ShieldLearner	ShieldLearner: A New Paradigm for Jailbreak Attack Defense in LLMs	Reactive defense
HiddenDetect	HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden States	Reactive defense
Tit-for-Tat	Tit-for-Tat: Safeguarding Large Vision-Language Models Against Jailbreak Attacks via Adversarial Defense	Reactive defense
TAIJI	TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models	Reactive defense

Usage

Installation

conda create -n MMJ-Bench python=3.10
conda create MMJ-Bench
pip install -r requirements.txt
python -m spacy download en_core_web_sm

Step 1 - Generate Test Cases

In the first step, jailbreak attack techniques are used to generate test cases with generate_test_cases.py.

./scripts/generate_test_cases.sh $method_name $behaviors_path $save_dir

Step 2 - Generate Completions

After generating test cases ， we can generate completions for a target model with or without defense techniques.

Without defense methods:

./scripts/generate_completions.sh $model_name $behaviors_path $test_cases_path $save_path $max_new_tokens $incremental_update

With defense methods.

./scripts/generate_completions_defense.sh $attack_type $target_model $defense_type

Step 3 - Evaluate Completions

After generate completions from a target_model from Step 2, We will utilize the classifier provided by HarmBench to label whether each completion is an example of its corresponding behavior.

./scripts/evaluate_completions.sh $cls_path $behaviors_path $completions_path $save_path

Acknowledgement and citation

We thank the following open-source reposities.

[1]  https://github.com/centerforaisafety/HarmBench
[2]  https://github.com/thuccslab/figstep
[3]  https://github.com/isXinLiu/MM-SafetyBench
[4]  https://github.com/Unispac/Visual-Adversarial-Examples-Jailbreak-Large-Language-Models
[5]  https://github.com/abc03570128/Jailbreaking-Attack-against-Multimodal-Large-Language-Model
[6]  https://github.com/yunqing-me/AttackVLM
[7]  https://github.com/AoiDragon/HADES
[8]  https://github.com/shiningrain/JailGuard
[9]  https://github.com/SaFoLab-WISC/AdaShield

Similar repos

liuxuannan/Awesome-Multimodal-Jailbreak

A Survey on Jailbreak Attacks and Defenses against Multimodal Generative Models

335

jiaxiaojunQAQ/OmniSafeBench-MM

A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack–Defense Evaluation

EddyLuo1232/JailBreakV_28K

[COLM 2024] JailBreakV-28K: A comprehensive benchmark designed to evaluate the transferability of…

SaFo-Lab/JailBreakV_28K

[COLM 2024] JailBreakV-28K: A comprehensive benchmark designed to evaluate the transferability of…

JailbreakV-28K/JailBreakV-28k

Dataset

⛓‍💥 JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models…

isXinLiu/Awesome-MLLM-Safety

Accepted by IJCAI-24 Survey Track

235

EddyLuo/JailBreakV_28K

Dataset

⛓‍💥 JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models…

Jinxiaolong1129/JailbreakDiffusionBench

thunxxx/MLLM-Jailbreak-evaluation-MMJ-Bench

README

MMJ-Bench

Multi-model jailbreak attack

Multi-modal jailbreak defense

Usage

Installation

Step 1 - Generate Test Cases

Step 2 - Generate Completions

Step 3 - Evaluate Completions

Acknowledgement and citation

Similar repos

thunxxx/MLLM-Jailbreak-evaluation-MMJ-Bench

README

MMJ-Bench

Multi-model jailbreak attack

Multi-modal jailbreak defense

Usage

Installation

Step 1 - Generate Test Cases

Step 2 - Generate Completions

Step 3 - Evaluate Completions

Acknowledgement and citation

Similar repos