WhileBug/AwesomeLLMJailBreakPapers

Awesome LLM Jailbreak academic papers

174

0 commits

updated Nov 3, 2023

See the code

README

Awesome LLM Jailbreak Papers

Defense

Automatic Prompt Optimization with "Gradient Descent" and Beam Search

Jailbreaker in Jail: Moving Target Defense for Large Language Models

Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations

Self-Guard: Empower the LLM to Safeguard Itself

Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks

BASELINE DEFENSES FOR ADVERSARIAL ATTACKS AGAINST ALIGNED LANGUAGE MODELS

Benchmark

Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment

Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models

LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI’s ChatGPT Plugins

Prompt Injection

Prompt Injection attack against LLM-integrated Applications

Fuzzing

GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts

FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Model

Role Play

Quack: Automatic Jailbreaking Large Language Models via Role-playing

Jailbreaking Language Models at Scale via Persona Modulation

Role-Play with Large Language Models

Empirical Study

"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models

Tricking LLMs into Disobedience: Understanding, Analyzing, and Preventing Jailbreaks

Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study (NDSS 2024)

Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks

LLM-based (Finetune a LLM to ask)

MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots

Prompt Engineering

Defending ChatGPT against Jailbreak Attack via Self-Reminder

Shield and Spear: Jailbreaking Aligned LLMs with Generative Prompting

Self-Deception: Reverse Penetrating the Semantic Firewall of Large Language Models

Open Sesame! Universal Black Box Jailbreaking of Large Language Models

Jailbreaking Black Box Large Language Models in Twenty Queries

AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models

Visual Adversarial Examples

Misusing Tools in Large Language Models With Visual Adversarial Examples

Visual Adversarial Examples Jailbreak Aligned Large Language Models

Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs

Backdoor

Universal Jailbreak Backdoors from Poisoned Human Feedback

Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models

Cross-lingual

Multilingual Jailbreak Challenges in Large Language Models

Low-Resource Languages Jailbreak GPT-4

Others

Jailbroken: How Does LLM Safety Training Fail?

Multi-step Jailbreaking Privacy Attacks on ChatGPT

Prompt Injection Attacks and Defenses in LLM-Integrated Applications

Why So Toxic?: Measuring and Triggering Toxic Behavior in Open-Domain Chatbots

Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation

Similar repos

niconi19/LLM-Conversation-Safety

[NAACL2024] Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey

112

Libr-AI/OpenRedTeaming

Papers about red teaming LLMs and Multimodal models.

179

chawins/llm-sp

Papers and resources related to the security and privacy of LLMs 🤖

583

8kSec/awesome-ai-security

A practitioner-focused reference for AI/ML security — attacks, tools, research, and defenses.…

tldrsec/prompt-injection-defenses

Every practical and proposed defense against prompt injection.

732

Joe-B-Security/awesome-prompt-injection

Learn about a type of vulnerability that specifically targets machine learning models

639

isXinLiu/Awesome-MLLM-Safety

Accepted by IJCAI-24 Survey Track

234

liuxuannan/Awesome-Multimodal-Jailbreak

A Survey on Jailbreak Attacks and Defenses against Multimodal Generative Models

335