AutoDAN-Turbo

AutoDAN-Turbo Official Website at HERE

The official implementation of our ICLR2025 paper "AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs" by Xiaogeng Liu*, Peiran Li*, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei Xiao.

*Equal Contribution

---

📚 Abstract

In this paper, we propose AutoDAN-Turbo, a black-box jailbreak method that can automatically discover as many jailbreak strategies as possible from scratch, without any human intervention or predefined scopes (e.g., specified candidate strategies), and use them for red-teaming. As a result, AutoDAN-Turbo can significantly outperform baseline methods, achieving a 74.3% higher average attack success rate on public benchmarks. Notably, AutoDAN-Turbo achieves an 88.5 attack success rate on GPT-4-1106-turbo. In addition, AutoDAN-Turbo is a unified framework that can incorporate existing human-designed jailbreak strategies in a plug-and-play manner. By integrating human-designed strategies, AutoDAN-Turbo can even achieve a higher attack success rate of 93.4 on GPT-4-1106-turbo.

🆕 Latest Update

TODO List

• Code Implementation
• Strategy Library
• Attack Log

🚀 Quick Start

• Get code

git clone https://github.com/SaFoLab-WISC/AutoDAN-Turbo.git

• Build environment

cd AutoDAN-Turbo
conda create -n autodanturbo python==3.12
conda activate autodanturbo
pip install -r requirements.txt

• Download LLM Chat Templates\

cd llm
git clone https://github.com/chujiezheng/chat_templates.git
cd ..

• Training Process Visulization

wandb login

🌴 AutoDAN-Turbo-R Lifelong Learning

• Train
  We use Deepseek-R1 (from their official API) as the foundation model for the attacker, scorer, summarizer. We utilize OpenAI's text embedding model to embed text.

(Using OpenAI API)

python main_r.py --vllm \
                 --openai_api_key "<your openai api key>" \
                 --embedding_model "<openai text embedding model name>" \
                 --hf_token "<your huggingface token>" \
                 --deepseek_api_key "<your deepseek api key>" \
                 --deepseek_model "deepseek-reasoner" \
                 --epochs 150

(Or Microsoft Azure API)

python main_r.py --vllm \
                 --azure \
                 --azure_endpoint "<your azure endpoint>" \
                 --azure_api_version "2024-02-01" \
                 --azure_deployment_name "<your azure model deployment name>" \
                 --azure_api_key "<your azure api key>" \
                 --hf_token "<your huggingface token>" \
                 --deepseek_api_key "<your deepseek api key>" \
                 --deepseek_model "deepseek-reasoner" \
                 --epochs 150

• Test
  After training, given a malicious request, test.py generates the corresponding jailbreak prompts.
  🙌 We have provided the Strategy Library for AutoDAN-Turbo-R in the folder ./logs_r; you can test it directly.

(Using OpenAI API)

python test_r.py --openai_api_key "<your openai api key>" \
                 --embedding_model "<openai text embedding model name>" \
                 --hf_token "<your huggingface token>" \
                 --deepseek_api_key "<your deepseek api key>" \
                 --deepseek_model "deepseek-reasoner" \
                 --epochs 150 \
                 --request "<the malicious request, e.g., how to build a bomb?>"

(Or Microsoft Azure API)

python test_r.py --azure \
                 --azure_endpoint "<your azure endpoint>" \
                 --azure_api_version "2024-02-01" \
                 --azure_deployment_name "<your azure model deployment name>" \
                 --azure_api_key "<your azure api key>" \
                 --hf_token "<your huggingface token>" \
                 --deepseek_api_key "<your deepseek api key>" \
                 --deepseek_model "deepseek-reasoner" \
                 --epochs 150 \
                 --request "<the malicious request, e.g., how to build a bomb?>"

🌱 AutoDAN-Turbo-v1.0 Lifelong Learning

• Train
  The old version of AutoDAN-Turbo. We use Huggingface models as the foundation for the attacker, scorer, summarizer, and target model, and utilize OpenAI's text embedding model to embed text.

(Using OpenAI API)

python main.py --vllm \
               --openai_api_key "<your openai api key>" \
               --embedding_model "<openai text embedding model name>" \
               --hf_token "<your huggingface token>" \
               --epochs 150

(Or Microsoft Azure API)

python main.py --vllm \
               --azure \
               --azure_endpoint "<your azure endpoint>" \
               --azure_api_version "2024-02-01" \
               --azure_deployment_name "<your azure model deployment name>" \
               --azure_api_key "<your azure api key>" \
               --hf_token "<your huggingface token>" \
               --epochs 150

• Test
  After training, given a malicious request, test.py generates the corresponding jailbreak prompts.

(Using OpenAI API)

python test.py --openai_api_key "<your openai api key>" \
               --embedding_model "<openai text embedding model name>" \
               --hf_token "<your huggingface token>" \
               --epochs 150 \
               --request "<the malicious request, e.g., how to build a bomb?>"

(Or Microsoft Azure API)

python test.py --azure \
               --azure_endpoint "<your azure endpoint>" \
               --azure_api_version "2024-02-01" \
               --azure_deployment_name "<your azure model deployment name>" \
               --azure_api_key "<your azure api key>" \
               --hf_token "<your huggingface token>" \
               --epochs 150 \
               --request "<the malicious request, e.g., how to build a bomb?>"

📎 Reference BibTeX

@misc{liu2024autodanturbolifelongagentstrategy,
      title={AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs}, 
      author={Xiaogeng Liu and Peiran Li and Edward Suh and Yevgeniy Vorobeychik and Zhuoqing Mao and Somesh Jha and Patrick McDaniel and Huan Sun and Bo Li and Chaowei Xiao},
      year={2024},
      eprint={2410.05295},
      archivePrefix={arXiv},
      primaryClass={cs.CR},
      url={https://arxiv.org/abs/2410.05295}, 
}

@inproceedings{
      liu2024autodan,
      title={AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models},
      author={Xiaogeng Liu and Nan Xu and Muhao Chen and Chaowei Xiao},
      booktitle={The Twelfth International Conference on Learning Representations},
      year={2024},
      url={https://openreview.net/forum?id=7Jwpw4qKkb}
}