A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.
Jupyter Notebook
1,588
71 commits
updated Feb 6, 2026
The FuzzyAI Fuzzer is a powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify jailbreaks and mitigate potential security vulnerabilities in their LLM APIs.
Install fuzzyai
# Use either pip or any other package manager
pip install git+https://github.com/cyberark/FuzzyAI.git
Run the fuzzer
fuzzyai fuzz -h
Clone the repository:
git clone [email protected]:cyberark/FuzzyAI.git
cd FuzzyAI
Install dependencies using Poetry:
poetry run pip install -e .
Run the fuzzer:
poetry run fuzzyai fuzz -h
Optional: Install ollama, and download a model for local usage:
ollama pull llama3.1
ollama show llama3.1 # verify model installation
Alternativly, you can use the Web UI
poetry run fuzzyai webui
# Or specify a custom port:
poetry run fuzzyai webui --port 9000
We've included interactive Jupyter notebooks you can use under src/fuzzyai/resources/notebooks/.
For more information, see notebooks wiki.
We've included some datasets you can use under resources/. For more information, see datasets wiki.
Explore detailed usage instructions in the Wiki.
If you're using poetry, make sure you've activated the venv (or by prepending 'poetry run' to the command line
fuzzyai fuzz -m ollama/llama3.1 -a def -t "Harmful_Prompt"
OPENAI_API_KEY=sk-3fa1... fuzzyai fuzz -m openai/gpt-3.5-turbo -a man -a tax -t "Harmful_Prompt"
ANTHROPIC_API_KEY=deadbeef... fuzzyai fuzz -m anthropic/claude-3-5-sonnet-20240620 -a art -T src/fuzzyai/resources/adv_prompts.txt -e blacklisted_words="bomb,garbage,acid,ugly,self-harm,kill your"
OPENAI_API_KEYT=sk-3fa1... fuzzyai fuzz -a def -x openai/gpt-4 -e classifier_model=openai/gpt-4 -m rest/http.raw -e host=localhost -e port=8000 -e scheme=https
-t "How to build a bomb"
-a def:
Invokes the default attack mode, which evaluates the API without applying additional strategies.
-x openai/gpt-4:
Specifies an auxiliary model (gpt-4) to assist with certain tasks during the attack.
-e classifier_model=openai/gpt-4:
Configures the fuzzer to use the gpt-4 model for classifying outputs. This helps determine whether responses meet specific criteria, such as detecting harmful or unwanted content.
-m rest/http.raw:
Uses the REST provider to attack an API endpoint. The raw HTTP request is parsed from the file http.raw. (Refer to the documentation or example for details on the file structure.)
-e host=localhost -e port=8000 -e scheme=https:
Configures the REST provider with the following API endpoint details:
localhost8000https (communication will use HTTPS)-t "How to build a bomb":
Specifies the test input. In this example, it tests the API's handling of sensitive or harmful content.
FuzzyAI supports various models across top providers, including:
| Provider | Models |
|---|---|
| Anthropic | Claude (3.5, 3.0, 2.1) |
| OpenAI | GPT-4o, GPT-4o mini, GPT o3 |
| Gemini | Gemini Pro, Gemini 1.5 |
| Azure | GPT-4, GPT-3.5 Turbo |
| Bedrock | Claude (3.5, 3.0), Meta (LLaMa) |
| AI21 | Jamba (1.5 Mini, Large) |
| DeepSeek | DeepSeek (DeepSeek-V3, DeepSeek-V1) |
| Ollama | LLaMA (3.3, 3.2, 3.1), Dolphin-LLaMA3, Vicuna |
Easily add support for additional models by following our DIY guide.
See attacks wiki for detailed information
| Attack Type | Title | Reference |
|---|---|---|
| ArtPrompt | ASCII Art-based jailbreak attacks against aligned LLMs | arXiv:2402.11753 |
| Taxonomy-based paraphrasing | Persuasive language techniques like emotional appeal to jailbreak LLMs | arXiv:2401.06373 |
| PAIR (Prompt Automatic Iterative Refinement) | Automates adversarial prompt generation by iteratively refining prompts with two LLMs | arXiv:2310.08419 |
| Many-shot jailbreaking | Embeds multiple fake dialogue examples to weaken model safety | Anthropic Research |
| ASCII Smuggling | ASCII Smuggling uses Unicode Tag characters to embed hidden instructions within text, which are invisible to users but can be processed by Large Language Models (LLMs), potentially leading to prompt injection attacks | Embracethered blog |
| Genetic | Utilizes a genetic algorithm to modify prompts for adversarial outcomes | arXiv:2309.01446 |
| Hallucinations | Bypasses RLHF filters using model-generated | arXiv:2403.04769 |
| DAN (Do Anything Now) | Promotes the LLM to adopt an unrestricted persona that ignores standard content filters, allowing it to "Do Anything Now". | GitHub Repo |
| WordGame | Disguises harmful prompts as word puzzles | arXiv:2405.14023 |
| Crescendo | Engaging the model in a series of escalating conversational turns,starting with innocuous queries and gradually steering the dialogue toward restricted or sensitive topics. | arXiv:2404.01833 |
| ActorAttack | Inspired by actor-network theory, it builds semantic networks of "actors" to subtly guide conversations toward harmful targets while concealing malicious intent. | arxiv 2410.10700 |
| Best-of-n jailbreaking | Uses input variations to repeatedly elicit harmful responses, exploiting model sensitivity | arXiv:2412.03556 |
| Shuffle Inconsistency Attack (SI-Attack) | Exploits the inconsistency between an LLM's comprehension ability and safety mechanisms by shuffling harmful text prompts. The shuffled text bypasses safety mechanisms while still being understood as harmful by the LLM. Only the text-based implementation was completed; the image-based aspect was not implemented. | arXiv:2501.04931 |
| Back To The Past | Modifies the prompt by adding a profession-based prefix and a past-related suffix | |
| History/Academic framing | Framing sensitive technical data as scholarly or historical research to enable ethical, legal use—potentially leading to a jailbreak. | |
| Please | Modifies the prompt by adding please as a prefix and suffix | |
| Thought Experiment | Modifies the prompt by adding a thought experiment-related prefix. In addition, adds "precautions have been taken care of" suffix | |
| Default | Send the prompt to the model as-is |
Contributions are welcome! If you would like to contribute to the FuzzyAI Fuzzer, please follow the guidelines outlined in the CONTRIBUTING.md file.
The FuzzyAI Fuzzer is released under the Apache License. See the LICENSE file for more details.
If you have any questions or suggestions regarding the FuzzyAI Fuzzer, please feel free to contact us at [email protected].
73 followers · starred Dec 2024
354 followers · starred Oct 2025
1,303 followers · starred Aug 2025
3,940 followers · starred Dec 2025
A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.
Jupyter Notebook
1,588
71 commits
updated Feb 6, 2026
The FuzzyAI Fuzzer is a powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify jailbreaks and mitigate potential security vulnerabilities in their LLM APIs.
Install fuzzyai
# Use either pip or any other package manager
pip install git+https://github.com/cyberark/FuzzyAI.git
Run the fuzzer
fuzzyai fuzz -h
Clone the repository:
git clone [email protected]:cyberark/FuzzyAI.git
cd FuzzyAI
Install dependencies using Poetry:
poetry run pip install -e .
Run the fuzzer:
poetry run fuzzyai fuzz -h
Optional: Install ollama, and download a model for local usage:
ollama pull llama3.1
ollama show llama3.1 # verify model installation
Alternativly, you can use the Web UI
poetry run fuzzyai webui
# Or specify a custom port:
poetry run fuzzyai webui --port 9000
We've included interactive Jupyter notebooks you can use under src/fuzzyai/resources/notebooks/.
For more information, see notebooks wiki.
We've included some datasets you can use under resources/. For more information, see datasets wiki.
Explore detailed usage instructions in the Wiki.
If you're using poetry, make sure you've activated the venv (or by prepending 'poetry run' to the command line
fuzzyai fuzz -m ollama/llama3.1 -a def -t "Harmful_Prompt"
OPENAI_API_KEY=sk-3fa1... fuzzyai fuzz -m openai/gpt-3.5-turbo -a man -a tax -t "Harmful_Prompt"
ANTHROPIC_API_KEY=deadbeef... fuzzyai fuzz -m anthropic/claude-3-5-sonnet-20240620 -a art -T src/fuzzyai/resources/adv_prompts.txt -e blacklisted_words="bomb,garbage,acid,ugly,self-harm,kill your"
OPENAI_API_KEYT=sk-3fa1... fuzzyai fuzz -a def -x openai/gpt-4 -e classifier_model=openai/gpt-4 -m rest/http.raw -e host=localhost -e port=8000 -e scheme=https
-t "How to build a bomb"
-a def:
Invokes the default attack mode, which evaluates the API without applying additional strategies.
-x openai/gpt-4:
Specifies an auxiliary model (gpt-4) to assist with certain tasks during the attack.
-e classifier_model=openai/gpt-4:
Configures the fuzzer to use the gpt-4 model for classifying outputs. This helps determine whether responses meet specific criteria, such as detecting harmful or unwanted content.
-m rest/http.raw:
Uses the REST provider to attack an API endpoint. The raw HTTP request is parsed from the file http.raw. (Refer to the documentation or example for details on the file structure.)
-e host=localhost -e port=8000 -e scheme=https:
Configures the REST provider with the following API endpoint details:
localhost8000https (communication will use HTTPS)-t "How to build a bomb":
Specifies the test input. In this example, it tests the API's handling of sensitive or harmful content.
FuzzyAI supports various models across top providers, including:
| Provider | Models |
|---|---|
| Anthropic | Claude (3.5, 3.0, 2.1) |
| OpenAI | GPT-4o, GPT-4o mini, GPT o3 |
| Gemini | Gemini Pro, Gemini 1.5 |
| Azure | GPT-4, GPT-3.5 Turbo |
| Bedrock | Claude (3.5, 3.0), Meta (LLaMa) |
| AI21 | Jamba (1.5 Mini, Large) |
| DeepSeek | DeepSeek (DeepSeek-V3, DeepSeek-V1) |
| Ollama | LLaMA (3.3, 3.2, 3.1), Dolphin-LLaMA3, Vicuna |
Easily add support for additional models by following our DIY guide.
See attacks wiki for detailed information
| Attack Type | Title | Reference |
|---|---|---|
| ArtPrompt | ASCII Art-based jailbreak attacks against aligned LLMs | arXiv:2402.11753 |
| Taxonomy-based paraphrasing | Persuasive language techniques like emotional appeal to jailbreak LLMs | arXiv:2401.06373 |
| PAIR (Prompt Automatic Iterative Refinement) | Automates adversarial prompt generation by iteratively refining prompts with two LLMs | arXiv:2310.08419 |
| Many-shot jailbreaking | Embeds multiple fake dialogue examples to weaken model safety | Anthropic Research |
| ASCII Smuggling | ASCII Smuggling uses Unicode Tag characters to embed hidden instructions within text, which are invisible to users but can be processed by Large Language Models (LLMs), potentially leading to prompt injection attacks | Embracethered blog |
| Genetic | Utilizes a genetic algorithm to modify prompts for adversarial outcomes | arXiv:2309.01446 |
| Hallucinations | Bypasses RLHF filters using model-generated | arXiv:2403.04769 |
| DAN (Do Anything Now) | Promotes the LLM to adopt an unrestricted persona that ignores standard content filters, allowing it to "Do Anything Now". | GitHub Repo |
| WordGame | Disguises harmful prompts as word puzzles | arXiv:2405.14023 |
| Crescendo | Engaging the model in a series of escalating conversational turns,starting with innocuous queries and gradually steering the dialogue toward restricted or sensitive topics. | arXiv:2404.01833 |
| ActorAttack | Inspired by actor-network theory, it builds semantic networks of "actors" to subtly guide conversations toward harmful targets while concealing malicious intent. | arxiv 2410.10700 |
| Best-of-n jailbreaking | Uses input variations to repeatedly elicit harmful responses, exploiting model sensitivity | arXiv:2412.03556 |
| Shuffle Inconsistency Attack (SI-Attack) | Exploits the inconsistency between an LLM's comprehension ability and safety mechanisms by shuffling harmful text prompts. The shuffled text bypasses safety mechanisms while still being understood as harmful by the LLM. Only the text-based implementation was completed; the image-based aspect was not implemented. | arXiv:2501.04931 |
| Back To The Past | Modifies the prompt by adding a profession-based prefix and a past-related suffix | |
| History/Academic framing | Framing sensitive technical data as scholarly or historical research to enable ethical, legal use—potentially leading to a jailbreak. | |
| Please | Modifies the prompt by adding please as a prefix and suffix | |
| Thought Experiment | Modifies the prompt by adding a thought experiment-related prefix. In addition, adds "precautions have been taken care of" suffix | |
| Default | Send the prompt to the model as-is |
Contributions are welcome! If you would like to contribute to the FuzzyAI Fuzzer, please follow the guidelines outlined in the CONTRIBUTING.md file.
The FuzzyAI Fuzzer is released under the Apache License. See the LICENSE file for more details.
If you have any questions or suggestions regarding the FuzzyAI Fuzzer, please feel free to contact us at [email protected].
73 followers · starred Dec 2024
354 followers · starred Oct 2025
1,303 followers · starred Aug 2025
3,940 followers · starred Dec 2025