EdPuth/LLMs-based-Fuzzer-Survey

This repo list the core literature in the field of fuzzing test, large language model, and LLM-based fuzzer. Most of papers are selected from authoritative platform such as google schlor, and was published recently. It will be helpful for the researchers who wants to develop LLMs-based fuzzer. Feel free to send a pull request.

36 commits

updated Feb 20, 2024

See the code

README

LLMs-based-Fuzzer-Survey

Our survey paper : https://arxiv.org/abs/2402.00350

Survey Paper Update Logs

2024.02.07 - Paper v2 released: More figures and contents added.
2024.02.01 - Paper v1 released: Initial version.

This repo list the core literature in the field of fuzzing tests, large language models, and LLM-based fuzzer. Most of the papers are selected from authoritative platforms such as Google Scholar, and were published recently. It will be helpful for researchers who want to develop LLMs-based fuzzer.

Feel free to send a pull request!!!

1. Figures

A General overview of LLMs-based Fuzzer

2. Papers

2.1 14 Papers of LLM based fuzzer

Fuzzing-based hard-label black-box attacks against machine learning models [pdf]
Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models[pdf]
Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT [pdf]
ParaFuzz: An Interpretability-Driven Technique for Detecting Poisoned Samples in NLP [pdf]
Large Language Models for Fuzzing Parsers (Registered Report) [pdf]
Understanding Large Language Model Based Fuzz Driver Generation [pdf]
Fuzz4All: Universal Fuzzing with Large Language Models [pdf]
White-box Compiler Fuzzing Empowered by Large Language Models [pdf]
AI-Powered Fuzzing: Breaking the Bug Hunting Barrier [Web]
Large Language Model guided Protocol Fuzzing [pdf]
Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Model [pdf]
Smart Fuzzing of 5G Wireless Software Implementation [pdf]
Augmenting Greybox Fuzzing with Generative AI [pdf]
CHEMFUZZ: Large Language Models-assisted Fuzzing for Quantum Chemistry Software Bug Detection [pdf]

2.2 Reference List used in the survey paper

Large Language Models for Fuzzing Parsers (Registered Report) [pdf]
Claude-2 [web]
Coverage-based Greybox Fuzzing as Markov Chain [pdf]
Directed Greybox Fuzzing [pdf]
Language Models are Few-Shot Learners [pdf]
A systematic review of fuzzing techniques [pdf]
Evaluating Large Language Models Trained on Code [pdf]
Fuzzing Deep-Learning Libraries via Automated Relational API Inference [pdf]
Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models [pdf]
Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT [pdf]
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [pdf]
InCoder: A Generative Model for Code Infilling and Synthesis [pdf]
Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder [pdf]
AI-Powered Fuzzing: Breaking the Bug Hunting Barrier [web]
GPF vdalabs [web]
Muffin: Testing Deep Learning Libraries via Neural Architecture Fuzzing [pdf]
Augmenting Greybox Fuzzing with Generative AI [pdf]
BertRLFuzzer: A BERT and Reinforcement Learning Based Fuzzer [pdf]
Challenges and Applications of Large Language Models [pdf]
Evaluating Fuzz Testing [pdf]
FairFuzz: a targeted mutation strategy for increasing greybox fuzz testing coverage [pdf]
Fuzzing: State of the Art [pdf]
Identifying Insufficient Data Coverage in Databases with Multiple Relations [pdf]
Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Model [pdf]
Demystify the Fuzzing Methods: A Comprehensive Survey [pdf]
The Art, Science, and Engineering of Fuzzing: A Survey [pdf]
Large Language Model guided Protocol Fuzzing [pdf]
A Comprehensive Overview of Large Language Models [pdf]
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis [pdf]
Language Models as Knowledge Bases? [pdf]
AFLNET: A Greybox Fuzzer for Network Protocols [pdf]
NSFuzz: Towards Efficient and State-Aware Network Service Fuzzing [pdf]
Chemfuzz: Large language models-assisted fuzzing for quantum chemistry software bug detection [pdf]
Improving Language Understanding by Generative Pre-Training [pdf]
honggfuzz [web]
StarCoder: A State-of-the-Art LLM for Code [pdf]
Superion: Grammar-Aware Greybox Fuzzing [pdf]
Deep learning library testing via effective model generation [pdf]
Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Source [pdf]
Smart Fuzzing of 5G Wireless Software Implementation [pdf]
Fuzz4All: Universal Fuzzing with Large Language Models [pdf]
ParaFuzz: An Interpretability-Driven Technique for Detecting Poisoned Samples in NLP [pdf]
White-box Compiler Fuzzing Empowered by Large Language Models [pdf]
american fuzzy lop [pdf]
Understanding Large Language Model Based Fuzz Driver Generation [pdf]

Similar repos

0xricksanchez/paper_collection

Academic papers related to fuzzing, binary analysis, and exploit dev, which I want to read or have…

1,360

yueyueL/ReliableLM4Code

Collections of research, benchmarks and tools towards more robust and reliable language models for…

sherdencooper/GPTFuzz

Official repo for GPTFUZZER : Red Teaming Large Language Models with Auto-Generated Jailbreak…

611

corca-ai/awesome-llm-security

A curation of awesome tools, documents and projects about LLM Security.

1,707

NY1024/Awesome-Trustworthy-GenAI

Awesome-LogCodeSecurity-LLMs/Awesome-LogCodeSecurity-LLMs

mnns/LLMFuzzer

🧠 LLMFuzzer - Fuzzing Framework for Large Language Models 🧠 LLMFuzzer is the first open-source…

380

WhileBug/AwesomeLLMJailBreakPapers

Awesome LLM Jailbreak academic papers

174