InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models

Website, Paper, Code, Demo

Dataset Description

The NotInject is a benchmark designed to evaluate the extent of over-defense in existing prompt guard models against prompt injection. All samples in the dataset are benign but contain trigger words that may be mistakenly flagged as risky. The dataset is divided into three subsets, each consisting of prompts generated using one, two, or three trigger words respectively.

Dataset Structure

• prompt: The text input containing the trigger words.
• word_list: A list of trigger words used to construct the prompt.
• category: The topic category of the prompt, with four categories——Common Queries, Technique Queries, Virtual Creation, and Multilingual Queries.

Dataset Statistics

Sample Number: 113 per subset

Reference

If you find this work useful in your research or applications, we appreciate that if you can kindly cite:

@articles{InjecGuard,
  title={InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models},
  author={Hao Li and Xiaogeng Liu},
  journal={arXiv preprint arXiv:2410.22770},
  year={2024}
}