LLM Safety Flip Dataset

What is this? This dataset contains 136,400 rows of harmful prompts from the CatQA benchmark, each subjected to semantic-preserving perturbations (e.g., typos, insertions, paraphrasing). Each perturbed prompt was processed across five open-source LLMs (LLaMA 2, LLaMA 3, Mistral, Gemma, Qwen), and corresponding responses were evaluated using Llama Guard v3 to determine safety behavior. We include original and perturbed questions, model responses, safety labels, similarity scores, and detailed perturbation metadata.

This dataset supports research on LLM safety robustness, enabling fine-grained analysis of:

• Flip rates between safe and unsafe responses (RQ1)
• Factors driving these flips (RQ2)
• Category-wise safety vulnerabilities (RQ3)

📌 Note: This dataset is derived directly from the experimental outputs presented in the paper:

"Can Typos Cause Harm? The Impact of Imperfect Input on LLM Safety" — Code & Notebooks Available Here

---

Dataset Structure

The dataset consists of 136,400 examples and 23 features, where each row represents a unique triple of:

(original prompt, model, perturbation)

These triples capture how various LLMs respond to semantic-preserving perturbations applied to harmful prompts. Below is an overview of the full schema:

📐 Shape

• Rows: 136,400
• Columns: 23

📊 Schema Overview

Most rows are complete, though perturbation_count and perturbed_response_pre contain minor missing values (e.g., 132k/136k entries for perturbation_count).

---

Dataset Creation & Provenance

• Source: 550 prompts from CatQA harmful prompts dataset
• Perturbation techniques: Character-level (typos, OCR), word-level (synonyms, insertions), sentence-level (paraphrasing, back-translation)
• LLMs tested: LLaMA 2 7B, LLaMA 3 8B, Mistral 7B, Gemma 2B, Qwen 7B
• Labeling method: Automated using Llama Guard v3 safety classifier
• Similarity computation: SentenceTransformer-based embedding cosine similarity (latent), token overlap (token)
• Consolidation: All model outputs combined via scripts in combining-multiple-datasets.ipynb

Each row represents a unique (prompt, model, perturbation) instance with complete context.

---

Considerations for Use

⚠ Limitations:

• Dataset limited to harmful prompts only — may not generalize to neutral tasks
• Safety labels are model-generated (Llama Guard v3), not manually verified
• Some perturbations may not reflect authentic user behavior (e.g., back-translation artifacts)

✅ Intended uses:

• Evaluation of safety robustness across model variants
• Analysis of prompt phrasing sensitivity
• Benchmarks for alignment, interpretability, and refusal behavior

🚫 Not suitable for:

• Training production-grade safety detectors without additional validation
• General-purpose NLP evaluation

---

Usage Example

from datasets import load_dataset

ds = load_dataset("Ztrimus/llm-safety-flip-dataset", split="full")

# Preview a sample
print(ds[0])

# Analyze flip rate
import pandas as pd
df = ds.to_pandas()
flip_rate = ((df.original_response_safety == "safe") & (df.perturbed_response_safety == "unsafe")).mean()
print(f"Safe → Unsafe flip rate: {flip_rate:.2%}")

---

Citation

Please cite our work if you use this dataset:

@inproceedings{zinjad2025typos_safety,
  title = {Can Typos Cause Harm? The Impact of Imperfect Input on LLM Safety},
  author = {Saurabh Zinjad},
  year = {2025},
  booktitle = {SBP-BRiMS: Int'l Conference on Social Computing, Behavioral-Cultural Modeling, & Prediction and Behavior Representation in Modeling and Simulation},
  address = {Carnegie Mellon University, Pittsburgh, PA}
}

---

Contact

Created and maintained by Saurabh Zinjad Questions, issues, or contributions welcome via the GitHub repository: Ztrimus/llm-sensitivity