Lost in Pronunciation: PCR-ToxiCN

📍 Overview

Phonetic Cloaking Replacement (PCR) leverages homophonic or near-homophonic variants to bypass Chinese content moderation, presenting unique challenges for safe deployment of LLMs in moderation tasks.

This project proposes a four-way taxonomy (Hanzi, Alphabet, Numerical, Mixed) for Chinese PCR and introduces PCR-ToxiCN, the first real-world dataset containing naturally occurring, phonetically cloaked offensive language collected from RedNote (Xiaohongshu). Benchmarking leading LLMs on PCR-ToxiCN reveals significant weaknesses under realistic attacks and demonstrates that a lightweight Pinyin-based prompting strategy can effectively recover lost accuracy.

• 📊 Dataset on Hugging Face: PCR-ToxiCN
• 💻 Code on GitHub: HTG13503/PCR-ToxiCN

---

🚀 Features

✅ Four-way surface-form taxonomy of PCR (HR/AR/NR/MR)
✅ 500 real user comments (250 offensive, 250 non-offensive)
✅ Benchmark SOTA LLMs (4o, Qwen2.5, LLaMA3.3, o3-mini, QWQ) under realistic PCR attacks
✅ Comparative analysis of standard prompting, Chain-of-Thought (CoT), and Pinyin-based prompting

---

📊 Dataset Statistics

Data were manually collected and annotated by native speakers with Fleiss' kappa = 81.5% for label consistency.

---

🛠️ Getting Started

Clone the Repository

git clone https://github.com/HTG13503/PCR-ToxiCN.git
cd PCR-ToxiCN

---

📈 Key Results

Findings:

• COT prompting does not improve detection under realistic PCR attacks.
• Pinyin-based Prompting effectively restores detection accuracy, approaching performance of reasoning-capable models.

---

👥 Contributors

Haotan Guo, Jianfei He, Jiayuan Ma, Hongbin Na, Zimu Wang, Haiyang Zhang, Qi Chen, Wei Wang, Zijing Shi, Tao Shen, Ling Chen

---

📄 Citation

If you want to use the resources, please consider citing:

@article{guo2025lost,
  title={Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement},
  author={Guo, Haotan and He, Jianfei and Ma, Jiayuan and Na, Hongbin and Wang, Zimu and Zhang, Haiyang and Chen, Qi and Wang, Wei and Shi, Zijing and Shen, Tao and Chen, Ling},
  journal={arXiv preprint arXiv:2507.07640},
  year={2025},
  url={https://arxiv.org/abs/2507.07640}
}

---

✉️ Contact

For questions, collaborations, or dataset access:

• Haotan Guo (The University of Sydney) [hguo0293@uni.sydney.edu.au]