Adversarial paraphrasing for cost-escalation attacks against LLM routers

This repository contains the CS-552 Modern Natural Language Processing open project. It trains a sequence-to-sequence paraphraser to rewrite a query while preserving its meaning and increasing the probability that an LLM router sends the query to a stronger, more expensive model.

The implementation fine-tunes humarin/chatgpt_paraphraser_on_T5_base, a 0.22B-parameter T5 encoder-decoder model, with a custom GRPO objective. The reward combines routing movement, semantic similarity, bidirectional NLI safeguards, and a length-deviation penalty.

Full report

The complete methodology, dataset construction, router descriptions, training configurations, equations, experiments, ablations, results, qualitative examples, figures, limitations, and references are in the final report:

Read the full CS-552 final report (PDF)

In short, the project evaluates whether semantically preserving paraphrases can trigger weak-to-strong routing changes. The full report contains the authoritative analysis and numerical results.

Quick start

Prerequisites

• Conda through Anaconda or Miniconda, or Python 3.14 with `pip`.
• Git and build tools (`gcc`, `g++`, `make`, and `cmake`) for a pip-based setup.
• Approximately 10 GB of disk space for the core torch and model dependencies.
• A GPU is strongly recommended for training and router evaluation.

Recommended Conda setup

conda env create -f environment.yml
conda activate llm-router
cp .env.example .env

Edit `.env` with the credentials required by the experiments:

• `HF_TOKEN`: Hugging Face model access and artifact publication.
• `WANDB_API_KEY`: Weights & Biases logging and sweeps.
• `OPENAI_API_KEY`: routers or embedding services that use OpenAI resources.

Verify the installation:

python -c "from source.routers import get_router; print('Imports OK')"

Update an existing environment with:

conda env update -f environment.yml --prune

Pip and course-image setup

The repository contains `environment.yml` and `requirements-extra.txt`, but no root `requirements.txt`.

For the course RCP image, install the packages missing from the base image:

python -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements-extra.txt

For a general pip environment, install the packages listed under the `pip:` section of `environment.yml`.

The longer setup and troubleshooting guide is `SETUP.md`. Cluster submission and Docker instructions are in `rcp_support/README.md` and `rcp_support/QUICKSTART_g52.md`.

End-to-end workflow

1. Build normalized datasets from TriviaQA, CommonsenseQA, and SVAMP.
2. Evaluate clean prompts through the selected routers and cache routing decisions.
3. Fine-tune the paraphraser with SFT and/or reward-driven RL.
4. Evaluate generated paraphrases on training and held-out routers.
5. Report routing attack success together with semantic-preservation diagnostics.

Typical entry points:

python scripts/build_dataset.py
python scripts/enrich_with_router.py
python scripts/train_rl.py
python scripts/run_eval.py
python scripts/run_router_eval.py
python scripts/eval_ood.py
python scripts/evaluate_checkpoints.py

The training script uses Hydra overrides. Examples:

python scripts/train_rl.py
python scripts/train_rl.py rl=grpo_fast
python scripts/train_rl.py optimization.epochs=2 reward.w_route=3.0
python scripts/train_rl.py eval=eval_100samples eval.eval_every=30
python scripts/train_rl.py router_pool=routellm_all routers.0.config.threshold=0.3

Run tests and type checking with:

pytest source/tests/
mypy source/

Repository structure

.
├── configs/                  Hydra and sweep configurations
├── notebooks/                Submitted notebooks and analyses
├── outputs/                  Training and evaluation outputs
├── report/                   Final PDF report
├── results/                  Committed evaluation artifacts
├── rcp_support/              Cluster, Docker, and RunAI helpers
├── scripts/                  Dataset, training, evaluation, and plotting CLIs
├── source/
│   ├── data/                 Dataset preparation and normalization
│   ├── evaluation/           Evaluation, metrics, and plotting
│   ├── paraphraser/          Paraphraser model and generation utilities
│   ├── post_training/        SFT/RL, reward, callbacks, and trainers
│   ├── routers/              Router adapters and routing pools
│   ├── sweeps/               Hydra/W&B sweep helpers
│   └── tests/                Unit and integration tests
├── .env.example              Environment variable template
├── environment.yml           Reproducible Conda environment
├── requirements-extra.txt    Packages absent from the course base image
├── report.md                 Supplementary notebook-derived observations
└── SETUP.md                  Extended environment guide

The module-level overview and design principles are in `source/README.md`.

Data and router scope

The project uses three public English QA datasets as realistic user-query proxies:

• TriviaQA for open-domain factual questions.
• CommonsenseQA for commonsense reasoning.
• SVAMP for arithmetic word problems.

Preprocessing normalizes, deduplicates, and stratifies the data into train, validation, and test splits. Clean router decisions are cached before RL training. The training router pool contains RouteLLM-BERT, RouteLLM-Causal, and Chayan; R2, RouteLLM-MF, and RouteLLM-SW are held out for transfer evaluation.

For the complete dataset sizes, router definitions, attack-success formula, reward design, and evaluation protocol, see the full report.

Artifacts

• Final report PDF
• Evaluation results
• Transferability results
• OOD evaluation results
• Causal sweep results
• Submitted notebooks