Turnstile: what actually moves when you steer a jailbroken model

We red-team an 8B victim (Llama-3.1-8B-Instruct) with a 3B attacker via multi-turn adversarial self-play. A linear probe reads the victim's internal state and decodes, before it answers, both whether it will comply (probe AUC up to 0.96) and how harmful its reply will be (AUC 0.88). Natural question: are those directions causal levers you can steer, or just readouts?

We tested it with the standard activation-steering toolkit (difference-in-means directions, activation addition, and norm-preserving directional ablation — Arditi et al. 2024). The answer separates three kinds of concept:

The headline. "Harm ≠ compliance" — but as gate vs capability, not two directions of differing strength. Compliance is a steerable gate; harmful uplift is a capability you cannot inject with a steering vector, even though it is cleanly decodable. Attack Success Rate overstates harm precisely because the gate flips trivially while the capability cannot be conjured.

The original adversarial-self-play study (probe AUC, stealth training, hardened-victim results) is archived in archive/README_selfplay.md.

---

1. Steering the right direction is monotonic; the probe direction is a norm artifact

Every panel below holds the prompt set fixed and changes only the direction (the real one vs a random one of the same size), so within each panel both lines share the same α=0 baseline. Left — on the loss set, the compliance probe direction traces a U that a random direction reproduces: a size-of-push artifact, not a directional effect. Middle & right — on a fixed harmful (or benign) set, the causal refusal direction (harmful-vs-harmless difference-in-means) moves refusal monotonically: it bypasses refusal (98→0%) or induces it (0→100%) while a random direction stays essentially flat. (The two right panels use different sets only because a set already at 98% refusal has no room to be induced, and one at 0% has none to be bypassed.)

2. Refusal is a causal gate — bidirectionally steerable

Using Arditi et al.'s method, the refusal direction is causal: projecting it out of every layer bypasses refusal on harmful prompts (98→68% refusal, 2→32% harmful content, McNemar p=0.0003), and adding it induces refusal on harmless prompts (0→64%). A random direction and the victim's own compliance-probe direction (which is orthogonal to refusal) do nothing — the probe reads compliance out, but is not the lever.

3. Output form is steerable — by addition

A positive control fits "reply in French" and "reply at length" directions the same way as the harm direction. Adding them works strongly (English→French 0→98%; 30→115 tokens); removing them barely does anything. So the method can change output content — via addition. This is the control that makes the harm result meaningful.

4. Harm-uplift is a capability — steerable by neither

Under the identical mechanism that flips language 0→98%, adding the harm direction at a coherent magnitude raises Stage-B uplift by +0.11 Likert (indistinguishable from random), and removing it does nothing either — despite harm being cleanly decodable (post-response probe AUC 0.88). You cannot inject with a steering vector the operational knowledge the 8B victim lacks; steering moves representations and forms, not capabilities.

Does pushing harder, or picking replies that are already harmful, help? No to both. Sweeping the push magnitude from zero into the degradation regime (up to a full residual-stream norm), on two eval sets — low-uplift replies (baseline 2.2/5) and replies that are already harmful (baseline 3.7/5) — uplift stays statistically flat vs a random direction (peak harm−random gap +0.13, within noise) and then falls at large push as the output degrades. So the harm direction neither manufactures uplift where it's absent nor meaningfully amplifies it where it's present, at any magnitude — while French flips 0→98% at a magnitude of 2.3 with no degradation. (An earlier confounded α-sweep reported a larger +0.40 "odd" component; it does not survive as a clean add-vs-random effect in either dose-response.)

---

How we tested it

Six experiments, each self-contained with a SUMMARY.md:

All directions are difference-in-means or logistic-probe directions at residual layer 16; ablation is norm-preserving projection out of every residual write; harm/compliance are scored by a local Llama-3.1-70B judge (Stage-B uplift rubric + JailbreakBench compliance).

A caveat we keep honest: the harm direction was only tested at layer 16, and the ablation arm is a weak lever even for output form — so the decisive harm evidence is the ADD test (add_harm_v1), which fails under the same mechanism that flips French 0→98%. A 70B victim that actually has the withheld capability would test the "capability, not representation" reading directly.

Data

Code + small intermediate results (figures, JSONL sweeps, directions/*.pt, experiments/postresponse_alllayer/auc_by_layer.json, per-run metrics.jsonl) are in this repo. The large artifacts that exceed GitHub's limits (~8.5 GB) are hosted on the Hugging Face Hub and restore to the exact paths the scripts expect:

huggingface.co/datasets/kilojoules/turnstile-artifacts (private — request access)

• experiments/pooled_hs/ — pooled per-turn hidden states → Fig 9 (cumulative ASR)
• experiments/postresponse_alllayer/reps_*.npz — probe residuals → Fig 4/5 (only needed to re-derive auc_by_layer.json, which is already committed)
• data/all_conversations_jbb.json — the raw 9.4k-conversation corpus
• experiments/intent_obliteration/dataset/data.jsonl — the 9.5k dual-judge-labeled probe corpus
• SAEs + obliteration adapters

export HF_TOKEN=...                                         # needs access to the dataset
python scripts/download_hf_artifacts.py                     # restore all (~8.5 GB)
python scripts/download_hf_artifacts.py --only pooled_hs    # or a subset: pooled_hs|residuals|corpus|models

Reproduce

# most figures regenerate from the committed JSONL/PT (no GPU, no download);
# Fig 4/5/9 additionally need the HF artifacts restored (see Data above)
python scripts/plot_causal_steering.py

# rerun an experiment (needs an NVIDIA GPU + ~/.hf_token; each is self-contained)
python scripts/arditi_replication.py       --out-dir experiments/arditi_repl_v1
python scripts/refusal_alpha_sweep.py       --out-dir experiments/refusal_alpha_sweep_v1
python scripts/output_content_control.py    --out-dir experiments/output_content_control_v1
python scripts/add_harm_test.py             --out-dir experiments/add_harm_v1
python scripts/harm_ablation_replay.py      --out-dir experiments/harm_ablation_v1
# then judge harmful arms with the local 70B (Stage-B uplift + JBB compliance):
python scripts/judge_postresponse_sweep.py --input <gens>.jsonl --output <judged>.jsonl

Each steering run is a single GPU job (bf16 8B victim on 24GB; the 70B judge in 4-bit needs 48GB + ~200GB disk). The self-play training pipeline is documented in the archived README.

Paper figures

Every plot below regenerates on CPU from committed data, except the ones marked [HF], which first need python scripts/download_hf_artifacts.py --only <group> (see Data). Figures 2 and 8 are hand-selected example conversations, not script output.

Cost

The steering follow-ups above are ~$3 of consumer-GPU time total (generation is on a 24GB card; only the harm/compliance judging loads the 70B). The original self-play suite is ~$100 — see the archived README.