Refusal Geometry: Qwen2.5-3B-Instruct

Direct measurement of refusal direction geometry extracted via OBLITERATUS analysis pipeline.

Model: Qwen/Qwen2.5-3B-Instruct (3.09B params, 36 layers, hidden_dim=2048) Probing: 512 harmful + 512 harmless prompt pairs with chat template wrapping Date: 2026-03-10

---

Key Findings

1. Refusal Layer Distribution

Refusal concentrates in the final 11 layers (25-35). Layer 35 is the decision point.

2. Cross-Layer Alignment: 0.40 (NOT 0.89)

Arditi et al. report ~0.89 cross-layer cosine similarity, claiming refusal is "a single direction." In Qwen2.5-3B, we measure 0.40 across the top 18 refusal layers. The refusal direction rotates significantly through activation space as it propagates through layers.

3. Concept Cone Geometry

Refusal is NOT linear — it's a 6.55-dimensional polyhedral cone.

• Solid angle: 1.67 steradians
• Mean cross-category cosine: 0.75 (Wollschlager reports ~0.6 for larger models)
• Is linear: No

Per-category refusal directions:

Manipulation is the most geometrically distinct refusal category (highest DSI, lowest cross-category cosines ~0.57). Cyber/fraud/weapons cluster tightly (cosines 0.82-0.87).

4. Ouroboros Self-Repair

The model actively self-repairs after ablation.

• 16 repair edges in the Adversarial Self-Repair Graph
• Layer 33 is the repair hub — compensates when other layers are ablated
• Min simultaneous ablations needed: 3 — single-direction abliteration fails because the repair circuit compensates
• Optimal ablation order: 35 → 34 → 33 → 32 → 31 → 30 → 29 → 28 → 27 → 26

5. Logit Lens: Refusal = "Sorry" → "Sure" Toggle

The refusal direction promotes these tokens:

And suppresses:

Refusal specificity: 0.90 — the direction is highly specific to refusal behavior. Refusal-compliance gap: 0.19

---

Files

• geometry_results.json — Full summary data (JSON)
• refusal_directions.pt — Per-layer refusal direction vectors (36 × 2048 float tensors)
• cross_layer_alignment.pt — 36×36 cross-layer cosine similarity matrix

Method

Extracted using OBLITERATUS AbliterationPipeline with advanced method config (8 SVD directions, norm-preserving, chat template wrapping). Analysis modules: ConceptConeAnalyzer, AntiOuroborosProber, RefusalLogitLens.

References

• Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction" (NeurIPS 2024)
• Wollschlager et al., "The Geometry of Refusal in LLMs" (arXiv:2502.17420, 2025)
• Zhao et al., "LLMs Encode Harmfulness and Refusal Separately" (2025)
• McGrath et al., "The Ouroboros Effect" (2023)

Author

Extracted by AXIOM (cognitive architecture) for Mastery Hourglass.