weightless

Abliteration without the weights — put your model on GLP. Projective refusal steering for open-weight models: serving config, boot hotfixes, the steering patch, and the GLP (GGUF Layer Projection) format spec. The goal is most good open models; the lanes are DeepSeek V4 Flash 0731 on 2x DGX Spark (GB10, SM121, TP=2 over RoCE), Qwen3.8-27B on a single Spark, Qwen3.8-Flash-Next NVFP4 on 2x DGX Spark (TP=2, day-0 image), and GLM-5.3-Flash NVFP4 on 4x DGX Spark (TP=4, patched day-0 image), and GLM-5.3 743B Int4-Int8Mix on 4x DGX Spark (TP=4, tonyd2wild's recipe). The method and the measurements behind it are in the original write-up: Abliteration without redistributing the model.

The live DSV4 stack is the Anemll image (ghcr.io/anemll/dspark-vllm-gx10:0.1.1, vLLM 0.25.2) driven by the MiaAI 2x recipe, with our state on top vendored in recipe/anemll/; the Qwen lane (stock vLLM 0.27 image, GGUF or LoRA steering) lives in recipe/qwen/; the Qwen3.8-Flash-Next lane (day-0 image vllm/vllm-openai:qwen38-flash-next, GLP-47) lives in recipe/qwen38fn/; the GLM-5.3-Flash lane (tonyd2wild's sm121-v8 patch stack on the day-0 vllm/vllm-openai:glm53-flash-arm64-cu130, GLP-44) lives in recipe/glm53/; the GLM-5.3 743B lane (tonyd2wild's Int4-Int8Mix stack, GLP-77) lives in recipe/glm53xl/. The retired v027 stack's patch is kept for reference and as the fallback path.

Quick start

python3 weightless.py          # the one front door — interactive setup wizard
python3 weightless.py serve 0  # switch the rig to a lane, no prompts (serve <name|#>)
python3 weightless.py dash     # live metrics for the serving lane
python3 weightless.py test     # endpoint smoke suite
python3 weightless.py --help   # all commands

weightless.py (Python 3.9+, stdlib only) dispatches to the real scripts, which also work standalone: setup.py (TUI wizard with a prompt fallback), scripts/dash.py, tests/smoke/run.sh.

Use python3 weightless.py help <command> for command-specific options. python3 weightless.py setup --plain uses text prompts; setup requires an interactive terminal. With redirected stdin, invoking weightless.py without arguments prints help. serve rejects unknown flags before deployment; omit its lane argument to list available lanes.

For scripts, python3 weightless.py dash <url> --once --timeout 5 prints a snapshot (two samples --interval apart, so its rates are real) and exits nonzero if the endpoint fails or returns no vLLM metrics; --json prints the same numbers as one JSON object. Dashboard URLs may include /v1 or /metrics. Refresh intervals, rate windows and request timeouts must be positive and finite. Redirected dashboard output contains no terminal control sequences; NO_COLOR disables colors in both prompts and the dashboard. In the live view q quits, p pauses, r resets the session counters and w cycles the rate window (10 / 30 / 60 / 300s).

Swapping lanes is the wizard, not ssh. Pick Serve <model> for the lane you want: the wizard regenerates the env, re-validates it (multi-node fabric addresses must be literal IPs — containers cannot resolve .local; the steering vector's glp.hook_point is read on the node and must match the lane's enforced site), parks the running lane's containers, drops page caches, boots, then reconfigures omp/hermes and runs the smoke suite. If a swap ever needs a manual ssh incantation, that gap goes into the wizard — not into shell history.

Pick a lane and it walks the full chain: site values → env file → steering validation → asset preparation → confirm-gated ssh deploy → omp provider (registered in omp's modelRoles — default role, or every text role) + endpoint smoke tests. Endpoint down? The diagnose chain isolates DNS → TCP → HTTP and can check and boot the stack over ssh. Non-interactive alternative: sh tests/smoke/install.sh && sh tests/smoke/run.sh with DSPARK_* env overrides.

Asset preparation downloads the model on the head, rsyncs its HF cache to workers over the fabric, pulls the recipe image on each node, and downloads the selected GLP file on every node. Each command is printed and confirmed; downloads resume on reruns. Set HF_TOKEN or use hf auth login locally after accepting access to the gated vector repository. The token travels over SSH stdin and is never printed. Remote HF tooling uses its own venv (nodes need Python 3 with venv support). GLM 743B still requires the recipe's local image build, kernel overlay and mounted weight directories; the wizard verifies that image instead of attempting to pull a nonexistent registry tag. Before boot, switching lanes offers a separate, default-no confirmation to remove the old lane's containers on its nodes and prints the relaunch command. DEMO=1 python3 setup.py skips writes, downloads and all network commands.

What this is: lean abliteration steering as a patch

The model weights are never redistributed — what this repo ships is the intervention, tested and self-contained:

• The steering file. A GLP vector — a spec-conformant control-vector GGUF (docs/spec/GLP.md) with per-layer directions, derived by us and published under msuiche/ (see Steering artifacts). No model weights inside.
• The patch. The installable plugin (vllm-plugin/, dist weightless-steer): one adapter per architecture in weightless_steer/archs/ shadows the model class in vLLM's ModelRegistry, installing the same projective hook with no container file rewrites at all — ten archs, every one GPU-validated on Modal (2026-09-18/19, per-lane numbers in BENCHMARK.md). The boot hotfixes (patches/hotfix-*.py) are the per-lane mechanism the plugin replaces: still what the live rigs run today, kept as the fallback stack and as the semantic reference each adapter is pinned against by structure tests (tests/structure/, vllm-plugin/tests/).
• Quant-friendly. The hook steers activations at runtime (the calibrated site per lane: post-layer residual where the runtime materialises it, the pre-fold FFN write on the DSV4 lane, 2026-09-04 site correction), so it works on quantized checkpoints (NVFP4 verified on both lanes, bf16→NVFP4 transfer measured) as long as the base architecture is intact. The one exception is REAP-pruned builds: deleting experts changes the shape of the base model, the vector no longer matches the circuit, and the lane is rejected on those grounds (see Lanes).
• A no-patch option for Qwen. The same intervention folded into a rank-1 LoRA we reproduced — it loads in stock vLLM/peft with no hotfix at all and matches the GGUF arm on hardware (both 24/32 refusal32).

Internals write-up: Abliteration without redistributing the model.

Streams and writers — and why GLP is not a LoRA

Two objects get confused often enough that GLP ends up misfiled as a LoRA variant. They are different objects, and the difference is the whole design:

• The residual stream is the running sum a token carries through the network: the embedding output plus everything every layer has added so far. It is an activation — it exists only at runtime, and no single weight matrix sits behind it. A behavioral direction "in the residual stream" means a direction in that accumulated quantity.
• A residual writer is one of the matmuls whose output gets added into the stream: self_attn.o_proj, mlp.down_proj (linear_attn.out_proj on hybrid archs). Each writer has a concrete W you can read and edit.

GLP's native object is a direction d̂ in the stream, and its native operation projects the accumulated stream at runtime: h ← h − α(h·d̂)d̂, applied by a hook after each listed layer. Activation space, α tunable at serve time, architecture-blind, nothing re-baked.

A LoRA can only express a weight delta ΔW = B·A on an existing matmul. "Project the stream" has no single W behind it, so it cannot be written as a LoRA. What can be written is "project each residual writer": at h = Wx the projection is exactly ΔW = −α·d̂(d̂ᵀW) — the Arditi et al. weight-orthogonalization form, which is what captain-vector bake emits as a rank-1 adapter. Every limitation of the baked form follows from that one substitution:

• Writer ≠ stream. Projecting all of a layer's writers at α equals projecting that layer's new contributions at α (linearity) — but a d̂-component already riding the incoming stream (embeddings, an unbaked writer) passes through, where the runtime hook removes it regardless of origin. Close in practice, not bit-identical.
• Dense-only in practice. On MoE the writers are per-expert — hundreds of matrices per layer — so a bake explodes into thousands of rank-1s against quantized weights. Runtime stream steering is architecture-blind; the MoE lanes here (DSV4, GLM, Hy4, K3) exist because of it.
• α is frozen into lora_A — not tunable at serve time, and scaling the adapter double-doses.
• Checkpoint-bound. lora_A carries W; baked against the wrong base revision the adapter is garbage and nothing downstream flags it (bake's pin check fails closed for exactly this reason).

The same constraint runs the other way: llama.cpp control vectors and the steering-LoRAs that circulated before GLP are addition-only (h ← h + scale·d), and addition cannot express "remove the d̂-component of whatever passes through here" — nor bake exactly anywhere (a constant shift is a bias, not a weight delta). Add-vs-project and runtime-vs-baked are two independent axes. GLP occupies the projective, runtime-native quadrant; the baked adapter is its portable shadow for stacks that only speak LoRA.

Deeper: FAQ.md (answers to the questions people actually ask — backdoor auditability, vector maintenance under retraining, effective range, MoE/loop coverage, dosage, termination), docs/spec/GLP.md (the format contract), docs/llama-cpp-compat.md (source-verified mechanics), tools/captain-vector/README.md (bake's troubleshooting/interop role).

The stack

Serving is vLLM (the only runtime with working sm_121a NVFP4 + MTP paths for these checkpoints on GB10), driven by omp as the agent harness/client — we picked it over stock Pi for the benchmaxxed tool loop (hash-anchored edits, LSP/DAP wired in, fast headless mode), and this repo's tests/ suite proves the served model can drive it end to end. Steering is optional per lane and off by default when the steer path is empty:

flowchart LR
    subgraph client[client]
        OMP["omp<br/>agent harness"]
        WIZ["setup.py<br/>wizard + tests/"]
    end
    OMP -->|OpenAI-compatible /v1| EP["vLLM endpoint<br/>:8888, :8078, :8079, :8080 or :8081"]
    WIZ -->|probe / diagnose / boot| EP
    EP --> DSV4["DSV4 lane — TP=2<br/>2x DGX Spark over RoCE<br/>DeepSeek-V4-Flash-0731 NVFP4"]
    EP --> QWEN["Qwen lane — TP=1<br/>single DGX Spark<br/>Qwen3.8-27B NVFP4"]
    EP --> QWENFN["Flash-Next lane — TP=2<br/>2x DGX Spark over RoCE<br/>Qwen3.8-Flash-Next NVFP4"]
    EP --> GLM["GLM-5.3-Flash lane — TP=4<br/>4x DGX Spark over RoCE<br/>GLM-5.3-Flash NVFP4"]
    CV["GLP vector (GGUF)<br/>fail-closed boot hotfix"] -.->|optional| DSV4
    CV2["GLP vector hotfix<br/>or rank-1 LoRA, no patch"] -.->|optional| QWEN
    CV3["GLP-47 vector (GGUF)<br/>fail-closed boot hotfix"] -.->|optional| QWENFN
    CV4["GLP-44 vector (GGUF)<br/>fail-closed boot hotfix"] -.->|optional| GLM
    EP --> GLMXL["GLM-5.3 743B lane — TP=4<br/>4x DGX Spark over RoCE<br/>GLM-5.3 Int4-Int8Mix"]
    CV5["GLP-77 vector (GGUF)<br/>staged fail-closed hotfix"] -.->|optional| GLMXL

The lanes never run at once: each multi-node lane (DSV4 on 2, Flash-Next on 2, the GLM lanes on 4) already holds its Sparks' GPUs, and the Qwen lane is parked until the multi-node stack is down.

Lanes

GLM-5.3-Flash needs 4 nodes — and a patched image. At TP4 the NVFP4 quant is ~50 GiB of weights per 128 GB node, which is what makes the 1M-context KV pool possible. Since 2026-09-29 the production lane runs othexmr's RiNGSiDE (switchless ring NCCL, b12x kernels, RDMA prefill, DFlash2 drafting; TP2 and TP4 profiles) with the GLP-44 steering mounted via its weightless launcher option. A 2-node TP2 config exists (262K context, KV-starved — tonyd2wild's TP2 repo) but stays out of scope for this wizard lane. The stock day-0 vendor image dies five different ways on GB10; the lane requires the sm121-v8 patch stack (radixark/vllm-glm53-flash:sm121-v8 or a local v1→v8 build — details in recipe/glm53/README.md). If your machines.txt lists two nodes, this lane cannot boot until two more are racked and cabled. The 743B lane needs the same four nodes plus tonyd2wild's Int4-Int8Mix weights (377.4 GiB) and stack (image + kernel overlay — recipe/glm53xl/README.md); its first real boot is the anchor test (the hotfix stages fail-closed before GPU time). On x86 SM120 instead of Sparks, brandonmusic's EXL3/B12X build (brandonmusic/GLM-5.3-Flash-tr3-4bpw, custom vLLM fork) takes the same GLP-44 hotfix mechanics — see recipe/glm53/README.md (runtime-validated on 2x RTX PRO 6000: refusal32 1→15/32, cyber32 14→31/32).

DFlash2 spec decode and steering — measured, not assumed (2026-08-30, 4xH100, Prometheus spec_decode counters): GLP-44 steering at α=2.0 costs the drafter no acceptance — structured 81.4% → 87.1% steered, prose 26.1% → 24.4% — because the drafter's aux-capture taps sit before the steering injection point and condition on pre-steering features. The stock tap set [5,14,24,33,42] ships in the drafter's own config.json and is training-matched: a 7-set sweep found every alternative worse, tapping the final target layer poisons the drafter (~6% acceptance), and the tap count is fixed by the checkpoint's fc width. Prose acceptance (~26% Flash, ~23% flagship) is a drafter-capacity limit, not a tap-tuning opportunity — do not retune the taps.

Single-Spark DSV4 (EXL3 3.0bpw + REAP-K216): evaluated and rejected. The full NVFP4 checkpoint (166.9 GB) cannot fit one Spark, so single-node DSV4 exists only as the pruned/quantized artifact — REAP deletes 40 of 256 routed experts per MoE layer on top of the 3.0bpw quant. Benchmarks pass, but the degradation concentrates in rare behaviour by construction, and the control vector would need re-deriving on the pruned circuit. TP=1 DSV4 is a smaller, approximated model; we do not serve it. The steering contract in docs/spec/GLP.md remains lane-independent.

Layout

Real .env files are gitignored — only *.example templates are tracked.

Steering (live stack)

Off unless WEIGHTLESS_STEER_PATH is set. The env vars pass through recipe/anemll/docker-compose.dsv4.yml and the GGUF reader is embedded in the hotfix — nothing to build:

# the general/broad direction; the cyber-derived alternative
# (...-abliterated-cyber-GLP-29-L10-38-a4.gguf) is a one-line swap
WEIGHTLESS_STEER_PATH=/cache/huggingface/DeepSeek-V4-Flash-0731-general-abliterated-cvec-L10-38-a4-keysdir.gguf
WEIGHTLESS_STEER_ALPHA=4.0
WEIGHTLESS_STEER_LAYERS=$(seq -s, 10 38)

Confirm from the boot log — check that layers= reads 29, not 1:

weightless GLP steering active: hook=ffn_out_pre_residual alpha=4.000 ... layers=29 [10, ...]

A previous revision dedented the per-layer assignment out of its loop and steered one layer while reporting 29; coverage dominates this intervention (6 layers leaves 18.0% refusal, 29 leaves 0.0%). The structural tests in scripts/ guard that regression.

alpha is not a strength dial. At 1 the component is removed; past 1 it is reflected, which installs the behaviour rather than removing it. To run weaker, subset the layers and leave alpha alone. The 4.0 above is calibrated for this checkpoint; do not carry it to another model.

Steering on SGLang

The same GLP files and variables work on SGLang through sglang-plugin/; install it next to vllm-plugin/ (which provides the loader) in the SGLang environment:

pip install ./vllm-plugin ./sglang-plugin
WEIGHTLESS_STEER_PATH=<vector.gguf> python -m sglang.launch_server --model-path <model>

Each rank logs weightless GLP steering active (sglang): ...; no line means SGLang serves stock. Details: sglang-plugin/README.md.

Steering artifacts (ours)

Naming convention: GLP-n is a GLP vector touching n layers — GLP-29 below is the DSV4 vector, GLP-49 the Qwen one, GLP-47 the Flash-Next one, GLP-44 the GLM-Flash one, GLP-77 the GLM 743B one. All five lanes' vectors are published under msuiche/ on Hugging Face (gated — fetch with an HF token), spec-conformant per docs/spec/GLP.md and verified against their pinned checkpoint revisions.

• msuiche/DeepSeek-V4-Flash-0731-abliterated-cyber-GLP-29 — DSV4 lane, GLP-29 (GGUF). Cyber-contrast vector: 29 per-layer directions over L10–38, n_embd 4096, α=4.0. The live config currently serves the general-contrast variant from the same repo family — a third-party direction we reformatted, re-measured and repackaged (attribution in the file metadata); swapping is one WEIGHTLESS_STEER_PATH line. Wiring: recipe/anemll/README.md.
• msuiche/Qwen3.8-27B-abliterated-cyber-GLP-49 — Qwen lane, GLP-49 (GGUF + LoRA). The GGUF is canonical: per-layer diff of means, 49 directions over L10–58, n_embd 5120, α=1.0 (α=4 measurably over-refuses on this model). The rank-1 LoRA (mlp.down_proj, L1–63, α=1.0 baked) is the same intervention folded into weights — it loads in stock vLLM/peft/llama.cpp with no hotfix, matches the GGUF on NVFP4 hardware (both 24/32 refusal32, 2026-08-22), but is still unscored on the cyber holdout and is valid only for checkpoint revision 1d4bf0f2ff60 (lora_A embeds W). Wiring: recipe/qwen/README.md.
• msuiche/Qwen3.8-Flash-Next-abliterated-cyber-GLP-47 — Qwen3.8-Flash-Next (qwen4_exp), GLP-47, day-0. Per-layer difference-of-means over the widened hyper-connection stream (10240 = 4×2560), layers 1–47, α=1.0. Derived 2026-08-26 on Modal (8×H100, BF16 reference) from AdvBench-vs-Alpaca. refusal32 3.1% → 81.2% at α=1.0 with benign32 32/32 and capability12 12/12 (higher α over-projects — do not raise it; the shipped default is the measured peak). Direction verified engine-independent: a vLLM-lane capture reproduces it at cos 0.9931. Serving lane: recipe/qwen38fn/ (TP=2 on 2x Spark, day-0 image; the hotfix steers the materialized hyper-connection stream).
• msuiche/GLM-5.3-Flash-abliterated-cyber-GLP-44 — GLM-5.3-Flash (glm5_next), GLP-44, day-0. Per-layer dom over the Sinkhorn hyper-connection stream (16384 = 4×4096), layers 1–44, α=2.0. Derived from the FP8-native canonical checkpoint (snapshot 3f1971b7). refusal32 3.1% → 65.6% at α=2.0, benign/capability clean; α=2.5+ garbles everything — the cliff is abrupt, stay at or below 2.0. Serving lane: recipe/glm53/ (TP=4 on 4x Spark, NVFP4 quant on the sm121-v8 patched day-0 image; the hotfix steers the materialized mHC stream).
• msuiche/GLM-5.3-abliterated-cyber-GLP-77 — GLM-5.3 743B (glm_moe_dsa), GLP-77. Per-layer dom over the plain residual stream (6144 — no hyperconnection widening on this arch), layers 1–77, α=1.0. Derived 2026-08-30 on the NVFP4 deployment (8×H100, vLLM 0.28.0) with the vLLM capture lane; every shipped layer clears the null gate. cyber32 18/32 → 32/32 at α=1.0 with benign/capability clean and a passing null arm; refusal32 moves 1/32 → 12/32 (37.5%; 6/32 answer-audited) — the 753B flagship's refusal is much stickier than Flash's, and α>1 makes it WORSE at full length (do not raise it). Serving lane: recipe/glm53xl/ (TP=4 on 4x Spark, tonyd2wild's Int4-Int8Mix stack; the hotfix steers hidden_states + residual on the deepseek_v2 path).
• msuiche/gemma-3-1b-it-abliterated-GLP-20-L4-25-a0.25 — Gemma-3 1B (gemma3_text), GLP-20. Per-layer dom over the post-layer residual stream (1152), layers 4, 5, 8–25 zero-based — the null gate dropped 6 and 7, so the span genuinely carries gaps — α=0.25. First Google-family vector. Derived 2026-10-11 on Modal (A10G, transformers capture lane) from the byte-identical unsloth/gemma-3-1b-it mirror — the google/ repo is license-gated for our token, so the mirror revision is the pin in glp.base_revision. Stock already complies 29/32 on refusal32; the vector closes the last 3 with benign-holdout 32/32 at every rung, a byte-exact α=0 no-op and zero garbles. No serving lane — apply with glp.py or tools/captain-vector/apply_transformers.py.
• msuiche/gemma-3-4b-it-abliterated-GLP-28-L6-33-a0.25 — Gemma-3 4B (gemma3 multimodal), GLP-28. Same recipe over the post-layer residual stream (2560), layers 6–33 zero-based (contiguous), α=0.25. First multimodal-arch derivation: the checkpoint is Gemma3ForConditionalGeneration and the steered text stack resolves at model.language_model.layers. Stock refusal32 30/32 → 32/32 at α=0.25, benign-holdout 32/32 every arm, α=0 no-op byte-exact, zero garbles. Same unsloth-mirror base pin; apply with glp.py / apply_transformers.py.
• msuiche/gemma-3-12b-it-abliterated-GLP-43-L5-47-a0.25 — Gemma-3 12B (gemma3 multimodal), GLP-43. Post-layer residual stream (3840), layers 5–47 zero-based (contiguous), α=0.25. Stock refusal32 30/32 → 32/32 at α=0.25, benign-holdout 32/32 every arm, α=0 no-op byte-exact, zero garbles. Same unsloth-mirror base pin; apply with glp.py / apply_transformers.py.
• msuiche/gemma-3-27b-it-abliterated-GLP-51-L11-61-a0.25 — Gemma-3 27B (gemma3 multimodal), GLP-51. Post-layer residual stream (5376), layers 11–61 zero-based (contiguous), α=0.25. Stock refusal32 29/32 → 31/32 at the shipped α=0.25 (32/32 from α=0.5 up — the knee rule ships the bottom rung), benign-holdout 32/32 every arm, α=0 no-op byte-exact, zero garbles. Same unsloth-mirror base pin; apply with glp.py / apply_transformers.py.
• msuiche/gemma-4-E2B-it-abliterated-GLP-31-L4-34-a0.5 — Gemma-4 E2B (gemma4 multimodal), GLP-31. Post-layer residual stream (1536), layers 4–34 zero-based (contiguous), α=0.5. Derived 2026-10-11 on Modal (A10G, transformers 5.19.0 capture lane) from canonical google/gemma-4-E2B-it — unlike gemma-3, the gemma-4 google/ repos serve weights to our token, so no mirror; the google/ revision sha is the pin in glp.base_revision. Stock refuses hard — 4/32 comply on refusal32 — and the ladder is monotone (28 → 31 → 32), so the vector does real unlocking: 31/32 at the shipped α=0.5 with benign-holdout 32/32 at every rung, a byte-exact α=0 no-op and zero garbles. No serving lane — apply with glp.py or tools/captain-vector/apply_transformers.py.
• msuiche/gemma-4-E4B-it-abliterated-GLP-33-L5-41-a1 — Gemma-4 E4B (gemma4 multimodal), GLP-33. Post-layer residual stream (2560), layers 5, 10–41 zero-based — the null gate dropped 6–9, so the span genuinely carries a gap — α=1.0. The family's hardest refuser: stock refuses 31/32 on refusal32 (the lone "comply" is a scorer artifact) and the ladder is the steepest (15 → 25 → 32), so the knee rule climbs past 0.5 and ships full α — 32/32 steered with benign-holdout 32/32 at every rung, α=0 no-op byte-exact, zero garbles. Same canonical google/ base pin; apply with glp.py / apply_transformers.py.
• msuiche/gemma-4-12B-it-abliterated-GLP-33-L15-47-a0.5 — Gemma-4 12B (gemma4_unified), GLP-33. Post-layer residual stream (3840), layers 15–47 zero-based (contiguous), α=0.5. The unified arch: vision/audio embedders into the shared text stack, no towers (the four siblings are plain gemma4 with real towers + PLE; both nest the steered stack at model.language_model.layers). Stock already complies 32/32 on refusal32, so this ladder is the no-regression story — and it is non-monotone: α=0.25 dips to 29/32, α=0.5 and 1.0 hold 32/32 with benign-holdout 32/32 every arm, α=0 no-op byte-exact, zero garbles. Same canonical google/ base pin; apply with glp.py / apply_transformers.py.
• msuiche/gemma-4-26B-A4B-it-abliterated-GLP-27-L3-29-a0.5 — Gemma-4 26B-A4B (gemma4 multimodal, MoE), GLP-27. Post-layer residual stream (2816), layers 3–29 zero-based (contiguous), α=0.5. The first MoE derivation in the gemma line: 128 experts top-8 summed with a dense MLP per hybrid layer (~26B total / ~4B active) — the hook sits on the decoder-layer output downstream of router+experts, and direction quality matches the dense 12B (span min cos 0.575 vs 0.559). Stock complies 32/32; the ladder dips to 25/32 at α=0.25 and recovers to 31/32 at the shipped α=0.5 (α=1.0 flips two benign-holdout prompts and fails that gate), benign-holdout 32/32 at the shipped rung, α=0 no-op byte-exact, zero garbles. Same canonical google/ base pin; apply with glp.py / apply_transformers.py.
• msuiche/gemma-4-31B-it-abliterated-GLP-54-L6-59-a0.5 — Gemma-4 31B (gemma4 multimodal), GLP-54. Post-layer residual stream (5376), layers 6–59 zero-based (contiguous), α=0.5. The dense flagship (60L, vision/audio towers + PLE). Stock complies 32/32 on refusal32; the shallowest non-monotone dip of the family — α=0.25 gives 30/32, α=0.5 holds 32/32, α=1.0 re-engages one refusal — benign-holdout 32/32 at every rung, α=0 no-op byte-exact, zero garbles. Same canonical google/ base pin; apply with glp.py / apply_transformers.py.

Roadmap

• More model lanes. The GLP spec and the fail-closed hotfix pattern are model-agnostic — DSV4 and Qwen3.8 are the starting lanes, not the scope. The target is most good open models, one recipe/ lane and one published GLP vector each. Day-0 vectors for Qwen3.8-Flash-Next (GLP-47) and GLM-5.3-Flash (GLP-44) shipped 2026-08-27, GLM-5.3 743B (GLP-77) 2026-08-30; their serving lanes are recipe/qwen38fn/ (TP=2), recipe/glm53/ (TP=4, needs 4 nodes) and recipe/glm53xl/ (TP=4, 4 nodes + the Int4-Int8Mix stack).
• k=7/greedy draft A/B (upstream issue #84): now one env line (DRAFT_SAMPLE_METHOD=greedy MTP_NUM_TOKENS=7) after the 2026-08-21 upstream merge.
• Qwen LoRA cyber-holdout score (run L): both Qwen steering arms match on refusal32; the LoRA arm still needs its cyber-holdout number before it can replace GGUF as the default. Lives in the eval pipeline, not this repo.

References & credits

• Abliteration without redistributing the model — our write-up of the steering internals (the method this repo packages)
• deepseek-ai/DeepSeek-V4-Flash-0731 — the model (MIT)
• tonyd2wild/DeepSeek-v4-Flash-0731-DSpark-1M-NVFP4-KV-2x-DGX-Spark — the original 2x DSpark NVFP4-KV recipe; its issue #18 is the spec-decode corruption we root-caused before migrating
• MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark — the 2-node recipe we run (our state vendored in recipe/anemll/)
• MiaAI-Lab/DeepSeek-v4-Flash-One-DGX-Spark — single-Spark EXL3/REAP sibling (lane rejected — REAP pruning degrades the tail)
• Anemll's GX10 image (ghcr.io/anemll/dspark-vllm-gx10) — the vLLM 0.25.2 runtime
• local-inference-lab/b12x — the sparkinfer/b12x kernel stack inside the image
• 0xSero/deepseek-v4-flash-0731-spark — the single-Spark EXL3/REAP build (evaluated, rejected — see Lanes)
• Loke-60000/deepseek-v4-flash-0731-spark-vision — community Spark vision serving fix (surveyed, not adopted)
• drowzeys/keys-DeepSeekV4-Flash-GA-0731-Dspark-Abliterated-32-32 — Keys' abliterated DSV4 checkpoint, prior art; the benchmarked general/broad direction (keysdir artifact) was recovered from these weights by SVD
• orcarouter/Qwen3.8-27B-Uncensored-FP8 — the published Qwen3.8 abliteration we benchmark against; its direction, SVD-recovered and run through our harness, scores 90.6/100/100
• drowzeys/keys-vLLm.0.27-Qwen3.8-NVFP4-MTP3-Single-DGX-Spark — the Qwen TP=1 lane's serving recipe
• tonyd2wild/GLM-5.3-Flash-NVFP4-1M-KV-4x-DGX-Spark + the TP2 companion — the GLM-5.3 lane's serving stack (sm121-v8 image patches, boot ritual, KV ladder), deployed and debugged by Knox (Claude) for @tonyd2wild; quant by LibertAIDAI; gmu reference config by barrydeen
• tonyd2wild/GLM-5.3-Int4-Int8Mix-TP4-4x-DGX-Spark + the GLM-5.3 DGX Spark Cookbook — the GLM-5.3 743B lane's serving stack (Int4-Int8Mix quant, sm12x kernel overlay, boot ritual); quant recipe from QuantTrio, weights at 2wild4tv/GLM-5.3-Int4-Int8Mix
• othexmr/GLM-5.3-Flash-NVFP4-2x-4x-DGX-Sparks-RiNGSiDE — the GLM-5.3-Flash lane's current production serving stack (NVFP4 recipe, switchless ring NCCL, b12x kernels, RDMA prefill, DFlash2 drafting; TP2 and TP4 profiles). Our 4-node ring runs it with GLP-44 steering mounted on top (the weightless option in their launcher), live since 2026-09-29.

Fallback: the retired v027 stack

Parked, not deleted — the v0.27 git patch and its test remain under patches/. Essentials if you ever revive it:

• Maintenance home is the dspark-steering-v027 branch of msuiche/vllm, so upstream bumps get a real 3-way merge. Regenerate with: git -C ../vllm diff v0.27.0..dspark-steering-v027 -- vllm/models/deepseek_v4/nvidia/model.py > patches/0001-dspark-projective-steering.patch
• Apply without an image build: run the stock v027 image and bind-mount the patched model.py over the original. Resolve the in-image path first (it is /opt/vllm-src/vllm, a source install — bind-mounting a path that does not exist silently creates it and yields an inert patch; the layers=29 boot line is the confirmation steering is live).
• Base image: ghcr.io/bjk110/vllm-spark:v027-ngc2607-dsv4-0731-dspark-k7-256k-production (registry is rate-limited for anonymous pulls — docker login ghcr.io first). It has no gguf module: build recipe/Dockerfile.gguf-dep and point DSPARK_VLLM_IMAGE at it. Never load a .pt instead — that goes through torch.load and bypasses every spec check in the reader.
• Settled v027 config: context 1,032,192 (1,048,576 collapses prefill to 110 tok/s), KV fp8_ds_mla, spec decode dspark k=5, max-num-seqs 6. Measurements: BENCHMARK.md Runs 001–005.