0
stars
12
commits
2
linked in READMEs
Jul 2, 2026
updated
activation-verbalization
gpt-oss
gpt-oss-20b
hidden-states
interpretability
mechanistic-interpretability
nla
vector-quantization
README
srt-nla-gptoss20b-artifacts β corpus-free NLA data for gpt-oss-20b
π¬ Live demo: RiverRider/srt-nla-gptoss20b-trace β the VQ codebook below powering a full inputβoutput trace of
openai/gpt-oss-20b.
Everything needed to reproduce, evaluate, or extend the NLA (natural-language
activation) pipeline on a frozen openai/gpt-oss-20b. All data is
self-generated by the backbone (unconditional sampling from BOS) β no
external corpus.
Files
| File | Size | What it is |
|---|---|---|
state_codebook_vq.pt | 96 MB | VQ state codebook: 4096 k-means++ centroids over 200K centered hidden states (L6/12/18/24), each with a canonical text (the exact generating prefix). Load with srt.nla.StateIndex.load. encode(v) β integer "magic number"; decode(v) β retrieval verbalization. This is the recommended decoder on this backbone β it beats the trained AV at any practical best-of-K (see kcurve.json). |
trace_pairs.jsonl | 57 MB | 200K records {target_idx, gold_ids, seq_idx, pos, layer, n_tokens} β one per (sequence, position, layer), balanced ~50K per layer. gold_ids is the exact token prefix that produced the state (causal-prefix property β exact at every layer). |
trace_pairs.jsonl.targets.pt | 2.3 GB | {"targets": (200000, 2880) float32} β the hidden states, aligned to target_idx. |
anchors_L18.json | β | Calibration anchors at L18 (centered fve, M=100, pool=2000): replay 0.999, NN retrieval 0.744, random floor 0.500, βΞΌβ = 4438. |
kcurve.json | β | Best-of-K curve of the released AV (K=1β¦64 β 0.541β¦0.642 centered). Documents that the AV stays below the NN baseline. |
Provenance
sample_targets.py --layers 6,12,18,24 --num-sequences 10000 --seq-len 64: frozen gpt-oss-20b generates 10K sequences from bare BOS; hidden states saved at every position Γ 4 layers with exact token ids.build_trace_pairs.py --layers all --position-stride 4 --max-pairs 200000: flatten to (v, prefix) pairs. Because attention is causal, position t's hidden equals the last-token hidden of prefixids[:t+1]β every prefix is an exact gold verbalization at every layer simultaneously.build_state_codebook.py --mode vq --k 4096: fit k-means++ on centered states; canonical text per code = highest-fidelity member prefix.
Sampling note: bare-BOS generations from this harmony-tuned model are coherent but skew toward assistant/reasoning register; the codebook inherits that distribution.
Quick start
import torch, json
from huggingface_hub import hf_hub_download
from srt.nla import StateIndex # pip install "srt-adapter @ git+https://github.com/space-bacon/SRT.git"
repo = "RiverRider/srt-nla-gptoss20b-artifacts"
si = StateIndex.load(hf_hub_download(repo, "state_codebook_vq.pt", repo_type="dataset"))
targets = torch.load(hf_hub_download(repo, "trace_pairs.jsonl.targets.pt",
repo_type="dataset"), weights_only=False)["targets"]
v = targets[123]
print("magic number:", si.encode(v))
print("retrieval decode:", si.decode(v))
Siblings
RiverRider/srt-adapter-gptoss20bβ SRT read-out adapter (regime ECE 0.0009 / AUROC 0.974)RiverRider/srt-nla-av-gptoss20bβ the trained AV (read its card's honest K-curve before using)
Contributors
RiverRider
12 commits
Similar repos
Linked in READMEs
0
stars
12
commits
2
linked in READMEs
Jul 2, 2026
updated
activation-verbalization
gpt-oss
gpt-oss-20b
hidden-states
interpretability
mechanistic-interpretability
nla
vector-quantization
README
srt-nla-gptoss20b-artifacts β corpus-free NLA data for gpt-oss-20b
π¬ Live demo: RiverRider/srt-nla-gptoss20b-trace β the VQ codebook below powering a full inputβoutput trace of
openai/gpt-oss-20b.
Everything needed to reproduce, evaluate, or extend the NLA (natural-language
activation) pipeline on a frozen openai/gpt-oss-20b. All data is
self-generated by the backbone (unconditional sampling from BOS) β no
external corpus.
Files
| File | Size | What it is |
|---|---|---|
state_codebook_vq.pt | 96 MB | VQ state codebook: 4096 k-means++ centroids over 200K centered hidden states (L6/12/18/24), each with a canonical text (the exact generating prefix). Load with srt.nla.StateIndex.load. encode(v) β integer "magic number"; decode(v) β retrieval verbalization. This is the recommended decoder on this backbone β it beats the trained AV at any practical best-of-K (see kcurve.json). |
trace_pairs.jsonl | 57 MB | 200K records {target_idx, gold_ids, seq_idx, pos, layer, n_tokens} β one per (sequence, position, layer), balanced ~50K per layer. gold_ids is the exact token prefix that produced the state (causal-prefix property β exact at every layer). |
trace_pairs.jsonl.targets.pt | 2.3 GB | {"targets": (200000, 2880) float32} β the hidden states, aligned to target_idx. |
anchors_L18.json | β | Calibration anchors at L18 (centered fve, M=100, pool=2000): replay 0.999, NN retrieval 0.744, random floor 0.500, βΞΌβ = 4438. |
kcurve.json | β | Best-of-K curve of the released AV (K=1β¦64 β 0.541β¦0.642 centered). Documents that the AV stays below the NN baseline. |
Provenance
sample_targets.py --layers 6,12,18,24 --num-sequences 10000 --seq-len 64: frozen gpt-oss-20b generates 10K sequences from bare BOS; hidden states saved at every position Γ 4 layers with exact token ids.build_trace_pairs.py --layers all --position-stride 4 --max-pairs 200000: flatten to (v, prefix) pairs. Because attention is causal, position t's hidden equals the last-token hidden of prefixids[:t+1]β every prefix is an exact gold verbalization at every layer simultaneously.build_state_codebook.py --mode vq --k 4096: fit k-means++ on centered states; canonical text per code = highest-fidelity member prefix.
Sampling note: bare-BOS generations from this harmony-tuned model are coherent but skew toward assistant/reasoning register; the codebook inherits that distribution.
Quick start
import torch, json
from huggingface_hub import hf_hub_download
from srt.nla import StateIndex # pip install "srt-adapter @ git+https://github.com/space-bacon/SRT.git"
repo = "RiverRider/srt-nla-gptoss20b-artifacts"
si = StateIndex.load(hf_hub_download(repo, "state_codebook_vq.pt", repo_type="dataset"))
targets = torch.load(hf_hub_download(repo, "trace_pairs.jsonl.targets.pt",
repo_type="dataset"), weights_only=False)["targets"]
v = targets[123]
print("magic number:", si.encode(v))
print("retrieval decode:", si.decode(v))
Siblings
RiverRider/srt-adapter-gptoss20bβ SRT read-out adapter (regime ECE 0.0009 / AUROC 0.974)RiverRider/srt-nla-av-gptoss20bβ the trained AV (read its card's honest K-curve before using)
Similar repos
Linked in READMEs
Contributors
RiverRider
12 commits