Qwen/Qwen3-0.6B — Pivotal Token Search

Pivotal reasoning events for Qwen/Qwen3-0.6B, at three representational scales in one file, produced with PTS.

latent meta-token / workspace event   (Latent PTS)   ← J-lens readout
        ↓
emitted pivotal token                 (Token PTS)    ← Phi-4 PTS
        ↓
sentence-level thought anchor          (Sentence PTS) ← Thought Anchors
        ↓
success / failure probability shift

All three are CausalReasoningEvent records — one schema, scored by the same principle. See the schema doc.

Contents

Both readout methods are included so the control below is reproducible from this file. Latent events store no context (recoverable from metadata.source_event_id); only categorized meta-tokens are kept — the ~92% uncategorized score-floor fragments carry no cross-scale signal.

What we found

This dataset exists to test one claim:

Many emitted pivotal tokens and thought-anchor sentences are preceded by latent verbalizable meta-tokens in the model's workspace.

Robust (large sample):

• Cross-scale chains are coherent. 65% of latent→token→sentence chains share a category across all three scales. The most common is calculate → calculate → "…divide 8 by 20…" (×83).
• Meta-tokens category-match the events they precede ~2.6× above a shuffled baseline (observed 43% vs 16% chance, n=123), leading by ~4 tokens.

Not conclusive at this scale (n≈100):

• Does the J-lens beat the logit-lens? This is the discriminating test: the logit lens reads what an activation would say now, the J-lens what it pushes the model to say later. Only the latter is evidence of a workspace. Here the J-lens trends higher (2.76× vs logit-lens 2.41×, bootstrapped), but the ranges overlap at this sample size. We do not claim the J-lens wins on this model. See the 1.5B model for a better-powered comparison.

Caveats

• Latent score is a readout probability, not a probability delta. prob_delta/is_positive are null on every latent event, by construction. Never compare latent scores with emitted deltas.
• Latent events are observational. No intervention was run; nothing here shows a meta-token caused an emitted event.
• Independent reimplementation. The J-lens is written from the equations in Anthropic's workspace paper (no reference code was released) and is not validated against their results. "Meta-token" is our term, not theirs.
• Model-specific. These events do not transfer to other models.

Reproduce

pip install pts
pts fit-jlens --model Qwen/Qwen3-0.6B --output-path ./jlens
pts enrich --input-path <pivotal tokens> --model Qwen/Qwen3-0.6B \
           --jlens-path ./jlens --readout-method jlens --with-latent --shuffle-control

Citation

If you use this dataset or the PTS tool in your research, please cite:

@software{pts,
  title = {PTS: Pivotal Token Search},
  author = {Asankhaya Sharma},
  year = {2025},
  publisher = {GitHub},
  url = {https://github.com/codelion/pts}
}