deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B — Pivotal Token Search

Pivotal reasoning events for deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, at three representational scales in one file, produced with PTS.

latent meta-token / workspace event   (Latent PTS)   ← J-lens readout
        ↓
emitted pivotal token                 (Token PTS)    ← Phi-4 PTS
        ↓
sentence-level thought anchor          (Sentence PTS) ← Thought Anchors
        ↓
success / failure probability shift

All three are CausalReasoningEvent records — one schema, scored by the same principle. See the schema doc.

Contents

Both readout methods are included so the control below is reproducible from this file. Latent events store no context (recoverable from metadata.source_event_id); only categorized meta-tokens are kept — the ~92% uncategorized score-floor fragments carry no cross-scale signal.

What we found

This dataset exists to test one claim:

Many emitted pivotal tokens and thought-anchor sentences are preceded by latent verbalizable meta-tokens in the model's workspace.

The J-lens beats the logit-lens control — the direction a workspace account predicts.

• Meta-tokens category-match the events they precede 3.6× above a shuffled baseline (J-lens observed 69% vs 19% chance, n=239), leading by ~3.6 tokens.
• The discriminating test comes out positive here. The logit lens reads what an activation would say now; the J-lens reads what it pushes the model to say later — only the latter is evidence of a workspace. The J-lens lift (3.64×) is above the logit-lens lift (3.29×), with n=239 — a clearer separation than on the 0.6B model, where the two overlapped. The effect strengthening with model size is what the workspace hypothesis predicts.

A real limitation:

• No complete latent→token→sentence chains formed (0 of them). This dataset has only 110 sentence events against 4,734 token events, so three-scale chains are too rare to measure. The cross-scale result above is latent→token only.

Still observational. The J-lens beating the logit-lens is consistent with a workspace, but nothing here is an intervention. Establishing that a meta-token causes a downstream event requires steering or ablating it and re-measuring success, which this dataset does not do.

Caveats

• Latent score is a readout probability, not a probability delta. prob_delta/is_positive are null on every latent event, by construction. Never compare latent scores with emitted deltas.
• Latent events are observational. No intervention was run; nothing here shows a meta-token caused an emitted event.
• Independent reimplementation. The J-lens is written from the equations in Anthropic's workspace paper (no reference code was released) and is not validated against their results. "Meta-token" is our term, not theirs.
• Model-specific. These events do not transfer to other models.

Reproduce

pip install pts
pts fit-jlens --model deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B --output-path ./jlens
pts enrich --input-path <pivotal tokens> --model deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
           --jlens-path ./jlens --readout-method jlens --with-latent --shuffle-control

Citation

If you use this dataset or the PTS tool in your research, please cite:

@software{pts,
  title = {PTS: Pivotal Token Search},
  author = {Asankhaya Sharma},
  year = {2025},
  publisher = {GitHub},
  url = {https://github.com/codelion/pts}
}