Cluster 653786

5 repos

Python · 4

Jupyter Notebook · 1

attention-mechanism ·165

attention-sink ·165

language-model ·165

large-language-models ·165

Related clusters

sail-sg/Attention-Sink

[ICLR 2025] When Attention Sink Emerges in Language Models: An Empirical View (Spotlight)

Python

165

8 commits

msakarvadia/AttentionLens

Interpretating the latent space representations of attention head outputs for LLMs

Python

137 commits

Maximo-Rulli/dlms-sinks

Official repository of: Attention Sinks in Diffusion Language Models

Python

2 commits

Dogacel/Attention-Drift

Code for the paper *Attention Drift: What Speculative Decoding Models Learn*.

Jupyter Notebook

2 commits

hpcgroup/loki

Algorithms for approximate attention in LLMs

Python

67 commits