lamalunderscore/manipulatte

Extensible library for different manipulations of the self-attention mechanism.

stars

commits

Python

primary language

Nov 5, 2025

updated

README

ManipuLatte :coffee:

This library implements all techniques to manipulate attention mechanisms, including attention-head sparsification and weight manipulations, that were used in Some Attention is All You Need for Retrieval

For questions, please reach out to the corresponding author in the mentioned paper.

Contributors

lamalunderscore

9 commits

Similar repos

meta-pytorch/attention-gym

Helpful tools and examples for working with flex-attention

1,242

thiomajid/distil_xlstm

Learning Attention Mechanisms through Recurrent Structures

sidijju/NestedTensorsTransformers

Implemented Nested Tensors for Transformers

JackMcKechnie/LSR4SAS

This repository accompanies the ICTIR 2026 submission titled "Learned Sparse Retrieval for…

extropic-ai/sparse-transformers

Sparse Transformers in JAX

goombalab/Gather-and-Aggregate

Experiments Notebook of "Understanding the Skill Gap in Recurrent Language Models: The Role of the…

mit-han-lab/Block-Sparse-Attention

A sparse attention kernel supporting mix sparse patterns

553

msakarvadia/AttentionLens

Interpretating the latent space representations of attention head outputs for LLMs

Languages

Python

100.0%

README

ManipuLatte :coffee:

For questions, please reach out to the corresponding author in the mentioned paper.