KVCachePapers

Papers

1. Fast Transformer Decoding: One Write-Head is All You Need

   Noam Shazeer [pdf], 2019.11

2. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

   Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré [pdf], 2022.5

3. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

   Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai [pdf], 2023.5

4. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

   DeepSeek-AI [pdf], 2024.5

5. Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

   William Brandon, Mayank Mishra, Aniruddha Nrusimha, Rameswar Panda, Jonathan Ragan Kelly [pdf], 2024.5

6. Layer-Condensed KV Cache for Efficient Inference of Large Language Models

   Haoyi Wu, Kewei Tu [pdf], 2024.5

7. You Only Cache Once: Decoder-Decoder Architectures for Language Models

   Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, Furu Wei [pdf], 2024.5