66RING/tiny-flash-attention

flash attention tutorial written in python, triton, cuda, cutlass

Cuda

535

33 commits

updated Jan 20, 2026

See the code

README

Tiny FlashAttention

WIP

A tiny flash attention implement in python, rust, cuda and c for learning purpose.

cutlass cute flash attention in action

my env: cutlass v3.4, torch 1.14, cuda 12.4

Similar repos

shizhengLi/cuda-triton-learning

CUDA & Triton Learning Project: Flash Attention 实现探索

43

kingbri1/flash-attention

Fast and memory-efficient exact attention

944

bdashore3/flash-attention

Fast and memory-efficient exact attention

944

HazyResearch/flash-attention

Fast and memory-efficient exact attention

25,008

Dao-AILab/flash-attention

Fast and memory-efficient exact attention

25,008

evintunador/triton_docs_tutorials

making the official triton tutorials actually comprehensible

208

mjun0812/flash-attention-prebuild-wheels

Provide with pre-build flash-attention 2 and 3 package wheels on Linux and Windows using GitHub…

1,749

lucidrains/ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

545

Contributors

66RING

32 commits

KevinZeng08

1 commits

Languages

Cuda

46.3%

C++

30.6%

Python

21.3%