Distil-xLSTM: Learning Attention Mechanisms through Recurrent Structures

This repository contains the code for the distillation approach described in our Distil-xLSTM paper.

Citation

If you use this codebase or otherwise found our work valuable, please cite:

@misc{thiombiano2025distilxlstmlearningattentionmechanisms,
      title={Distil-xLSTM: Learning Attention Mechanisms through Recurrent Structures}, 
      author={Abdoul Majid O. Thiombiano and Brahim Hnich and Ali Ben Mrad and Mohamed Wiem Mkaouer},
      year={2025},
      eprint={2503.18565},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2503.18565}, 
}