EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference

The Official PyTorch implementation of EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference

We are still updating the code and its instructions in the coming weeks. Please watch (and leave a star if you like our work) this space for continued update.

Setup

conda env create -f environment.yml
conda activate eigenlora

Usage

In order to find the EigenLoRAs Principal Components, start with a few pretrained LoRA adapters for the same base model.

Refer to NLU/ and Diffusion/ for usage examples.

Citation

If you find EigenLoRAx useful, please consider giving a star and citation:

@misc{kaushik2025eigenloraxrecyclingadaptersprincipal,
      title={EigenLoRAx: Recycling Adapters to Find Principal Subspaces for Resource-Efficient Adaptation and Inference}, 
      author={Prakhar Kaushik and Ankit Vaidya and Shravan Chaudhari and Alan Yuille},
      year={2025},
      eprint={2502.04700},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2502.04700}, 
}