KoronA_finetuning

KornA is a repository for fine-tuning and evaluating transformer-based models with Kronecker-based adapters (KronA). It supports tasks such as GLUE benchmarks and mathematical reasoning datasets, leveraging efficient and scalable adapter modules to enhance model performance.

Features

• Kronecker-Based Adapters: Implements KronA, KronAB, and KronABRes modules for efficient parameter adaptation.
• GLUE Benchmark Support: Fine-tune models on GLUE tasks such as CoLA, SST-2, MRPC, QQP, MNLI, QNLI, RTE, and STS-B.
• Mathematical Reasoning: Fine-tune models on mathematical reasoning datasets like MetaMathQA and GSM8K.
• Customizable Training: Flexible configurations for learning rate, batch size, epochs, and adapter types.
• Integration with Hugging Face Transformers: Leverages Hugging Face's transformers library for model loading and tokenization.

---