Fed-PLoRA

Heterogeneous Federated Fine-Tuning with Parallel One-Rank Adaptation, ICLR 2026

This repository contains an official reference implementation of Fed-PLoRA.

Install dependencies

uv venv .venv
source .venv/bin/activate
uv pip install -r requirements.txt

Run

python main.py

🔗 Our Federated Fine-Tuning Series

This repository is part of our broader research effort on efficient and heterogeneous federated fine-tuning of foundation models. Our recent works explore federated fine-tuning from multiple complementary perspectives, including rank-wise adaptation, layer-wise LoRA allocation, and cross-domain benchmarking.

Rank-wise Federated Fine-Tuning

• Heterogeneous Federated Fine-Tuning with Parallel One-Rank Adaptation (this repo)
  ICLR 2026
  We propose a rank-wise federated fine-tuning method that mitigates initialization noise and aggregation noise under heterogeneous client settings.
  [Paper] [Code]

Layer-wise Federated Fine-Tuning

• Fed-HeLLo: Efficient Federated Foundation Model Fine-Tuning with Heterogeneous LoRA Allocation
  IEEE TNNLS 2025
  We study heterogeneous layer-wise LoRA allocation for efficient federated fine-tuning of foundation models.
  [Paper]

• Fed-pilot: Optimizing LoRA Allocation for Efficient Federated Fine-Tuning with Heterogeneous Clients
  ArXiv 2024
  We introduce an optimization framework for allocating LoRA modules across layers under heterogeneous client resources.
  [Paper]

Benchmark

• FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models
  NeurIPS 2025, Datasets and Benchmarks Track
  In collaboration with FlowerLabs, we build a cross-domain benchmark for evaluating federated fine-tuning of large language models.
  [Paper] [Project]