Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks [arXiv]

Yang Liu^†,*,1, Kejia Zhang^*,1, Bingjie Yan¹, Tianyuan Zou², Jianqing Zhang^2,3, Zixuan Gu^2,5, Xiangsen Chen¹, Jianbing Ding⁴, Xidong Wang⁴, Jingyi Li⁴, Xiaozhou Ye⁴, Ye Ouyang⁴, Qiang Yang¹, Ya-Qin Zhang²

¹The Hong Kong Polytechnic University, ²Institute for AI Industry Research, Tsinghua University, ³Shanghai Jiao Tong University, ⁴AsiaInfo Technologies, ⁵School of Software, Tsinghua University

* Equal Contribution. † Corresponding Author (yang-veronica.liu@polyu.edu.hk).

[!TIP] If this repository or our survey paper helps your work, please consider citing it.

We also welcome pull requests and issues for missing papers, metadata fixes, taxonomy improvements, and content clarification.

📝 Citation

@article{liu2025towards,
  title={Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks},
  author={Liu, Yang and Zhang, Kejia and Yan, Bingjie and Zou, Tianyuan and Zhang, Jianqing and Gu, Zixuan and Chen, Xiangsen and Ding, Jianbing and Wang, Xidong and Li, Jingyi and Ye, Xiaozhou and Ouyang, Ye and Yang, Qiang and Zhang, Ya-Qin},
  journal={arXiv preprint arXiv:2504.17421},
  year={2026}
}

🔥 News

• [2026-06-28] Released the survey paper and this public repository.

🎯 Motivation

Domain tasks often involve private data, proprietary models, and limited local resources, which make centralized adaptation of large models impractical. LM-SM collaboration addresses this setting by combining the broad generalization of large models with the efficiency and locality of small models. This repository organizes representative methods by transfer direction and inference-time collaboration, with attention to privacy, model security, and resource constraints.

Taxonomy

---

📚 Contents

Badge legend

• arXiv papers
• conference/journal papers
• code repository stars
• method categories

Knowledge Transfer from LMs to SMs

Distillation-based Transfer

Title & Authors	Method	Category	Links
FedMD: Heterogenous Federated Learning via Model Distillation Daliang Li, Junpu Wang	FedMD		Paper Code
Ensemble Attention Distillation for Privacy-Preserving Federated Learning Xuan Gong, Abhishek Sharma, Srikrishna Karanam, et al.	PPFL		Paper
Zero-Shot Knowledge Distillation from a Decision-Based Black-Box Model Zi Wang	ZSKD		Paper Code
Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation Xuan Gong, Abhishek Sharma, Srikrishna Karanam, et al.	FedKD		Paper
Towards Data-Free Model Stealing in a Hard Label Setting Sunandini Sanyal, Sravanti Addepalli, R Venkatesh Babu	Hard-label Stealing		Paper Code
Offsite-Tuning: Transfer Learning without Full Model Guangxuan Xiao, Ji Lin, Song Han	Offsite-Tuning		Paper Code
CRaSh: Clustering, Removing, and Sharing Enhance Fine-Tuning without Full Large Language Model Kai-yan Zhang, Ning Ding, Biqing Qi, et al.	CRaSh		Paper
IDEAL: Query-Efficient Data-Free Learning from Black-Box Models Jie Zhang, et al.	IDEAL		Paper Code
Data Shunt: Collaboration of Small and Large Models for Lower Costs and Better Performance Dong Chen, Yueting Zhuang, Shuo Zhang, et al.	EC-KD		Paper Code
FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning Jianqing Zhang, Yang Liu, Yang Hua, et al.	FedTGP		Paper Code
An Upload-Efficient Scheme for Transferring Knowledge from a Server-Side Pre-Trained Generator to Clients in Heterogeneous Federated Learning Jianqing Zhang, Yang Liu, Yang Hua, et al.	FedKTL		Paper Code
AMD: Automatic Multi-Step Distillation of Large-Scale Vision Models Cheng Han, Qifan Wang, Sohail A Dianat, et al.	AMD		Paper
Orchestration of Emulator Assisted 6G Mobile Edge Tuning for AI Foundation Models: A Multi-Agent Deep Reinforcement Learning Approach Wenhan Yu, Terence Jie Chua, Jun Zhao	Emulator-Adapter		Paper
FedPFT: Federated Proxy Fine-Tuning of Foundation Models Zhaopeng Peng, Xiaoliang Fan, Yufan Chen, et al.	FedPFT		Paper
FedGMKD: An Efficient Prototype Federated Learning Framework Through Knowledge Distillation and Discrepancy-Aware Aggregation Jian-qiao Zhang, Cai-feng Shan, Jungong Han	FedGMKD		Paper
ScaleOT: Privacy-Utility-Scalable Offsite-Tuning with Dynamic LayerReplace and Selective Rank Compression Kai Yao, zhaorui Tan, Tiandi Ye, et al.	ScaleOT		Paper
GradOT: Training-Free Gradient-Preserving Offsite-Tuning for Large Language Models Kai Yao, Zhaorui Tan, Penglei Gao, et al.	GradOT		Paper
Synthetic Data Distillation Enables the Extraction of Clinical Information at Scale Elizabeth Geena Woo, Michael C Burkhart, Emily Alsentzer, et al.	SD-CL		Paper

(back to top ↑)

Generation-based Transfer

Title & Authors	Method	Category	Links
ZeroGen: Efficient Zero-Shot Learning via Dataset Generation Jiacheng Ye, Jiahui Gao, Qintong Li, et al.	ZeroGen		Paper Code
ProGen: Progressive Zero-Shot Dataset Generation via in-Context Feedback Jiacheng Ye, Jiahui Gao, Zhiyong Wu, et al.	ProGen		Paper Code
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding Yu Meng, Jiaxin Huang, Yu Zhang, et al.	SuperGen		Paper Code
Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfer Yongheng Deng, Ziqing Qiao, Ju Ren, et al.	CrossLM		Paper
Retrieval-Based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression Jiduan Liu, Jiahao Liu, Qifan Wang, et al.	RetriKT		Paper
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning Jiahui Gao, Renjie Pi, Lin Yong, et al.	SunGen		Paper Code
No More Hard Prompts: SoftSRV Prompting for Synthetic Data Generation Giulia DeSalvo, Jean-Fracois Kagy, Lazaros Karydas, et al.	SoftSRV		Paper
Evolving Knowledge Distillation with Large Language Models and Active Learning Chengyuan Liu, Fubang Zhao, Kun Kuang, et al.	EKD		Paper
Prompt Public Large Language Models to Synthesize Data for Private On-device Applications Shanshan Wu, Zheng Xu, Yanxiang Zhang, et al.	PubSynth		Paper
Fusegen: PLM Fusion for Data-Generation Based Zero-Shot Learning Tianyuan Zou, Yang Liu, Peng Li, et al.	Fusegen		Paper Code
Zero-Shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detection Gaetan Latouche, Marc-André Carbonneau, Ben Swanson	CLTGen		Paper
GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation Mohsen Gholami, Mohammad Akbari, Tianxi Hu, et al.	GOLD		Paper Code
Distilling On-device Language Models for Robot Planning with Minimal Human Intervention Zachary Ravichandran, Ignacio Hounie, Fernando Cladera, et al.	PRISM		Paper Code
Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment Wenrui Cai, Chengyu Wang, Junbing Yan, et al.	CRV		Paper
Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning Xiaochuan Li, Zichun Yu, Chenyan Xiong	Montessori-Instruct		Paper Code
Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data Juanhui Li, Sreyashi Nag, Hui Liu, et al.	LLKD		Paper Code
Synthetic Data Distillation Enables the Extraction of Clinical Information at Scale Elizabeth Geena Woo, Michael C Burkhart, Emily Alsentzer, et al.	SD-CL		Paper Code

(back to top ↑)

Parameter-based Transfer

Title & Authors	Method	Category	Links
Direct Distillation Between Different Domains Jialiang Tang, Shuo Chen, Gang Niu, et al.	4Ds		Paper
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective Ming Zhong, Chenxin An, Weizhu Chen, et al.	PK-LoRA		Paper Code
From Instance Training to Instruction Learning: Task Adapters Generation from Instructions Huanxuan Liao, Shizhu He, Yao Xu, et al.	TAGI		Paper Code
MLAS-LoRA: Language-Aware Parameters Detection and Lora-Based Knowledge Transfer for Multilingual Machine Translation Tianyu Dong, Bo Li, Jinsong Liu, et al.	MLAS-LoRA		Paper
Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models Kyeonghyun Kim, Jinhee Jang, Juhwan Choi, et al.	PiFi		Paper
TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs Lanxiang Hu, Tajana Rosing, Hao Zhang	TrimLLM		Paper
LoRA-Gen: Specializing Large Language Model via Online LoRA Generation Yicheng Xiao, Lin Song, Rui Yang, et al.	LoRA-Gen		Paper
In-Context Meta LoRA Generation Yihua Shao, Minxi Yan, Yang Liu, et al.	ICM-LoRA		Paper

(back to top ↑)

Knowledge Transfer from SMs to LMs

Distillation-based Transfer

Title & Authors	Method	Category	Links
FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion Sijie Cheng, Jingwen Wu, Yanghua Xiao, et al.	FedGEM		Paper
Learning to Teach with Student Feedback Yi-tao Liu, Tian-xiang Sun, Xi-peng Qiu, et al.	IKD		Paper
Meta Pseudo Labels Hieu Pham, Zihang Dai, Qizhe Xie, et al.	Meta-Pseudo-Labels		Paper Code
Dual Knowledge Distillation for Bidirectional Neural Machine Translation Huaao Zhang, Shigui Qiu, Shilong Wu	DKDB		Paper
BERT Learns to Teach: Knowledge Distillation with Meta Learning Wangchunshu Zhou, Canwen Xu, Julian McAuley	MetaDistil		Paper Code
Shadow Knowledge Distillation: Bridging Offline and Online Knowledge Transfer Lujun Li, Zhe Jin	SHAKE		Paper Code
IDEAL: Query-Efficient Data-Free Learning from Black-Box Models Jie Zhang, et al.	IDEAL		Paper Code
Multimodal Federated Learning via Contrastive Representation Ensemble Qiying Yu, Yang Liu, Yimu Wang, et al.	CreamFL		Paper Code
DataShunt: Collaboration of Small and Large Models for Lower Costs and Better Performance Dong Chen, Yueting Zhuang, Shuo Zhang, et al.	DataShunt		Paper Code
Weak-To-Strong Generalization: Eliciting Strong Capabilities with Weak Supervision Collin Burns, et al.	W2S		Paper
Reverse Knowledge Distillation: Training a Large Model Using a Small One for Retinal Image Matching on Limited Data Sahar Almahfouz Nasser, Nihar Gupte, Amit Sethi	RKD		Paper Code
Beyond Output Matching: Bidirectional Alignment for Enhanced in-Context Learning Chengwei Qin, Wenhan Xia, Fangkai Jiao, et al.	BiAlign		Paper
Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models Jiongran Wu, Jiahao Liu, Dongsheng Li, et al.	LLMD4Rec		Paper
BiLD: Bi-Directional Logits Difference Loss for Large Language Model Distillation Minchong Li, Feng Zhou, Xiaohui Song	BiLD		Paper Code
FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models Tao Fan, Guoqiang Ma, Yan Kang, et al.	FedMKT		Paper Code
Evidential Knowledge Distillation Liangyu Xiang, Junyu Gao, Changsheng Xu	EKD		Paper Code
SLMRec: Distilling Large Language Models into Small for Sequential Recommendation Wujiang Xu, Qitian Wu, Zujie Liang, et al.	SLMRec		Paper Code

(back to top ↑)

Generation-based Transfer

Title & Authors	Method	Category	Links
Dataset Distillation Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, et al.	Dataset Distillation		Paper Code
Mixup: Beyond Empirical Risk Minimization Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, et al.	Mixup		Paper Code
Distilled One-Shot Federated Learning Yanlin Zhou, George Pu, Xiyao Ma, et al.	DOSFL		Paper
Dataset Condensation with Gradient Matching Bo Zhao, et al.	Dataset Condensation		Paper Code
FedProto: Federated Prototype Learning Across Heterogeneous Clients Yue Tan, et al.	FedProto		Paper Code
Federated Learning with Gan-Based Data Synthesis for Non-IID Clients Zi-jian Li, Jia-wei Shao, Yu-yi Mao, et al.	FL-GDS		Paper
Stable Federated Learning with Dataset Condensation Seong-Woong Kim, et al.	SFLD		Paper Code
Federated Learning from Pre-Trained Models: A Contrastive Learning Approach Yue Tan, Guodong Long, Jie Ma, et al.	FedPCL		Paper Code
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe Xiang Yue, Huseyin Inan, Xuechen Li, et al.	DP Transformers		Paper
Harnessing Large-Language Models to Generate Private Synthetic Text Alexey Kurakin, Natalia Ponomareva, Umar Syed, et al.	PST		Paper
Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfer Yongheng Deng, Ziqing Qiao, Ju Ren, et al.	CrossLM		Paper
Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models WANG Ruida, Wangchunshu Zhou, Mrinmaya Sachan	S3		Paper Code
Retrieval-Based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression Jiduan Liu, Jiahao Liu, Qifan Wang, et al.	RetriKT		Paper
Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments Rui Song, Dai Liu, Dave Zhenyu Chen, et al.	FedD3		Paper
GFL: Federated Learning on Non-IID Data via Privacy-Preserving Synthetic Data Yihang Cheng, Lan Zhang, Anran Li	GFL		Paper
FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning Jianqing Zhang, Yang Liu, Yang Hua, et al.	FedTGP		Paper Code
Prompt Public Large Language Models to Synthesize Data for Private On-device Applications Shan-Shan Wu, Zheng Xu, Yanxiang Zhang, et al.	DPSD		Paper Code
An Upload-Efficient Scheme for Transferring Knowledge from a Server-Side Pre-Trained Generator to Clients in Heterogeneous Federated Learning Jianqing Zhang, Yang Liu, Yang Hua, et al.	FedKTL		Paper Code
Differentially Private Synthetic Data via Foundation Model APIs 1: Images Zinan Lin, et al.	PE		Paper Code
Differentially Private Synthetic Data via Foundation Model APIs 2: Text Chulin Xie, Zinan Lin, Arturs Backurs, et al.	DPE		Paper Code
Privacy-Preserving Instructions for Aligning Large Language Models Da Yu, Peter Kairouz, Sewoong Oh, et al.	PPI		Paper Code
Generate Synthetic Text Approximating the Private Distribution with Differential Privacy Wenhao Zhao, Shaoyang Song, Chunlai Zhou	DP Text		Paper
One-Shot Federated Learning via Synthetic Distiller-Distillate Communication Junyuan Zhang, Songhua Liu, Xinchao Wang	FedSD2C		Paper Code
Federated Graph Condensation with Information Bottleneck Principles Bo Yan, Sihao He, Cheng Yang, et al.	FedGC		Paper Code
FedC4: Graph Condensation Meets Client-Client Collaboration for Efficient and Private Federated Graph Learning Ze-kai Chen, Xun-kai Li, Yin-lin Zhu, et al.	FedC4		Paper
Model-Based Large Language Model Customization as Service Zhaomin Wu, Jizhou Guo, Junyi Hou, et al.	Llamdex		Paper
Data-Adaptive Differentially Private Prompt Synthesis for in-Context Learning Fengyu Gao, Ruida Zhou, Tianhao Wang, et al.	AdaDPSyn		Paper

(back to top ↑)

Parameter-based Transfer

Title & Authors	Method	Category	Links
Prefix-tuning: Optimizing Continuous Prompts for Generation Xiang-Lisa Li, et al.	Prefix-tuning		Paper Code
FedPETuning: When Federated Learning Meets the Parameter-Efficient Tuning Methods of Pre-Trained Language Models Zhuo Zhang, Yuanhang Yang, Yong Dai, et al.	FedPETuning		Paper Code
Offsite-tuning: Transfer Learning without Full Model Guangxuan Xiao, Ji Lin, Song Han	Offsite-tuning		Paper Code
BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning Changdae Oh, Hyeji Hwang, Hee-young Lee, et al.	BlackVIP		Paper Code
Parameter-Efficient Tuning for Large Language Model without Calculating Its Gradients Feihu Jin, Jiajun Zhang, Chengqing Zong	PET		Paper
Tunable Soft Prompts Are Messengers in Federated Learning Chenhe Dong, Yuexiang Xie, Bolin Ding, et al.	FedSP		Paper Code
FedPrompt: Communication-Efficient and Privacy-Preserving Prompt Tuning in Federated Learning Haodong Zhao, Wei Du, Fangqi Li, et al.	FedPrompt		Paper
Black-Box Prompt Optimization: Aligning Large Language Models without Model Training Jiale Cheng, Xiao Liu, Kehan Zheng, et al.	BPO		Paper Code
Mixture of LoRA Experts Xun Wu, Shaohan Huang, Furu Wei	MoLE		Paper
DoRA: Weight-Decomposed Low-Rank Adaptation Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, et al.	DoRA		Paper Code
InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models6518 Lichang Chen, Jiuhai Chen, Tom Goldstein, et al.	InstructZero		Paper Code
Federated Adaptation for Foundation Model-Based Recommendations Chun-xu Zhang, Guo-dong Long, Hong-kuan Guo, et al.	FedPA		Paper Code

(back to top ↑)

Cross-silo Collaborative Inference

Split Execution

Title & Authors	Method	Category	Links
Split Learning for Health: Distributed Deep Learning without Sharing Raw Patient Data Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, et al.	SplitNN		Paper
FedBERT: When Federated Learning Meets Pre-Training Yuanyishu Tian, Yao Wan, Lingjuan Lyu, et al.	FedBERT		Paper Code
A Split-and-Privatize Framework for Large Language Model Fine-Tuning Linxiao Cao, Yifei Zhu, Wei Gong	SFPrompt		Paper
Privatelora for Efficient Privacy Preserving LLM Yiming Wang, Yu Lin, Xiaodong Zeng, et al.	PrivateLoRA		Paper Code
Adaptive Layer Splitting for Wireless LLM Inference in Edge Computing: A Model-Based Reinforcement Learning Approach Yuxuan Chen, Rongpeng Li, Xiaoxue Yu, et al.	ALS		Paper
SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimization Akrit Mudvari, Yuang Jiang, Leandros Tassiulas	SplitLLM		Paper Code
Introducing Edge Intelligence to Smart Meters via Federated Split Learning Yehui Li, Dalin Qin, H Vincent Poor, et al.	SFSL		Paper
Vertical Federated Learning: Concepts, Advances, and Challenges Yang Liu, Yan Kang, Tianyuan Zou, et al.	VFL		Paper
Cross-Silo Feature Space Alignment for Federated Learning on Clients with Imbalanced Data Zhuang Qi, Lei Meng, Zhaochuan Li, et al.	Fed-FSA		Paper

(back to top ↑)

Collaborative Decoding

Title & Authors	Method	Category	Links
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts Alisa Liu, Maarten Sap, Ximing Lu, et al.	DExperts		Paper Code
Contrastive Decoding: Open-Ended Text Generation as Optimization Xiang Lisa Li, Ari Holtzman, Daniel Fried, et al.	CD		Paper Code
CombLM: Adapting Black-Box Language Models Through Small Fine-Tuned Models Aitor Ormazabal, Mikel Artetxe, Eneko Agirre	CombLM		Paper Code
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan, Matan Kalman, Yossi Matias	FIT-SD		Paper Code
CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following Kaiyan Zhang, Jianyu Wang, Ermo Hua, et al.	CoGenesis		Paper Code
Controlled Text Generation for Black-Box Language Models via Score-Based Progressive Editor Sangwon Yu, Changmin Lee, Hojin Lee, et al.	ScoPE		Paper Code
Small Models Are Valuable Plug-Ins for Large Language Models Canwen Xu, Yichong Xu, Shuohang Wang, et al.	SuperICL		Paper Code
Tuning Language Models by Proxy A-lisa Liu, Xiao-chuang Han, Yi-zhong Wang, et al.	Proxy-Tune		Paper Code
Mitigating Object Hallucinations in Large Vision-Language Models Through Visual Contrastive Decoding Sicong Leng, Hang Zhang, Guanzheng Chen, et al.	VCD		Paper Code
Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation Esteban Garces Arias, Julian Rodemann, Meimingwei Li, et al.	ACS		Paper Code
An Emulator for Fine-Tuning Large Language Models Using Small Language Models Eric Mitchell, Rafael Rafailov, Archit Sharma, et al.	EFT		Paper
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models Yung-Sung Chuang, Yujia Xie, Hongyin Luo, et al.	DoLa		Paper Code
Online Speculative Decoding Xiao-xuan Liu, Lan-xiang Hu, Peter Bailis, et al.	OSD		Paper Code
Trusting Your Evidence: Hallucinate Less with Context-Aware Decoding Weijia Shi, Xiaochuang Han, Mike Lewis, et al.	CAD		Paper
Cascade Speculative Drafting for Even Faster LLM Inference Zi-yi Chen, Xiao-cong Yang, Jia-cheng Lin, et al.	CS		Paper Code
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan, Zhenyi Lu, Wei Wei, et al.	Dynamic Logits Fusion		Paper Code
SpecTr: Fast Speculative Decoding via Optimal Transport Zi-teng Sun, Ananda Theertha Suresh, Jae Hun Ro, et al.	SpecTr		Paper
Speculative Decoding with Big Little Decoder Sehoon Kim, Karttikeya Mangalam, Suhong Moon, et al.	BiLD		Paper Code
CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steering Hang Lv, Sheng Liang, Hao Wang, et al.	CoSterr		Paper
Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation Hongxiang Zhang, Hao Chen, Muhao Chen, et al.	ActLCD		Paper
Faster Cascades via Speculative Decoding Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, et al.	FCSD		Paper
Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment Gregor Bachmann, Sotiris Anagnostidis, Albert Pumarola, et al.	JD		Paper
Logits Are All We Need to Adapt Closed Models Gaurush Hiranandani, Haolun Wu, Subhojyoti Mukherjee, et al.	Plugin		Paper
Privacy Preserving in-Context-Learning Framework for Large Language Models Bishnu Bhusal, Manoj Acharya, Ramneet Kaur, et al.	DP-ICL		Paper
SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation Hang Lv, Sheng Liang, Hao Wang, et al.	SpecSteer		Paper
Thinking by Subtraction: Confidence-Driven Contrastive Decoding for LLM Reasoning Lexiang Tang, Weihao Gao, Bingchen Zhao, et al.	CCD		Paper
Training-Free Adaptation of New-Generation LLMs Using Legacy Clinical Models Sasha Ronaghi, Chloe Stanwyck, Asad Aali, et al.	CAPT		Paper
DP Fusion: Token-Level Differentially Private Inference for Large Language Models Rushil Thareja, Preslav Nakov, Praneeth Vepakomma, et al.	DP Fusion		Paper

(back to top ↑)

Context-Augmented Collaboration

Retrieval Collaboration

Title & Authors	Method	Category	Links
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Patrick Lewis, Ethan Perez, Aleksandra Piktus, et al.	RAG-LLM		Paper Code
Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, et al.	RAG-end2end		Paper Code
RA-DIT: Retrieval-Augmented Dual Instruction Tuning Xi Victoria Lin, Xilun Chen, Mingda Chen, et al.	RA-DIT		Paper
Self-RAG: Learning to Retrieve, Generate, and Critique Through Self-Reflection Akari Asai, Zeqiu Wu, Yizhong Wang, et al.	Self-RAG		Paper Code
REPLUG: Retrieval-Augmented Black-Box Language Models Weijia Shi, Sewon Min, Michihiro Yasunaga, et al.	REPLUG		Paper Code
Seakr: Self-Aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation Zijun Yao, Weijian Qi, Liangming Pan, et al.	Seakr		Paper Code
UniRAG: Unified Query Understanding Method for Retrieval Augmented Generation Rui Li, Liyang He, Qi Liu, et al.	UniRAG		Paper
RemoteRAG: A Privacy-Preserving LLM Cloud RAG Service Yihang Cheng, Lan Zhang, Junyang Wang, et al.	RemoteRAG		Paper
Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data Shenglai Zeng, Jiankun Zhang, Pengfei He, et al.	SAGE		Paper

Agentic Workflow

Title & Authors	Method	Category	Links
MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework Sirui Hong, Mingchen Zhuge, Jonathan Chen, et al.	MetaGPT		Paper Code
HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends in Hugging Face Yongliang Shen, Kaitao Song, Xu Tan, et al.	HuggingGPT		Paper Code
Reflexion: Language Agents with Verbal Reinforcement Learning Noah Shinn, Federico Cassano, Ashwin Gopinath, et al.	Reflexion		Paper Code
ChatDev: Communicative Agents for Software Development Chen Qian, Wei Liu, Hongzhang Liu, et al.	ChatDev		Paper Code
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs Justin Chen, Swarnadeep Saha, Mohit Bansal	ReConcile		Paper Code
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations Qingyun Wu, Gagan Bansal, Jieyu Zhang, et al.	AutoGen		Paper Code
Encouraging Divergent Thinking in Large Language Models Through Multi-Agent Debate Tian Liang, Zhiwei He, Wenxiang Jiao, et al.	MAD		Paper Code
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs Yaobo Liang, Chenfei Wu, Ting Song, et al.	Taskmatrix		Paper Code
Mobile-Agent-V2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration Junyang Wang, Haiyang Xu, Haitao Jia, et al.	Mobile-Agent		Paper Code
PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving Mihir Parmar, Xin Liu, Palash Goyal, et al.	PlanGEN		Paper
Smurfs: Multi-Agent System Using Context-Efficient DFSDT for Tool Planning Junzhi Chen, Juhao Liang, Benyou Wang	Smurfs		Paper Code
CORE: Reducing UI Exposure in Mobile Agents via Collaboration Between Cloud and Local LLMs Gucongcong Fan, Chaoyue Niu, Fan Wu, et al.	CORE		Paper Code
EcoAgent: An Efficient Edge-Cloud Collaborative Multi-Agent Framework for Mobile Automation Biao Yi, Xavier Hu, Yurun Chen, et al.	EcoAgent		Paper Code

(back to top ↑)

---

📄 License

This project is licensed under the MIT License.

---

✨ Star History