🔥🔥🔥 Awesome-LLM-Ensemble

"Harnessing Multiple Large Language Models: A Survey on LLM Ensemble" (IJCAI Survey 2026)

Zhijun Chen, Xiaodong Lu, Jingzheng Li, Pengpeng Chen, Zhuoran Li, Kai Sun, Yuankai Luo, Qianren Mao, Ming Li, Likang Xiao, Dingqi Yang, Xiao Huang, Yikun Ban, Hailong Sun, Philip S. Yu

If you like our project, please give it a star ⭐ to show your support！Thank you:)

📣 Notices

🔥🔥🔥 This is a collection of papers on LLM Ensemble.

[2026-05] Accepted by IJCAI Survey 2026!

🔥🔥🔥 [2026-04] We have updated our arXiv paper with a new version! Stay tuned for our journal-style paper in recent months.

[Always] [Add your papers in this repo] Thank you to all the papers that have cited our survey.
We will add all related citing papers to this GitHub repo, in a timely manner, to help increase the visibility of your contributions.

[Always] [Maintain] We will make this list updated frequently!
If you found any missed/new paper, please don't hesitate to contact us or Pull requests.

 

🍀 Citation

If you find this survey useful, please consider citing our paper:

@article{chen2025harnessing,
  title={Harnessing Multiple Large Language Models: A Survey on LLM Ensemble},
  author={Chen, Zhijun and Lu, Xiaodong and Li, Jingzheng and Chen, Pengpeng and Li, Zhuoran and Sun, Kai and Luo, Yuankai and Mao, Qianren and Li, Ming and Xiao, Likang and Yang, Dingqi and Huang, Xiao and Ban, Yikun and Sun, Hailong and Yu, Philip S},
  journal={arXiv preprint arXiv:2502.18036},
  year={2025}
}

   
 

• Contents
  • 1. LLM Ensemble and Taxonomy
    • 1.1 LLM Ensemble
    • 1.2 Taxonomy
  • 2. Papers
    • 2.1 Ensemble Before Inference
      • 2.1.1 (a,1) Pre-Trained Router
      • 2.1.2 (a,2) Non pre-trained router
    • 2.2 Ensemble During Inference
      • 2.2.1 (b,1) Token-Level Ensemble
      • 2.2.2 (b,2) Span-Level Ensemble
      • 2.2.3 (b,3) Process-Level Ensemble
    • 2.3 Ensemble After Inference
      • 2.3.1 (c,1) Non Cascade
      • 2.3.2 (c,2) Cascade
    • 2.4 Others: Benchmarks, Applications, Systems and Related Surveys
      • 2.4.1 Benchmarks
      • 2.4.2 Applications
      • 2.4.3 Systems
      • 2.4.4 Related Surveys
  • 3 Others: Some public implementations of the LLM Ensemble methods
  • 4 Others: Some other related interesting papers
    • 4.1 Test-Time Scaling
    • 4.2 LLM Collaboration and Others
  • 5 Summarization

1. LLM Ensemble and Taxonomy

1.1 LLM Ensemble

Paper Abstract:

LLM Ensemble---which involves the comprehensive use of multiple large language models (LLMs), each aimed at handling user queries during the downstream inference, to benefit from their individual strengths---has gained substantial attention recently. The widespread availability of LLMs, coupled with their varying strengths and out-of-the-box usability, has profoundly advanced the field of LLM Ensemble. This paper presents the first systematic review of recent developments in LLM Ensemble. First, we introduce our taxonomy of LLM Ensemble and discuss several related research problems. Then, we provide a more in-depth classification of methods under the broad categories of ``ensemble-before-inference, ensemble-during-inference, ensemble-after-inference'', and review all relevant methods. Finally, we introduce related benchmarks and applications, summarize existing studies, and suggest several future research directions. A curated list of papers on LLM Ensemble is available at https://github.com/junchenzhi/Awesome-LLM-Ensemble.

1.2 Taxonomy

• (a) Ensemble before inference.
  Since the ensemble-before-inference methods require routing a query to the most suitable LLM before LLM inference, the core of such methods lies in predicting the utility of candidate models for a given query under certain preferences (e.g., performance or cost). Based on how they formulate the utility of candidate LLMs, we divide existing methods into two categories:

  • (a1) Discrete utility methods, discretize the model utility into categorical labels;
  • (a2) Continuous utility methods model LLM utility as real-valued variables, such as response length or performance scores. This formulation enables a fine-grained characterization of model behavior, capturing subtle performance differences obscured by categorical definitions.

• (b) Ensemble during inference.
  As the most granular form of ensemble among the three broad categories, this type of approach encompasses:

  • (b1) Token-level ensemble methods, which integrate the token-level outputs of multiple models at the finest granularity of decoding;
  • (b2) Span-level ensemble methods, which conduct ensemble at the level of a sequence fragment (e.g., a span of four words);
  • (b3) Process-level ensemble methods, which select the optimal reasoning process step-by-step within the reasoning chain for a given complex reasoning task. Note that for these ensemble-during-inference methods, the aggregated text segments will be concatenated with the previous text and fed again to models.

• (c) Ensemble after inference.
  These methods can be classified into two categories:

  • (c1) Non cascade methods, which perform ensemble using multiple complete responses contributed from all LLM candidates;
  • (c2) Cascade methods, which consider both performance and inference costs, progressively reasoning through a chain of LLM candidates largely sorted by model size to find the most suitable inference response.

   
 

2. Papers

2.1 Ensemble Before Inference

2.1.1 (a,1) Discrete utility methods

2.1.2 (a,2) Continuous utility methods

2.2 Ensemble During Inference

2.2.1 (b,1) Token-Level Ensemble

2.2.2 (b,2) Span-Level Ensemble

2.2.3 (b,3) Process-Level Ensemble

 

2.3 Ensemble After Inference

2.3.1 (c,1) Non Cascade

2.3.2 (c,2) Cascade

 

2.4 Others: Benchmarks, Applications, Systems and Related Surveys

2.4.1 Benchmarks

2.4.2 Applications

Beyond the methods presented before, the concept of LLM Ensemble has found applications in a variety of more specialized tasks and domains. Here we give some examples:

2.4.3 Systems

2.4.4 Related Surveys

   
 

3 Others: Some public implementations of the LLM Ensemble methods

   
 

4 Others: Some other related interesting papers

Here we briefly list some related papers, which are either discovered by us or suggested by the authors to this repository. They mainly focus on LLM Collaboration.

4.1 Test-Time Scaling

4.2 LLM Collaboration and Others

   
 

5 Summarization

   
 

---