Awesome-Routing-LLMs

A curated list of awesome works in Routing LLMs paradigm

🤩👉 Welcome to submit your contributions to this code repository.

---

Survey

Routing Papers: Pre‑judgment Routing

Pre‑judgment routing makes a routing decision before any generation starts. It relies only on the initial query and static model information, making it suitable for fast, one‑shot allocation. The three main strategies are feature matching, predictive scoring, and constrained optimization.

Sub‑Category: Constrained Optimization

In deployment, routing often faces hard resource constraints (e.g., maximum latency or cost). This paradigm casts decision‑making as constrained utility maximization, using optimization strategies, multi‑objective formulations, or adaptive policies such as reinforcement learning to trade off quality and cost under budget limits.

Sub‑Category: Feature Matching

Feature matching aligns the query representation and model capability representations in a shared space, then selects the model whose embedding is closest to the query. This approach can be lightweight and training‑free, but may degrade under distribution shift; adaptive methods use attention, policy learning, or prototype matching to stay robust.

| Brick: Spatial Capability Routing for the Mixture-of-Models (MoM) Paradigm (Brick) | ArXiv 2026 | General, Code, Math, Reasoning, Knowledge, Agent | github |

Sub‑Category: Predictive Scoring

Predictive scoring learns a parametric estimator (e.g., a small neural network) that predicts the expected utility of each model for a given query. This enables quality‑cost trade‑offs, multi‑objective optimization, and robustness under distribution shift through techniques like minimax, causal inference, or uncertainty estimation.

---

Routing Papers: Verification Routing

Verification routing expands the decision space to intermediate generation states (partial outputs, confidence scores, or reasoning steps). It can revise routing decisions during inference, making the process more adaptive. Two common patterns are self‑assessment (reactive fallback) and collaborative generation (proactive, token‑level interleaving).

Sub‑Category: Collaborative Generation

Collaborative generation integrates decision points directly into the generation process. Instead of a one‑shot model choice, multiple models collaborate step‑wise or token‑wise – e.g., switching control at reasoning boundaries or invoking expert models only at critical tokens. This enables fine‑grained, online adaptation.

Sub‑Category: Self‑Assessment

Self‑assessment uses the model’s own confidence, uncertainty, or self‑consistency signals to decide whether to keep using a lightweight model or escalate to a stronger one. Decisions can be made at sequence end or at reasoning bottlenecks, often with a confidence threshold that triggers fallback.

---

Routing Papers: Memory‑based Routing

Memory‑based routing conditions decisions not only on the current query but also on persistent historical memory – past queries, outcomes, or user interactions. This grounds uncertain online prediction in reusable evidence, enabling cross‑domain and multi‑turn adaptation. The main approaches are retrieval‑based and structured (graph‑based) routing.

Sub‑Category: Retrieval

Retrieval‑based routing finds a semantic neighborhood of historical samples similar to the current query, then aggregates observed model utilities from those samples (e.g., via kNN or proximity weighting). This local estimation adapts to query variations without heavy retraining and can handle distribution shifts gracefully.

Sub‑Category: Structured

Structured routing represents interactions, queries, tasks, and models as nodes in a topological history graph. Routing becomes a graph‑based scoring problem – e.g., using a Graph Neural Network to encode query and model nodes and predict their routing affinity. This supports relational reasoning, personalization, and long‑term planning.

Routing Papers: System Analysis

Analytical work studies system‑level behavior: theoretical bounds, benchmark evaluation, and safety risks. These works treat routing as part of a larger ecosystem with multiple objectives, constraints, and potential adversaries. The three sub‑categories cover theoretical modeling, benchmark & evaluation, and safety analysis.

Sub‑Category: Theoretical Modeling

Theoretical modeling applies tools from pricing theory, queueing analysis, and game theory to understand routing in constrained, dynamic, or strategic settings. For example, Stackelberg games model LLM service pricing, while contextual queueing bandits capture user retrials and scheduling. These studies reveal interdependencies among routing quality, latency, and cost.

Sub‑Category: Benchmark and Evaluation

Benchmarks assess routers across diverse tasks, model pools, and deployment constraints (latency, cost, robustness). They extend evaluation from static model selection to dynamic, multi‑dimensional trade‑offs. Representative benchmarks include RouterBench, RouterEval, VL‑RouterBench (multimodal), and AgentSelect (agentic scenarios). A unified framework is needed to jointly evaluate quality, latency, cost, robustness, and resource efficiency.

Sub‑Category: Safety Analysis

Safety analysis studies routing failures under adversarial, unreliable, or privacy‑sensitive conditions. Attackers can redirect queries to expensive models or bypass policies (e.g., confounder gadgets, adversarial suffixes). Defenses include contrastive learning, embedding‑based filtering, and hybrid surrogate models. Future routers must embed safety, privacy, and security as core design constraints.