Shadowgraph
TrendingAbout
Sign in

MemVerge/semantic-router

A programmable Mixture-of-Models router for heterogeneous LLM inference

Go

1

1,977 commits

updated Sep 17, 2026

vllm-sr.ai

See the code

README

vLLM Semantic Router

Make Your Mixture-of-Models Programmable.

Documentation | Playground | Blog | Publications | Hugging Face | Slack

About

vLLM Semantic Router is a programmable routing layer for building Mixture-of-Models systems across heterogeneous LLM infrastructure. It evaluates request signals, user preferences, and application policies to select—or compose—the right model path for each request.

Use it to improve quality, cost, latency, privacy, and safety without hard-coding routing logic into applications.

DimensionFragmented todayWith vLLM SR
ModelsModels specialize in different work.Compose personalized model paths.
ComputeGPUs, accelerators, edge, and cloud coexist.Route across heterogeneous compute.
LocationInference spans edge, private, and cloud.Keep data within its boundaries.
Preference"Best" changes by user and workload.Make every preference executable.

Explore how it works →

Getting Started

Install

curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev

For platform notes, detailed setup options, and troubleshooting, see the Installation Guide.

Online playground

Try the online playground at https://app.vllm-sr.ai/playground.

Credentials:

  • Username: love@vllm-sr.ai
  • Password: vllm-sr-read

Latest News

Earlier announcements

More announcements are available on the Blog and Publications pages.

Community

For questions, feedback, or to contribute, please join the #semantic-router channel in vLLM Slack. Track contributors, workgroups, and weekly activity at community.vllm-sr.ai.

Community Meetings

We host community meetings on the first and third Tuesday of each month to sync with contributors across different time zones:

Contributing

If you want to contribute, start with CONTRIBUTING.md.

For repository-native development workflow and validation commands, use AGENTS.md as the entrypoint and tools/agent/docs/README.md as the canonical index.

Citation

If you find Semantic Router helpful in your research or projects, please consider citing it:

@misc{semanticrouter2025,
  title={vLLM Semantic Router},
  author={vLLM Semantic Router Team},
  year={2025},
  howpublished={\url{https://github.com/vllm-project/semantic-router}},
}

Sponsors

We are grateful to our sponsors who support us:

AMD provides us with GPU resources and ROCm™ software for training and researching frontier router models, enhancing E2E testing, and building the online models playground.

Contributors

(top 30 of 194)

Xunzhuo

450 commits

rootfs

204 commits

theohsiung

92 commits

samzong

67 commits

Similar repos

vllm-project/semantic-router

A programmable Mixture-of-Models router for heterogeneous LLM inference

5,856

ulab-uiuc/LLMRouter

LLMRouter: An Open-Source Library for LLM Routing

2,877

NVIDIA-AI-Blueprints/llm-router

Route LLM requests to the best model for the task at hand.

349

aurelio-labs/semantic-router

Superfast AI decision making and intelligent processing of multi-modal data.

3,899

MCERQUA/LLM-Runner-Router

LLM-Runner-Router is not just another model loader - it's a full-stack agnostic neural…

1

XiaomiMiMo/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

31

CarrotShoo/vllm-old

A high-throughput and memory-efficient inference and serving engine for LLMs

0

congcongchen123/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

0

Languages

Go

48.8%

Python

21.4%

TypeScript

15.0%

Rust

6.5%

CSS

3.1%

Shell

1.7%

MemVerge/semantic-router

A programmable Mixture-of-Models router for heterogeneous LLM inference

Go

1

1,977 commits

updated Sep 17, 2026

vllm-sr.ai

See the code

README

vLLM Semantic Router

Make Your Mixture-of-Models Programmable.

Documentation | Playground | Blog | Publications | Hugging Face | Slack

About

vLLM Semantic Router is a programmable routing layer for building Mixture-of-Models systems across heterogeneous LLM infrastructure. It evaluates request signals, user preferences, and application policies to select—or compose—the right model path for each request.

Use it to improve quality, cost, latency, privacy, and safety without hard-coding routing logic into applications.

DimensionFragmented todayWith vLLM SR
ModelsModels specialize in different work.Compose personalized model paths.
ComputeGPUs, accelerators, edge, and cloud coexist.Route across heterogeneous compute.
LocationInference spans edge, private, and cloud.Keep data within its boundaries.
Preference"Best" changes by user and workload.Make every preference executable.

Explore how it works →

Getting Started

Install

curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev

For platform notes, detailed setup options, and troubleshooting, see the Installation Guide.

Online playground

Try the online playground at https://app.vllm-sr.ai/playground.

Credentials:

  • Username: love@vllm-sr.ai
  • Password: vllm-sr-read

Latest News

Earlier announcements

More announcements are available on the Blog and Publications pages.

Community

For questions, feedback, or to contribute, please join the #semantic-router channel in vLLM Slack. Track contributors, workgroups, and weekly activity at community.vllm-sr.ai.

Community Meetings

We host community meetings on the first and third Tuesday of each month to sync with contributors across different time zones:

Contributing

If you want to contribute, start with CONTRIBUTING.md.

For repository-native development workflow and validation commands, use AGENTS.md as the entrypoint and tools/agent/docs/README.md as the canonical index.

Citation

If you find Semantic Router helpful in your research or projects, please consider citing it:

@misc{semanticrouter2025,
  title={vLLM Semantic Router},
  author={vLLM Semantic Router Team},
  year={2025},
  howpublished={\url{https://github.com/vllm-project/semantic-router}},
}

Sponsors

We are grateful to our sponsors who support us:

AMD provides us with GPU resources and ROCm™ software for training and researching frontier router models, enhancing E2E testing, and building the online models playground.

Similar repos

vllm-project/semantic-router

A programmable Mixture-of-Models router for heterogeneous LLM inference

5,856

ulab-uiuc/LLMRouter

LLMRouter: An Open-Source Library for LLM Routing

2,877

NVIDIA-AI-Blueprints/llm-router

Route LLM requests to the best model for the task at hand.

349

aurelio-labs/semantic-router

Superfast AI decision making and intelligent processing of multi-modal data.

3,899

MCERQUA/LLM-Runner-Router

LLM-Runner-Router is not just another model loader - it's a full-stack agnostic neural…

1

XiaomiMiMo/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

31

CarrotShoo/vllm-old

A high-throughput and memory-efficient inference and serving engine for LLMs

0

congcongchen123/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

0

Contributors

(top 30 of 194)

Xunzhuo

450 commits

rootfs

204 commits

theohsiung

92 commits

samzong

67 commits

Languages

Go

48.8%

Python

21.4%

TypeScript

15.0%

Rust

6.5%

CSS

3.1%

Shell

1.7%