Shadowgraph
TrendingBuzzTop stargazers
Chrome extension
Sign in

neuralmagic/nm-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Python

267

988 commits

updated Dec 4, 2025

nm-vllm.readthedocs.io

See the code

README

nm-vllm

THIS REPO HAS BEEN ARCHIVED AS OF SEPTEMBER 2024. NEURAL MAGIC IS STILL RELEASING ENTERPRISE PACKAGES RELATED TO VLLM. OUR RELEASE REPO HAS JUST GONE PRIVATE.

To learn more about nm-vllm Enterprise, visit the nm-vllm product page.

To contribute and to see our contributions to vLLM, visit vLLM.

To view the latest releases, benchmarking, models, and evaluations from Neural Magic, visit nm-vllm-certs.

Neural Magic maintains a variety of optimized models on our Hugging Face organization profiles:

Project health

PyPI

nm-vllm

Latest release

0.6.3.0 · Nov 6, 2024

Releases this year

0

Significant stargazers

Kevin Ko

941 followers · starred May 2024

Kyle Sayers

82 followers · starred Jul 2024

Junbum Lee

954 followers · starred May 2024

Dipika Sikka

103 followers · starred Mar 2024

Similar repos

tenstorrent/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

30

CarrotShoo/vllm-old

A high-throughput and memory-efficient inference and serving engine for LLMs

0

UpstageAI/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

2

Taichu-AI/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

3

local-inference-lab/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

42

XiaomiMiMo/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

33

rahul-sarvam/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

4

wayneworkman/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

0

neuralmagic/nm-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Python

267

988 commits

updated Dec 4, 2025

nm-vllm.readthedocs.io

See the code

README

nm-vllm

THIS REPO HAS BEEN ARCHIVED AS OF SEPTEMBER 2024. NEURAL MAGIC IS STILL RELEASING ENTERPRISE PACKAGES RELATED TO VLLM. OUR RELEASE REPO HAS JUST GONE PRIVATE.

To learn more about nm-vllm Enterprise, visit the nm-vllm product page.

To contribute and to see our contributions to vLLM, visit vLLM.

To view the latest releases, benchmarking, models, and evaluations from Neural Magic, visit nm-vllm-certs.

Neural Magic maintains a variety of optimized models on our Hugging Face organization profiles:

Project health

PyPI

nm-vllm

Latest release

0.6.3.0 · Nov 6, 2024

Releases this year

0

Significant stargazers

Kevin Ko

941 followers · starred May 2024

Kyle Sayers

82 followers · starred Jul 2024

Junbum Lee

954 followers · starred May 2024

Dipika Sikka

103 followers · starred Mar 2024

Similar repos

tenstorrent/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

30

CarrotShoo/vllm-old

A high-throughput and memory-efficient inference and serving engine for LLMs

0

UpstageAI/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

2

Taichu-AI/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

3

local-inference-lab/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

42

XiaomiMiMo/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

33

rahul-sarvam/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

4

wayneworkman/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

0