Following CacheBlend, we implemented a demonstration system based on vLLM. This prototype focuses solely on functionality rather than performance. At present, we have open-sourced only the demo system. Our full production system—similar in nature to LMCache—is still under active development, and we plan to open-source it as well once it reaches a stable stage.
For convenience, you may compare (diff) this project with vLLM 0.7.0 to quickly identify the modifications we introduced.
| Documentation | Blog | Paper | Discord | Twitter/X | Developer Slack |
Latest News 🔥
vLLM is a fast and easy-to-use library for LLM inference and serving.
Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evloved into a community-driven project with contributions from both academia and industry.
vLLM is fast with:
Performance benchmark: We include a performance benchmark at the end of our blog post. It compares the performance of vLLM against other LLM serving engines (TensorRT-LLM, SGLang and LMDeploy). The implementation is under nightly-benchmarks folder and you can reproduce this benchmark using our one-click runnable script.
vLLM is flexible and easy to use with:
vLLM seamlessly supports most popular open-source models on HuggingFace, including:
Find the full list of supported models here.
Install vLLM with pip or from source:
pip install vllm
Visit our documentation to learn more.
We welcome and value any contributions and collaborations. Please check out CONTRIBUTING.md for how to get involved.
vLLM is a community project. Our compute resources for development and testing are supported by the following organizations. Thank you for your support!
Cash Donations:
Compute Resources:
Slack Sponsor: Anyscale
We also have an official fundraising venue through OpenCollective. We plan to use the fund to support the development, maintenance, and adoption of vLLM.
If you use vLLM for your research, please cite our paper:
@inproceedings{kwon2023efficient,
title={Efficient Memory Management for Large Language Model Serving with PagedAttention},
author={Woosuk Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica},
booktitle={Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles},
year={2023}
}
6,861 followers · starred Mar 2026
42 followers · starred Mar 2026
Following CacheBlend, we implemented a demonstration system based on vLLM. This prototype focuses solely on functionality rather than performance. At present, we have open-sourced only the demo system. Our full production system—similar in nature to LMCache—is still under active development, and we plan to open-source it as well once it reaches a stable stage.
For convenience, you may compare (diff) this project with vLLM 0.7.0 to quickly identify the modifications we introduced.
| Documentation | Blog | Paper | Discord | Twitter/X | Developer Slack |
Latest News 🔥
vLLM is a fast and easy-to-use library for LLM inference and serving.
Originally developed in the Sky Computing Lab at UC Berkeley, vLLM has evloved into a community-driven project with contributions from both academia and industry.
vLLM is fast with:
Performance benchmark: We include a performance benchmark at the end of our blog post. It compares the performance of vLLM against other LLM serving engines (TensorRT-LLM, SGLang and LMDeploy). The implementation is under nightly-benchmarks folder and you can reproduce this benchmark using our one-click runnable script.
vLLM is flexible and easy to use with:
vLLM seamlessly supports most popular open-source models on HuggingFace, including:
Find the full list of supported models here.
Install vLLM with pip or from source:
pip install vllm
Visit our documentation to learn more.
We welcome and value any contributions and collaborations. Please check out CONTRIBUTING.md for how to get involved.
vLLM is a community project. Our compute resources for development and testing are supported by the following organizations. Thank you for your support!
Cash Donations:
Compute Resources:
Slack Sponsor: Anyscale
We also have an official fundraising venue through OpenCollective. We plan to use the fund to support the development, maintenance, and adoption of vLLM.
If you use vLLM for your research, please cite our paper:
@inproceedings{kwon2023efficient,
title={Efficient Memory Management for Large Language Model Serving with PagedAttention},
author={Woosuk Kwon and Zhuohan Li and Siyuan Zhuang and Ying Sheng and Lianmin Zheng and Cody Hao Yu and Joseph E. Gonzalez and Hao Zhang and Ion Stoica},
booktitle={Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles},
year={2023}
}
6,861 followers · starred Mar 2026
42 followers · starred Mar 2026