RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Python
1,361
1,561 commits
updated Oct 12, 2026
| Documentation | Contact Us |
RTP-LLM is a Large Language Model (LLM) inference acceleration engine developed by Alibaba's Foundation Model Inference Team. It is widely used within Alibaba Group, supporting LLM service across multiple business units including Taobao, Tmall, Idlefish, Cainiao, Amap, Ele.me, AE, and Lazada.
RTP-LLM is a sub-project of the havenask project.
Trusted and deployed across numerous LLM scenarios:
Learn more about RTP-LLM's performance in our benchmark reports:
Our project is mainly based on FasterTransformer, and on this basis, we have integrated some kernel implementations from TensorRT-LLM. We also draw inspiration from vllm, transformers, llava, and qwen-vl. We thank these projects for their inspiration and help.
If you find RTP-LLM useful in your research or project, please consider citing:
@Misc{rtp-llm,
author = {Alibaba},
title = {RTP-LLM: A High-Performance LLM Inference Engine},
howpublished = {\url{https://github.com/alibaba/rtp-llm}},
year = {2025},
}
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Python
1,361
1,561 commits
updated Oct 12, 2026
| Documentation | Contact Us |
RTP-LLM is a Large Language Model (LLM) inference acceleration engine developed by Alibaba's Foundation Model Inference Team. It is widely used within Alibaba Group, supporting LLM service across multiple business units including Taobao, Tmall, Idlefish, Cainiao, Amap, Ele.me, AE, and Lazada.
RTP-LLM is a sub-project of the havenask project.
Trusted and deployed across numerous LLM scenarios:
Learn more about RTP-LLM's performance in our benchmark reports:
Our project is mainly based on FasterTransformer, and on this basis, we have integrated some kernel implementations from TensorRT-LLM. We also draw inspiration from vllm, transformers, llava, and qwen-vl. We thank these projects for their inspiration and help.
If you find RTP-LLM useful in your research or project, please consider citing:
@Misc{rtp-llm,
author = {Alibaba},
title = {RTP-LLM: A High-Performance LLM Inference Engine},
howpublished = {\url{https://github.com/alibaba/rtp-llm}},
year = {2025},
}