SGLang-Omni

SGLang-Omni is an ecosystem project for SGLang. Omni models refer to models that have multi-modal inputs and multi-modal outputs. These models typically consist of multiple stages, making SGLang's LLM-specific architecture no longer suitable. Therefore, SGLang-Omni is designed to provide the ability to orchestrate multi-stage pipeline with high performance and real-time API support. Our core features include:

• Native Integration with SGLang for performance
• Multi-Stage Pipeline Framework for Omni Models
• OpenAI-Compatible Server with Real-Time API support

Quick Start

We will host the documentation upon open-sourcing. For now, you can find the documentation in the docs folder.

• Get Started
• Developer Reference
• Benchmarks
• Examples