noooop/wde

Workflow Defined Engine

Python

188 commits

updated Nov 4, 2025

See the code

README

介绍

Workflow Defined Engine, 针对不同架构的模型实现不同的模块，并按需加载所需的模块。
Asynchronous scheduling, 异步调度，提高GPU利用率，增加吞吐降低延迟
Microservices linked by Zeromq，减少序列化反序列化和io开销，进一步增加吞吐降低延迟

文档

应用

Acknowledgement

vLLM

Similar repos

sskim-ai/Dynamo

ai-dynamo/dynamo

A Datacenter Scale Distributed Inference Serving Framework

8,173

ztxz16/fastllm

fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署Deep…

5,077

llm-d/llm-d

Achieve state of the art inference performance with modern accelerators on Kubernetes

4,667

CURRENTF/SparseEngine

A sparse-first inference engine (SparseEngine).

PaddlePaddle/FastDeploy

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

3,719

WJX3078/mini-vllm

从零实现的 mini vLLM 推理引擎：Block 级 KV Cache (PagedAttention) · Continuous Batching · Prefix Caching ·…

xLLM-AI/xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI…

1,582

Languages

Python

88.8%

Jupyter Notebook

11.1%

noooop/wde

Workflow Defined Engine

Python

188 commits

updated Nov 4, 2025

See the code

README

介绍

Workflow Defined Engine, 针对不同架构的模型实现不同的模块，并按需加载所需的模块。
Asynchronous scheduling, 异步调度，提高GPU利用率，增加吞吐降低延迟
Microservices linked by Zeromq，减少序列化反序列化和io开销，进一步增加吞吐降低延迟

文档

应用

Acknowledgement

vLLM

Similar repos

sskim-ai/Dynamo

ai-dynamo/dynamo

A Datacenter Scale Distributed Inference Serving Framework

8,173

ztxz16/fastllm

5,077

llm-d/llm-d

Achieve state of the art inference performance with modern accelerators on Kubernetes

4,667

CURRENTF/SparseEngine

A sparse-first inference engine (SparseEngine).

PaddlePaddle/FastDeploy

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

3,719

WJX3078/mini-vllm

从零实现的 mini vLLM 推理引擎：Block 级 KV Cache (PagedAttention) · Continuous Batching · Prefix Caching ·…

xLLM-AI/xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI…

1,582

Languages

Python

88.8%

Jupyter Notebook

11.1%