Cluster 438948

12 repos

Python · 6

Jupyter Notebook · 2

C++ · 1

HTML · 1

Rust · 1

TeX · 1

llm-serving ·55,627

llm ·45,425

llmops ·42,968

llm-inference ·40,746

llm-training ·36,208

mlops ·27,193

machine-learning ·24,619

deep-learning ·24,393

model-serving ·23,484

distributed-training ·14,657

Related clusters

LLM Inference Optimization & Serving15 LLM Serving and Inference Optimization36 GPU-Accelerated ML & CUDA Kernels37 Cluster 4386336 LLM Fine-tuning and LoRA Optimization15 Large Language Models and NLP16

bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps,…

Python

8,834

3,743 commits

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

C++

2,800

4,658 commits

predibase/lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Python

3,829

876 commits

microsoft/aici

AICI: Prompts as (Wasm) Programs

Rust

2,077

1,617 commits

liguodongiot/llm-action

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

HTML

25,038

952 commits

efeslab/Nanoflow

A throughput-oriented high-performance serving framework for LLMs

Jupyter Notebook

977

147 commits

mosecorg/mosec

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to…

Python

902

480 commits

OpenMachine-ai/transformer-tricks

A collection of tricks and tools to speed up transformer models

TeX

226

210 commits

rohan-paul/LLM-FineTuning-Large-Language-Models

LLM (Large Language Model) FineTuning

Jupyter Notebook

578

166 commits

Scottcjn/ram-coffers

LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on…

Python

164

90 commits

skypilot-org/skypilot

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI…

Python

10,592

5,674 commits

FedML-AI/FedML

FEDML - The unified and scalable ML library for large-scale distributed training, model serving,…

Python

4,065

10,439 commits