Cluster 650666

4 repos

C · 2

Python · 2

llm-inference ·464

speculative-decoding ·464

retrieval ·220

offloading ·132

large-language-models ·71

mips ·41

nintendo-64 ·41

power8 ·41

powerpc ·41

cost-optimization ·41

Related clusters

fvliang/DART

Official Implementation of DART (DART: Low-Latency Parallel Drafting with Continuity-Aware Tree…

Python

1 commits

FasterDecoding/REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024

220

12 commits

Infini-AI-Lab/UMbreLLa

LLM Inference on consumer devices

Python

132

149 commits

Scottcjn/rpi-inference

RPI (Resonant Permutation Inference) — Zero-multiply text generation. 18K tok/s. 868 KB models.…

19 commits