ML Model Serving & Text Inference
17 repos
Infrastructure and libraries for efficiently serving machine learning models, with a focus on text generation, embeddings, and retrieval tasks. The cluster centers on deployment tooling and optimization for large language models, including serialization formats (safetensors), inference engines, and semantic search components like rerankers and text embedding services. Most repos here are utility libraries and server implementations rather than end-user applications.
Python · 1
text-generation ·22
safetensors ·22
text-ranking ·20
text-embeddings-inference ·20
endpoints_compatible ·20
qwen2 ·20
reranker ·20
retrieval ·20
transformers ·20
reasoning ·17
