A Rust, Python and gRPC server for sentence-transformer embeddings.

Overview

This is the beginning of the repo so we're pretty light on the info. More will come! The basic gist is that we intend to create the equivalent of Huggingface's Text Generation Inference API but for sentence-transformer embeddings. This repo is a fork of the text-generation-inference repo.

The current state of the repo is that we have forked the codebase to run the base Sentence-Transformer class from the sentence-transformers library. The server will run manually but we have not completed the dockerfile or actions to create a build pipeline. More to come!

Develop

make server-dev
make router-dev

Testing

# python
make python-server-tests
make python-client-tests
# or both server and client tests
make python-tests
# rust cargo tests
make rust-tests
# integration tests
make integration-tests