Shadowgraph
TrendingAbout
Sign in

fjklqq/text-embeddings-inference-npu

Text Embeddings Inference适配Ascend环境及接入（MindIETorch、ATB、Pytorch）模型后端

HTML

5

2 commits

updated Apr 6, 2025

See the code

README

Text Embeddings Inference Ascend Plugin

Get Started

Build

docker build . -f Dockerfile-cann

Usage

Torch 后端 (默认)

TEI_NPU_BACKEND=torch text-embeddings-inference --model-id $model

MindIE Torch 后端

参考选择MindIE Torch作为模型后端

TEI_NPU_BACKEND=mindietorch text-embeddings-inference --model-id $model

MindIE Torch 后端

参考选择ATB作为模型后端

TEI_NPU_BACKEND=atb TEI_COVER_REMOTE_CODE=true text-embeddings-inference --model-id $model

Contributors

fjklqq

2 commits

Similar repos

yuweimian-shy/text-embeddings-inference1

用于text-embeddings-inference

0

UHubara/TEI

0

huggingface/text-embeddings-inference

A blazing fast inference solution for text embeddings models

5,056

OrigamiDream/jina-embeddings-inference

A blazing fast inference solution for Jina-embeddings-v3

1

michaelfeil/infinity

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models,…

2,942

huggingface/text-generation-inference

Large Language Model Text Generation Inference

10,885

PeterYaoNYU/LLMInfer

0

sunny3/tgi_eos_bias

0

Languages

HTML

52.0%

C++

29.8%

Cuda

15.1%

Python

2.0%

fjklqq/text-embeddings-inference-npu

Text Embeddings Inference适配Ascend环境及接入（MindIETorch、ATB、Pytorch）模型后端

HTML

5

2 commits

updated Apr 6, 2025

See the code

README

Text Embeddings Inference Ascend Plugin

Get Started

Build

docker build . -f Dockerfile-cann

Usage

Torch 后端 (默认)

TEI_NPU_BACKEND=torch text-embeddings-inference --model-id $model

MindIE Torch 后端

参考选择MindIE Torch作为模型后端

TEI_NPU_BACKEND=mindietorch text-embeddings-inference --model-id $model

MindIE Torch 后端

参考选择ATB作为模型后端

TEI_NPU_BACKEND=atb TEI_COVER_REMOTE_CODE=true text-embeddings-inference --model-id $model

Similar repos

yuweimian-shy/text-embeddings-inference1

用于text-embeddings-inference

0

UHubara/TEI

0

huggingface/text-embeddings-inference

A blazing fast inference solution for text embeddings models

5,056

OrigamiDream/jina-embeddings-inference

A blazing fast inference solution for Jina-embeddings-v3

1

michaelfeil/infinity

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models,…

2,942

huggingface/text-generation-inference

Large Language Model Text Generation Inference

10,885

PeterYaoNYU/LLMInfer

0

sunny3/tgi_eos_bias

0

Contributors

fjklqq

2 commits

Languages

HTML

52.0%

C++

29.8%

Cuda

15.1%

Python

2.0%