Shadowgraph
TrendingBuzzTop stargazers
Chrome extension
Sign in

marcovwu/llm-onnx-runner

Export Breeze

Python

1

2 commits

updated Apr 11, 2024

See the code

README

Porting with Optimum

# install
pip install .

# export
optimum-cli export onnx -m {HUGGINGFACE_MODEL_PATH}--batch_size 1 --sequence_length SEQUENCE_LENGTH {YOUR_OUTPUT_FOLDER}

# run & check onnx
python ortrunner/runner.py

Similar repos

huggingface/optimum-onnx

🤗 Optimum ONNX: Export your model to ONNX and run inference with ONNX Runtime

161

naomili0924/inference_driven_model_compiler

101

kazssym/optimum-study-1

Experiments of the StableLM 3B 4E1T model with ONNX Runtime

0

dsyubaev/onnx_bert

2

natke/llama

0

rui-ren/turnkeyml

TurnkeyML

0

gravitee-io/Llama-Prompt-Guard-2-onnx

ipeline to convert Llama-Prompt-Guard-2 models into ONNX format

1

gravitee-io-labs/Llama-Prompt-Guard-2-onnx

ipeline to convert Llama-Prompt-Guard-2 models into ONNX format

1

marcovwu/llm-onnx-runner

Export Breeze

Python

1

2 commits

updated Apr 11, 2024

See the code

README

Porting with Optimum

# install
pip install .

# export
optimum-cli export onnx -m {HUGGINGFACE_MODEL_PATH}--batch_size 1 --sequence_length SEQUENCE_LENGTH {YOUR_OUTPUT_FOLDER}

# run & check onnx
python ortrunner/runner.py

Similar repos

huggingface/optimum-onnx

🤗 Optimum ONNX: Export your model to ONNX and run inference with ONNX Runtime

161

naomili0924/inference_driven_model_compiler

101

kazssym/optimum-study-1

Experiments of the StableLM 3B 4E1T model with ONNX Runtime

0

dsyubaev/onnx_bert

2

natke/llama

0

rui-ren/turnkeyml

TurnkeyML

0

gravitee-io/Llama-Prompt-Guard-2-onnx

ipeline to convert Llama-Prompt-Guard-2 models into ONNX format

1

gravitee-io-labs/Llama-Prompt-Guard-2-onnx

ipeline to convert Llama-Prompt-Guard-2 models into ONNX format

1