SlyEcho/open_llama_3b_ggml

Model

ggml versions of OpenLLaMa 3B

10 commits

3 linked in READMEs

updated Jun 22, 2023

See the code

README

ggml versions of OpenLLaMa 3B

Version: 1T tokens final version
Project: OpenLLaMA: An Open Reproduction of LLaMA
Model: openlm-research/open_llama_3b
llama.cpp: build 607(ffb06a3) or later

Use with llama.cpp

Support is now merged to master branch.

Newer quantizations

There are now more quantization types in llama.cpp, some lower than 4 bits. Currently these are not supported, maybe because some weights have shapes that don't divide by 256.

Perplexity on wiki.test.raw

Q	chunk	600BT	1000BT
F16	[616]	8.4656	7.7861
Q8_0	[616]	8.4667	7.7874
Q5_1	[616]	8.5072	7.8424
Q5_0	[616]	8.5156	7.8474
Q4_1	[616]	8.6102	8.0483
Q4_0	[616]	8.6674	8.0962

Similar repos

hfl/llama-3-chinese-8b-gguf

Model

Llama-3-Chinese-8B-GGUF

LinkSoul/Chinese-Llama-2-7b-ggml

Model

GGML转换脚本详见:

TheBloke/wizardLM-13B-1.0-GGML

Model

WizardLM's WizardLM 13B 1.0 GGML

leafspark/Meta-Llama-3.1-405B-Instruct-GGUF

Model

Meta-Llama-3.1-405B-Instruct-GGUF

TheBloke/Llama-2-13B-GGML

Model

Llama 2 13B - GGML

173

prism-ml/Ternary-Bonsai-4B-gguf

Model

Ternary-Bonsai-4B-gguf

prism-ml/Ternary-Bonsai-1.7B-gguf

Model

Ternary-Bonsai-1.7B-gguf

hfl/llama-3-chinese-8b-instruct-gguf

Model

Llama-3-Chinese-8B-Instruct-GGUF

Linked in READMEs

openlm-research/open_llama

README badge

ggerganov/llama.cpp

LLM inference in C/C++

129,814

README badge

SciSharp/LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

3,803

README badge

SlyEcho/open_llama_3b_ggml

Model

ggml versions of OpenLLaMa 3B

10 commits

3 linked in READMEs

updated Jun 22, 2023

See the code

README

ggml versions of OpenLLaMa 3B

Version: 1T tokens final version
Project: OpenLLaMA: An Open Reproduction of LLaMA
Model: openlm-research/open_llama_3b
llama.cpp: build 607(ffb06a3) or later

Use with llama.cpp

Support is now merged to master branch.

Newer quantizations

There are now more quantization types in llama.cpp, some lower than 4 bits. Currently these are not supported, maybe because some weights have shapes that don't divide by 256.