10
stars
20
commits
1
linked in READMEs
Jan 24, 2024
updated
README
Chinese-Alpaca-2-13B-GGUF
This repository contains the GGUF-v3 models (llama.cpp compatible) for Chinese-Alpaca-2-13B.
Performance
Metric: PPL, lower is better
| Quant | original | imatrix (-im) |
|---|---|---|
| Q2_K | 13.7636 +/- 0.19446 | 20.6803 +/- 0.31594 |
| Q3_K | 9.5388 +/- 0.13078 | 9.1016 +/- 0.12565 |
| Q4_0 | 9.1694 +/- 0.12668 | - |
| Q4_K | 8.6633 +/- 0.11957 | 8.6377 +/- 0.11932 |
| Q5_0 | 8.6745 +/- 0.12020 | - |
| Q5_K | 8.5161 +/- 0.11796 | 8.5210 +/- 0.11803 |
| Q6_K | 8.4943 +/- 0.11759 | 8.5011 +/- 0.11775 |
| Q8_0 | 8.4595 +/- 0.11718 | - |
| F16 | 8.4550 +/- 0.11713 | - |
The model with -im suffix is generated with important matrix, which has generally better performance (not always though).
Others
For Hugging Face version, please see: https://huggingface.co/hfl/chinese-alpaca-2-13b
Please refer to https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/ for more details.
Contributors
20 commits
Similar repos
10
stars
20
commits
1
linked in READMEs
Jan 24, 2024
updated
README
Chinese-Alpaca-2-13B-GGUF
This repository contains the GGUF-v3 models (llama.cpp compatible) for Chinese-Alpaca-2-13B.
Performance
Metric: PPL, lower is better
| Quant | original | imatrix (-im) |
|---|---|---|
| Q2_K | 13.7636 +/- 0.19446 | 20.6803 +/- 0.31594 |
| Q3_K | 9.5388 +/- 0.13078 | 9.1016 +/- 0.12565 |
| Q4_0 | 9.1694 +/- 0.12668 | - |
| Q4_K | 8.6633 +/- 0.11957 | 8.6377 +/- 0.11932 |
| Q5_0 | 8.6745 +/- 0.12020 | - |
| Q5_K | 8.5161 +/- 0.11796 | 8.5210 +/- 0.11803 |
| Q6_K | 8.4943 +/- 0.11759 | 8.5011 +/- 0.11775 |
| Q8_0 | 8.4595 +/- 0.11718 | - |
| F16 | 8.4550 +/- 0.11713 | - |
The model with -im suffix is generated with important matrix, which has generally better performance (not always though).
Others
For Hugging Face version, please see: https://huggingface.co/hfl/chinese-alpaca-2-13b
Please refer to https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/ for more details.
Similar repos
Linked in READMEs
Contributors
20 commits