| | Wiki | C4 | PIQA | ARC-E | ARC-C | HellaSwag | Wino | Avg. |
0
5 commits
2 linked in READMEs
updated Aug 4, 2024
README
| Wiki | C4 | PIQA | ARC-E | ARC-C | HellaSwag | Wino | Avg. | |
|---|---|---|---|---|---|---|---|---|
| Unquantized | 6.31 | 9.60 | 79.16 | 82.20 | 52.65 | 60.71 | 74.03 | 69.75 |
| W4G64 | 6.24 | 10.06 | 79.38 | 81.61 | 51.54 | 59.57 | 73.56 | 69.13 |
| W3G64 | 7.23 | 11.83 | 77.91 | 76.98 | 46.33 | 56.74 | 70.32 | 65.66 |
Revisions available in this repository:
main(W4G64, scales learned);nfl_w3g64(W3G64, scales learned);
Evaluations are provided for models with learned scales.
Benchmark scores (zero-shot) are computed with lm-evaluation-harness.
llama
safetensors
| | Wiki | C4 | PIQA | ARC-E | ARC-C | HellaSwag | Wino | Avg. |
0
5 commits
2 linked in READMEs
updated Aug 4, 2024
README
| Wiki | C4 | PIQA | ARC-E | ARC-C | HellaSwag | Wino | Avg. | |
|---|---|---|---|---|---|---|---|---|
| Unquantized | 6.31 | 9.60 | 79.16 | 82.20 | 52.65 | 60.71 | 74.03 | 69.75 |
| W4G64 | 6.24 | 10.06 | 79.38 | 81.61 | 51.54 | 59.57 | 73.56 | 69.13 |
| W3G64 | 7.23 | 11.83 | 77.91 | 76.98 | 46.33 | 56.74 | 70.32 | 65.66 |
Revisions available in this repository:
main(W4G64, scales learned);nfl_w3g64(W3G64, scales learned);
Evaluations are provided for models with learned scales.
Benchmark scores (zero-shot) are computed with lm-evaluation-harness.
llama
safetensors