Shadowgraph
TrendingBuzzResearch
Chrome extension
Sign in

ISTA-DASLab/Llama-2-13b-AQLM-2Bit-2x8-hf

Model

Official AQLM quantization of meta-llama/Llama-2-13b-hf.

0

4 commits

1 linked in READMEs

updated Jun 4, 2024

See the code

README

Official AQLM quantization of meta-llama/Llama-2-13b-hf.

For this quantization, we used 2 codebook of 8 bits.

Selected evaluation results for this and other models:

ModelQuantizationWikiText 2 PPLModel size, Gb
Llama-2-13b-4.5726.0
2x85.633.8
8-bit
aqlm
endpoints_compatible
llama
safetensors
text-generation
text-generation-inference
transformers

Similar repos

ISTA-DASLab/Llama-2-13b-AQLM-2Bit-1x16-hf

Model

Official AQLM quantization of meta-llama/Llama-2-13b-hf.

0

ISTA-DASLab/Llama-2-7b-AQLM-2Bit-8x8-hf

Model

Official AQLM quantization of meta-llama/Llama-2-7b-hf.

0

ISTA-DASLab/Llama-2-70b-AQLM-2Bit-2x8-hf

Model

Official AQLM quantization of meta-llama/Llama-2-7b-hf.

1

ISTA-DASLab/Llama-2-7b-AQLM-2Bit-2x8-hf

Model

Official AQLM quantization of meta-llama/Llama-2-7b-hf.

2

ISTA-DASLab/Llama-2-70b-AQLM-2Bit-1x16-hf

Model

Official AQLM quantization of meta-llama/Llama-2-70b-hf.

6

ISTA-DASLab/Llama-2-7b-AQLM-2Bit-1x16-hf

Model

Official AQLM quantization of meta-llama/Llama-2-7b-hf.

5

ISTA-DASLab/Llama-2-7b-AQLM-PV-2Bit-2x8-hf

Model

An official quantization of meta-llama/Llama-2-7b using PV-Tuning on top of AQLM.

0

ISTA-DASLab/Llama-2-13b-AQLM-PV-2Bit-1x16-hf

Model

An official quantization of meta-llama/Llama-2-13b using PV-Tuning on top of AQLM.

0

Linked in READMEs

Vahe1994/AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive…

1,330

README badge

ISTA-DASLab/Llama-2-13b-AQLM-2Bit-2x8-hf

Model

Official AQLM quantization of meta-llama/Llama-2-13b-hf.

0

4 commits

1 linked in READMEs

updated Jun 4, 2024

See the code

README

Official AQLM quantization of meta-llama/Llama-2-13b-hf.

For this quantization, we used 2 codebook of 8 bits.

Selected evaluation results for this and other models:

ModelQuantizationWikiText 2 PPLModel size, Gb
Llama-2-13b-4.5726.0
2x85.633.8
8-bit
aqlm
endpoints_compatible
llama
safetensors
text-generation
text-generation-inference
transformers

Similar repos

ISTA-DASLab/Llama-2-13b-AQLM-2Bit-1x16-hf

Model

Official AQLM quantization of meta-llama/Llama-2-13b-hf.

0

ISTA-DASLab/Llama-2-7b-AQLM-2Bit-8x8-hf

Model

Official AQLM quantization of meta-llama/Llama-2-7b-hf.

0

ISTA-DASLab/Llama-2-70b-AQLM-2Bit-2x8-hf

Model

Official AQLM quantization of meta-llama/Llama-2-7b-hf.

1

ISTA-DASLab/Llama-2-7b-AQLM-2Bit-2x8-hf

Model

Official AQLM quantization of meta-llama/Llama-2-7b-hf.

2

ISTA-DASLab/Llama-2-70b-AQLM-2Bit-1x16-hf

Model

Official AQLM quantization of meta-llama/Llama-2-70b-hf.

6

ISTA-DASLab/Llama-2-7b-AQLM-2Bit-1x16-hf

Model

Official AQLM quantization of meta-llama/Llama-2-7b-hf.

5

ISTA-DASLab/Llama-2-7b-AQLM-PV-2Bit-2x8-hf

Model

An official quantization of meta-llama/Llama-2-7b using PV-Tuning on top of AQLM.

0

ISTA-DASLab/Llama-2-13b-AQLM-PV-2Bit-1x16-hf

Model

An official quantization of meta-llama/Llama-2-13b using PV-Tuning on top of AQLM.

0

Linked in READMEs

Vahe1994/AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive…

1,330

README badge