Shadowgraph
TrendingBuzzResearch
Chrome extension
Sign in

ISTA-DASLab/Meta-Llama-3-70B-AQLM-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3-70B

14

9 commits

1 linked in READMEs

updated May 4, 2024

See the code

README

Official AQLM quantization of meta-llama/Meta-Llama-3-70B .

For this quantization, we used 1 codebook of 16 bits.

Results (measured with lm_eval==4.0):

ModelQuantizationMMLU (5-shot)ArcCArcEHellaswagWinograndePiQAModel size, Gb
meta-llama/Meta-Llama-3-70B-0.78680.60410.86700.66340.82480.8090141.2
1x160.74860.50090.78960.63460.79540.781421.9
aqlm
conversational
endpoints_compatible
facebook
llama
llama-3
meta
safetensors
text-generation
text-generation-inference
transformers

Similar repos

ISTA-DASLab/Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3-70B-Instruct

20

ISTA-DASLab/Meta-Llama-3-8B-AQLM-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3-8B

8

ISTA-DASLab/Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3.1-70B-Instruct

45

ISTA-DASLab/Meta-Llama-3-8B-Instruct-AQLM-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3-8B-Instruct

12

ISTA-DASLab/Meta-Llama-3.1-8B-AQLM-PV-2Bit-1x16-hf

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3.1-8B

3

ISTA-DASLab/Meta-Llama-3.1-8B-AQLM-PV-1Bit-1x16-hf

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3.1-8B

1

ISTA-DASLab/Meta-Llama-3-70B-AQLM-PV-2Bit-1x16

Model

An official quantization of meta-llama/Meta-Llama-3-70B using PV-Tuning on top of AQLM .

4

ISTA-DASLab/Meta-Llama-3-70B-AQLM-PV-1Bit-1x16

Model

An official quantization of meta-llama/Meta-Llama-3-70B using PV-Tuning on top of AQLM .

1

Linked in READMEs

Vahe1994/AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive…

1,330

README badge

ISTA-DASLab/Meta-Llama-3-70B-AQLM-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3-70B

14

9 commits

1 linked in READMEs

updated May 4, 2024

See the code

README

Official AQLM quantization of meta-llama/Meta-Llama-3-70B .

For this quantization, we used 1 codebook of 16 bits.

Results (measured with lm_eval==4.0):

ModelQuantizationMMLU (5-shot)ArcCArcEHellaswagWinograndePiQAModel size, Gb
meta-llama/Meta-Llama-3-70B-0.78680.60410.86700.66340.82480.8090141.2
1x160.74860.50090.78960.63460.79540.781421.9
aqlm
conversational
endpoints_compatible
facebook
llama
llama-3
meta
safetensors
text-generation
text-generation-inference
transformers

Similar repos

ISTA-DASLab/Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3-70B-Instruct

20

ISTA-DASLab/Meta-Llama-3-8B-AQLM-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3-8B

8

ISTA-DASLab/Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3.1-70B-Instruct

45

ISTA-DASLab/Meta-Llama-3-8B-Instruct-AQLM-2Bit-1x16

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3-8B-Instruct

12

ISTA-DASLab/Meta-Llama-3.1-8B-AQLM-PV-2Bit-1x16-hf

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3.1-8B

3

ISTA-DASLab/Meta-Llama-3.1-8B-AQLM-PV-1Bit-1x16-hf

Model

Official AQLM quantization of [meta-llama/Meta-Llama-3.1-8B

1

ISTA-DASLab/Meta-Llama-3-70B-AQLM-PV-2Bit-1x16

Model

An official quantization of meta-llama/Meta-Llama-3-70B using PV-Tuning on top of AQLM .

4

ISTA-DASLab/Meta-Llama-3-70B-AQLM-PV-1Bit-1x16

Model

An official quantization of meta-llama/Meta-Llama-3-70B using PV-Tuning on top of AQLM .

1

Linked in READMEs

Vahe1994/AQLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive…

1,330

README badge