USTC-KnowledgeComputingLab/Llama3-KALE-LM-Chem-1.5-8B

Model

Llama3-KALE-LM-Chem-1.5-8B

8 commits

1 linked in READMEs

updated Apr 7, 2025

See the code

README

Llama3-KALE-LM-Chem-1.5-8B

Introduction

We are thrilled to present Llama3-KALE-LM-Chem-1.5-8B, a new version of our open-source KALE-LM for science, which specializes in chemistry.

We have trained our model with a larger amount of data.

Benchmarks

Open Benchmarks

Models	ChemBench	MMLU	MMLU-Chem	SciQ	IE(Acc)	IE(LS)
GPT-3.5	47.15	69.75	53.32	89.6	52.98	68.28
GPT-4	53.72	78.67	63.70	94.10	54.20	69.74
Llama3-8B-Instruct	46.02	68.3	51.10	93.30	45.83	61.22
LlaSMol	28.47	54.47	33.24	72.30	2.16	3.23
ChemDFM	44.44	58.11	45.60	86.70	7.61	11.49
ChemLLM-7B-Chat	34.16	61.79	48.39	94.00	29.66	39.17
ChemLLM-7B-Chat-1.5-SFT	42.75	63.56	49.63	95.10	14.96	19.61
Llama3-KALE-LM-Chem-1.5-8B	57.01	68.06	54.83	91.60	71.70	81.98

ChemBench Details (Evaluated By OpenCompass)

Models	NC	PP	M2C	C2M	PP	RS	YP	TP	SP	Average
GPT-3.5	46.93	56.98	85.28	38.25	43.67	42.33	30.33	42.57	38	47.15
GPT-4	54.82	65.02	92.64	52.88	62.67	52.67	42.33	24.75	35.67	53.72
Llama3-8B-Instruct	51.31	27.79	90.30	40.88	34.00	30.00	45.33	60.89	33.67	46.02
LlaSMol	27.78	29.34	31.44	23.38	25.67	24.00	37.33	34.65	22.67	28.47
ChemDFM	36.92	55.57	83.95	42.00	40.00	37.33	39.00	33.17	32.00	44.44
ChemLLM-7B-Chat	41.05	29.76	85.28	26.12	26.00	24.00	20.00	24.26	31.00	34.16
ChemLLM-7B-Chat-1.5-SFT	50.06	49.51	85.28	38.75	38.00	26.67	28.33	31.68	33.67	42.44
Llama3-KALE-LM-Chem-1.5-8B	61.33	43.44	90.30	53.62	72.67	53.67	46.00	47.03	45.00	57.01

Cite This Work

@article{dai2024kale,
  title={KALE-LM: Unleash The Power Of AI For Science Via Knowledge And Logic Enhanced Large Model},
  author={Dai, Weichen and Chen, Yezeng and Dai, Zijie and Huang, Zhijie and Liu, Yubo and Pan, Yixuan and Song, Baiyang and Zhong, Chengli and Li, Xinhe and Wang, Zeyu and others},
  journal={arXiv preprint arXiv:2409.18695},
  year={2024}
}

chemistry

conversational

KALE-LM

llama

safetensors

science

text-generation

Contributors

WeeeicheN

6 commits

KnowledgeComputingLab

2 commits

Similar repos

AI4Chem/ChemLLM-7B-Chat

Model

ChemLLM-7B-Chat: LLM for Chemistry and Molecule Science

OpenDFM/ChemDFM-13B-v1.0

Model

ChemDFM: A Large Language Foundation Model for Chemistry

OpenDFM/ChemDFM-v1.0-13B

Model

ChemDFM: A Large Language Foundation Model for Chemistry

OpenDFM/ChemDFM

[Cell Reports Physical Science] Open-sourced dialogue foundation model for Chemistry and molecule…

110

OpenDFM/ChemDFM-v1.5-8B

Model

ChemDFM: A Large Language Foundation Model for Chemistry

lamalab-org/chembench

How good are LLMs at chemistry?

147

osunlp/LlaSMol-Llama2-7B

Model

Paper: LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive,…

osunlp/LlaSMol-CodeLlama-7B

Model

Paper: LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive,…

Linked in READMEs

yuzhimanhua/Awesome-Scientific-Language-Models

A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific…

665

README badge

USTC-KnowledgeComputingLab/Llama3-KALE-LM-Chem-1.5-8B

Model

Llama3-KALE-LM-Chem-1.5-8B

8 commits

1 linked in READMEs

updated Apr 7, 2025

See the code

README

Llama3-KALE-LM-Chem-1.5-8B

Introduction

We are thrilled to present Llama3-KALE-LM-Chem-1.5-8B, a new version of our open-source KALE-LM for science, which specializes in chemistry.

We have trained our model with a larger amount of data.

Benchmarks

Open Benchmarks

Models	ChemBench	MMLU	MMLU-Chem	SciQ	IE(Acc)	IE(LS)
GPT-3.5	47.15	69.75	53.32	89.6	52.98	68.28
GPT-4	53.72	78.67	63.70	94.10	54.20	69.74
Llama3-8B-Instruct	46.02	68.3	51.10	93.30	45.83	61.22
LlaSMol	28.47	54.47	33.24	72.30	2.16	3.23
ChemDFM	44.44	58.11	45.60	86.70	7.61	11.49
ChemLLM-7B-Chat	34.16	61.79	48.39	94.00	29.66	39.17
ChemLLM-7B-Chat-1.5-SFT	42.75	63.56	49.63	95.10	14.96	19.61
Llama3-KALE-LM-Chem-1.5-8B	57.01	68.06	54.83	91.60	71.70	81.98

ChemBench Details (Evaluated By OpenCompass)

Models	NC	PP	M2C	C2M	PP	RS	YP	TP	SP	Average
GPT-3.5	46.93	56.98	85.28	38.25	43.67	42.33	30.33	42.57	38	47.15
GPT-4	54.82	65.02	92.64	52.88	62.67	52.67	42.33	24.75	35.67	53.72
Llama3-8B-Instruct	51.31	27.79	90.30	40.88	34.00	30.00	45.33	60.89	33.67	46.02
LlaSMol	27.78	29.34	31.44	23.38	25.67	24.00	37.33	34.65	22.67	28.47
ChemDFM	36.92	55.57	83.95	42.00	40.00	37.33	39.00	33.17	32.00	44.44
ChemLLM-7B-Chat	41.05	29.76	85.28	26.12	26.00	24.00	20.00	24.26	31.00	34.16
ChemLLM-7B-Chat-1.5-SFT	50.06	49.51	85.28	38.75	38.00	26.67	28.33	31.68	33.67	42.44
Llama3-KALE-LM-Chem-1.5-8B	61.33	43.44	90.30	53.62	72.67	53.67	46.00	47.03	45.00	57.01

Cite This Work

@article{dai2024kale,
  title={KALE-LM: Unleash The Power Of AI For Science Via Knowledge And Logic Enhanced Large Model},
  author={Dai, Weichen and Chen, Yezeng and Dai, Zijie and Huang, Zhijie and Liu, Yubo and Pan, Yixuan and Song, Baiyang and Zhong, Chengli and Li, Xinhe and Wang, Zeyu and others},
  journal={arXiv preprint arXiv:2409.18695},
  year={2024}
}

chemistry

conversational

KALE-LM

llama

safetensors

science

text-generation