chenzongxiong/nerdme

Jupyter Notebook

2 commits

updated Nov 18, 2025

See the code

README

NERdME Replication Package

check replication file replication-package.ipynb

Performance comparison of various language models :label: tbl-exp-classwise

	Models	CONFERENCE	DATASET	EVALMETRIC	LICENSE	ONTOLOGY	PROGLANG	PROJECT	PUBLICATION	SOFTWARE	WORKSHOP
F1-P.	Mistral-7B	29.08	30.85	6.49	69.50	45.98	10.13	33.27	37.39	10.99	20.48
	Llama3.1-8B	62.61	40.07	34.29	77.33	33.90	9.93	49.62	55.84	29.96	0.00
	Deepseek-Chat	60.62	50.98	37.07	79.68	33.90	29.97	38.10	70.42	28.50	5.29
	Gemini-2.0-Flash	77.38	41.70	28.28	75.61	45.98	34.41	53.72	61.07	15.18	12.99
	GPT-4o-mini	75.48	43.86	19.76	87.34	23.14	18.82	56.23	63.51	22.20	20.68
	SciBERT	78.88	69.94	60.18	86.62	79.43	82.14	38.21	90.16	77.25	0.01
	RoBERTa-Base	82.29	66.04	53.88	84.76	0.00	82.72	47.36	86.99	69.66	13.27
F1-E.	Mistral-7B	17.59	21.51	6.49	66.97	13.89	6.70	4.66	32.05	6.46	20.48
	Llama3.1-8B	33.10	30.85	33.37	72.40	18.52	8.34	49.62	48.55	22.41	0.00
	Deepseek-Chat	36.68	45.30	33.94	77.73	18.52	27.37	38.10	64.95	23.12	0.00
	Gemini-2.0-Flash	47.63	38.03	26.97	71.12	13.89	31.69	53.72	58.99	13.48	5.41
	GPT-4o-mini	42.44	37.11	19.76	86.15	0.00	13.75	48.21	54.14	18.68	20.68
	SciBERT	58.38	63.94	44.78	51.87	73.13	70.99	34.22	82.12	72.46	0.00
	RoBERTa-Base	35.84	58.95	35.01	78.38	0.00	79.96	33.26	78.11	64.48	0.00

Contributors

chenzongxiong

2 commits

Similar repos

mtkresearch/RAD-Bench

RAD-Bench: A Benchmark for Evaluating Large Language Models with Context Utilization in Multi-Turn…

McGill-NLP/LLM2Vec-Llama-2-7b-chat-hf-mntp-supervised

Model

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp-supervised

Model

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

LiquidAI/LFM2-ColBERT-350M

Model

src="https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6…

153

McGill-NLP/LLM2Vec-Llama-2-7b-chat-hf-unsup-simcse

Model

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-unsup-simcse

Model

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-supervised

Model

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

JoelNiklaus/LEXTREME

This repository provides scripts for evaluating NLP models on the LEXTREME benchmark, a set of…

Languages

Jupyter Notebook

77.3%

Python

22.3%

chenzongxiong/nerdme

Jupyter Notebook

2 commits

updated Nov 18, 2025

See the code

README

NERdME Replication Package

check replication file replication-package.ipynb

Performance comparison of various language models :label: tbl-exp-classwise

	Models	CONFERENCE	DATASET	EVALMETRIC	LICENSE	ONTOLOGY	PROGLANG	PROJECT	PUBLICATION	SOFTWARE	WORKSHOP
F1-P.	Mistral-7B	29.08	30.85	6.49	69.50	45.98	10.13	33.27	37.39	10.99	20.48
	Llama3.1-8B	62.61	40.07	34.29	77.33	33.90	9.93	49.62	55.84	29.96	0.00
	Deepseek-Chat	60.62	50.98	37.07	79.68	33.90	29.97	38.10	70.42	28.50	5.29
	Gemini-2.0-Flash	77.38	41.70	28.28	75.61	45.98	34.41	53.72	61.07	15.18	12.99
	GPT-4o-mini	75.48	43.86	19.76	87.34	23.14	18.82	56.23	63.51	22.20	20.68
	SciBERT	78.88	69.94	60.18	86.62	79.43	82.14	38.21	90.16	77.25	0.01
	RoBERTa-Base	82.29	66.04	53.88	84.76	0.00	82.72	47.36	86.99	69.66	13.27
F1-E.	Mistral-7B	17.59	21.51	6.49	66.97	13.89	6.70	4.66	32.05	6.46	20.48
	Llama3.1-8B	33.10	30.85	33.37	72.40	18.52	8.34	49.62	48.55	22.41	0.00
	Deepseek-Chat	36.68	45.30	33.94	77.73	18.52	27.37	38.10	64.95	23.12	0.00
	Gemini-2.0-Flash	47.63	38.03	26.97	71.12	13.89	31.69	53.72	58.99	13.48	5.41
	GPT-4o-mini	42.44	37.11	19.76	86.15	0.00	13.75	48.21	54.14	18.68	20.68
	SciBERT	58.38	63.94	44.78	51.87	73.13	70.99	34.22	82.12	72.46	0.00
	RoBERTa-Base	35.84	58.95	35.01	78.38	0.00	79.96	33.26	78.11	64.48	0.00