Shadowgraph
About
Sign in

mattdax/544-nlp-project

1

stars

24

commits

Jupyter Notebook

primary language

Dec 18, 2024

updated

README

544-nlp-project

Evaluate models

Some baseline models to try:

  • defog/sqlcoder-7b-2
  • seeklhy/codes-1b
  • seeklhy/codes-7b
  • seeklhy/codes-7b-merged

Running on CARC:

cd baseline_models
sbatch evaluate_model.sh "model_name"

Running locally (GPU recommended):

cd baseline_models
./evaluate_model_local.sh "model_name"

Contributors

mattdax

13 commits

co-li

9 commits

evanshiu

2 commits

Similar repos

darvinrio/local-eval

0

Beomi/ko-lm-evaluation-harness

Forked repo from https://github.com/EleutherAI/lm-evaluation-harness/commit/1f66adc

81

eval4nlp/SharedTask2023

11

webis-de/EMNLP-24

0

w32zhong/LLM-common-eval

Evaluating / benchmarking common NLP datasets using LLMs.

0

mtkresearch/TCEval

2

cy0307/lm-eval-harness

Model

0

vianandi/S2-AI-NLP-with-BERT

0

Languages

Jupyter Notebook

99.5%

mattdax/544-nlp-project

1

stars

24

commits

Jupyter Notebook

primary language

Dec 18, 2024

updated

README

544-nlp-project

Evaluate models

Some baseline models to try:

  • defog/sqlcoder-7b-2
  • seeklhy/codes-1b
  • seeklhy/codes-7b
  • seeklhy/codes-7b-merged

Running on CARC:

cd baseline_models
sbatch evaluate_model.sh "model_name"

Running locally (GPU recommended):

cd baseline_models
./evaluate_model_local.sh "model_name"

Similar repos

darvinrio/local-eval

0

Beomi/ko-lm-evaluation-harness

Forked repo from https://github.com/EleutherAI/lm-evaluation-harness/commit/1f66adc

81

eval4nlp/SharedTask2023

11

webis-de/EMNLP-24

0

w32zhong/LLM-common-eval

Evaluating / benchmarking common NLP datasets using LLMs.

0

mtkresearch/TCEval

2

cy0307/lm-eval-harness

Model

0

vianandi/S2-AI-NLP-with-BERT

0

Contributors

mattdax

13 commits

co-li

9 commits

evanshiu

2 commits

Languages

Jupyter Notebook

99.5%