crumb/askmistral-2-15-tophalf-111m

Model

Model Card for Model ID

6 commits

1 linked in READMEs

updated Mar 1, 2024

See the code

README

Model Card for Model ID

Model Details

Model Description

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
arc_challenge	1	none	25	acc	0.1775	±	0.0112
		none	25	acc_norm	0.2150	±	0.0120
truthfulqa_mc2	2	none	0	acc	0.4571	±	0.0154
winogrande	1	none	5	acc	0.5107	±	0.014
hellaswag	1	none	10	acc	0.2735	±	0.0044
		none	10	acc_norm	0.2801	±	0.0045
gsm8k	3	strict-match	5	exact_match	0.0030	±	0.0015
		flexible-extract	5	exact_match	0.0099	±	0.0027

(0.25676491228070175, 0.004430608580958243)

Tasks	Filter	n-shot	Metric	Value		Stderr
world_religions	none	5	acc	0.2281	±	0.0322
virology	none	5	acc	0.2289	±	0.0327
us_foreign_policy	none	5	acc	0.3000	±	0.0461
sociology	none	5	acc	0.2438	±	0.0304
security_studies	none	5	acc	0.2327	±	0.0270
public_relations	none	5	acc	0.2091	±	0.0390
professional_psychology	none	5	acc	0.2516	±	0.0176
professional_medicine	none	5	acc	0.4522	±	0.0302
professional_law	none	5	acc	0.2484	±	0.0110
professional_accounting	none	5	acc	0.2518	±	0.0259
prehistory	none	5	acc	0.2654	±	0.0246
philosophy	none	5	acc	0.2315	±	0.0240
nutrition	none	5	acc	0.2059	±	0.0232
moral_scenarios	none	5	acc	0.2380	±	0.0142
moral_disputes	none	5	acc	0.2486	±	0.0233
miscellaneous	none	5	acc	0.2874	±	0.0162
medical_genetics	none	5	acc	0.2900	±	0.0456
marketing	none	5	acc	0.2009	±	0.0262
management	none	5	acc	0.1845	±	0.0384
machine_learning	none	5	acc	0.2857	±	0.0429
logical_fallacies	none	5	acc	0.3190	±	0.0366
jurisprudence	none	5	acc	0.2685	±	0.0428
international_law	none	5	acc	0.2149	±	0.0375
human_sexuality	none	5	acc	0.2137	±	0.0360
human_aging	none	5	acc	0.2466	±	0.0289
high_school_world_history	none	5	acc	0.2616	±	0.0286
high_school_us_history	none	5	acc	0.2402	±	0.0300
high_school_statistics	none	5	acc	0.4722	±	0.0340
high_school_psychology	none	5	acc	0.2128	±	0.0175
high_school_physics	none	5	acc	0.2781	±	0.0366
high_school_microeconomics	none	5	acc	0.3067	±	0.0300
high_school_mathematics	none	5	acc	0.2630	±	0.0268
high_school_macroeconomics	none	5	acc	0.2590	±	0.0222
high_school_government_and_politics	none	5	acc	0.3005	±	0.0331
high_school_geography	none	5	acc	0.3030	±	0.0327
high_school_european_history	none	5	acc	0.2667	±	0.0345
high_school_computer_science	none	5	acc	0.2900	±	0.0456
high_school_chemistry	none	5	acc	0.2956	±	0.0321
high_school_biology	none	5	acc	0.2871	±	0.0257
global_facts	none	5	acc	0.2600	±	0.0441
formal_logic	none	5	acc	0.1667	±	0.0333
elementary_mathematics	none	5	acc	0.2566	±	0.0225
electrical_engineering	none	5	acc	0.2414	±	0.0357
econometrics	none	5	acc	0.2719	±	0.0419
conceptual_physics	none	5	acc	0.3319	±	0.0308
computer_security	none	5	acc	0.2000	±	0.0402
college_physics	none	5	acc	0.1863	±	0.0387
college_medicine	none	5	acc	0.1965	±	0.0303
college_mathematics	none	5	acc	0.2700	±	0.0446
college_computer_science	none	5	acc	0.2300	±	0.0423
college_chemistry	none	5	acc	0.2000	±	0.0402
college_biology	none	5	acc	0.2431	±	0.0359
clinical_knowledge	none	5	acc	0.2226	±	0.0256
business_ethics	none	5	acc	0.2100	±	0.0409
astronomy	none	5	acc	0.1842	±	0.0315
anatomy	none	5	acc	0.3407	±	0.0409
abstract_algebra	none	5	acc	0.2400	±	0.0429

Developed by: me
Funded by [optional]: nobody
Shared by [optional]: me
Model type: mistral
Language(s) (NLP): english (pile)
License: apache
Finetuned from model [optional]: none

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

endpoints_compatible

mistral

safetensors

text-generation

text-generation-inference

transformers

Similar repos

crumb/minipile-111m

Model

Model Card for Model ID

MilaWang/Mistral-7B-Instruct-v0.2-gad-slianogram3-merged

Model

Model Card for Model ID

MilaWang/Mistral-7B-Instruct-v0.2-gad-cp8-merged

Model

Model Card for Model ID

MilaWang/Mistral-7B-Instruct-v0.2-gad-bv4nogram3-merged

Model

Model Card for Model ID

HINT-lab/mistral-7b-hermes-rm-skywork

Model

Model Card for Model ID

HINT-lab/mistral-7b-hermes-cdpo-v0.2

Model

Model Card for Model ID

HINT-lab/mistral-7b-hermes-dpo-v0.2

Model

Model Card for Model ID

HINT-lab/mistral-7b-ppo-clean-hermes

Model

Model Card for Model ID

Linked in READMEs

aicrumb/aicrumb

README badge

crumb/askmistral-2-15-tophalf-111m

Model

Model Card for Model ID

6 commits

1 linked in READMEs

updated Mar 1, 2024

See the code

README

Model Card for Model ID

Model Details

Model Description

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
arc_challenge	1	none	25	acc	0.1775	±	0.0112
		none	25	acc_norm	0.2150	±	0.0120
truthfulqa_mc2	2	none	0	acc	0.4571	±	0.0154
winogrande	1	none	5	acc	0.5107	±	0.014
hellaswag	1	none	10	acc	0.2735	±	0.0044
		none	10	acc_norm	0.2801	±	0.0045
gsm8k	3	strict-match	5	exact_match	0.0030	±	0.0015
		flexible-extract	5	exact_match	0.0099	±	0.0027

(0.25676491228070175, 0.004430608580958243)

Tasks	Filter	n-shot	Metric	Value		Stderr
world_religions	none	5	acc	0.2281	±	0.0322
virology	none	5	acc	0.2289	±	0.0327
us_foreign_policy	none	5	acc	0.3000	±	0.0461
sociology	none	5	acc	0.2438	±	0.0304
security_studies	none	5	acc	0.2327	±	0.0270
public_relations	none	5	acc	0.2091	±	0.0390
professional_psychology	none	5	acc	0.2516	±	0.0176
professional_medicine	none	5	acc	0.4522	±	0.0302
professional_law	none	5	acc	0.2484	±	0.0110
professional_accounting	none	5	acc	0.2518	±	0.0259
prehistory	none	5	acc	0.2654	±	0.0246
philosophy	none	5	acc	0.2315	±	0.0240
nutrition	none	5	acc	0.2059	±	0.0232
moral_scenarios	none	5	acc	0.2380	±	0.0142
moral_disputes	none	5	acc	0.2486	±	0.0233
miscellaneous	none	5	acc	0.2874	±	0.0162
medical_genetics	none	5	acc	0.2900	±	0.0456
marketing	none	5	acc	0.2009	±	0.0262
management	none	5	acc	0.1845	±	0.0384
machine_learning	none	5	acc	0.2857	±	0.0429
logical_fallacies	none	5	acc	0.3190	±	0.0366
jurisprudence	none	5	acc	0.2685	±	0.0428
international_law	none	5	acc	0.2149	±	0.0375
human_sexuality	none	5	acc	0.2137	±	0.0360
human_aging	none	5	acc	0.2466	±	0.0289
high_school_world_history	none	5	acc	0.2616	±	0.0286
high_school_us_history	none	5	acc	0.2402	±	0.0300
high_school_statistics	none	5	acc	0.4722	±	0.0340
high_school_psychology	none	5	acc	0.2128	±	0.0175
high_school_physics	none	5	acc	0.2781	±	0.0366
high_school_microeconomics	none	5	acc	0.3067	±	0.0300
high_school_mathematics	none	5	acc	0.2630	±	0.0268
high_school_macroeconomics	none	5	acc	0.2590	±	0.0222
high_school_government_and_politics	none	5	acc	0.3005	±	0.0331
high_school_geography	none	5	acc	0.3030	±	0.0327
high_school_european_history	none	5	acc	0.2667	±	0.0345
high_school_computer_science	none	5	acc	0.2900	±	0.0456
high_school_chemistry	none	5	acc	0.2956	±	0.0321
high_school_biology	none	5	acc	0.2871	±	0.0257
global_facts	none	5	acc	0.2600	±	0.0441
formal_logic	none	5	acc	0.1667	±	0.0333
elementary_mathematics	none	5	acc	0.2566	±	0.0225
electrical_engineering	none	5	acc	0.2414	±	0.0357
econometrics	none	5	acc	0.2719	±	0.0419
conceptual_physics	none	5	acc	0.3319	±	0.0308
computer_security	none	5	acc	0.2000	±	0.0402
college_physics	none	5	acc	0.1863	±	0.0387
college_medicine	none	5	acc	0.1965	±	0.0303
college_mathematics	none	5	acc	0.2700	±	0.0446
college_computer_science	none	5	acc	0.2300	±	0.0423
college_chemistry	none	5	acc	0.2000	±	0.0402
college_biology	none	5	acc	0.2431	±	0.0359
clinical_knowledge	none	5	acc	0.2226	±	0.0256
business_ethics	none	5	acc	0.2100	±	0.0409
astronomy	none	5	acc	0.1842	±	0.0315
anatomy	none	5	acc	0.3407	±	0.0409
abstract_algebra	none	5	acc	0.2400	±	0.0429

Developed by: me
Funded by [optional]: nobody
Shared by [optional]: me
Model type: mistral
Language(s) (NLP): english (pile)
License: apache
Finetuned from model [optional]: none

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

endpoints_compatible

mistral

safetensors

text-generation

text-generation-inference

transformers

Similar repos

crumb/minipile-111m

Model

Model Card for Model ID

MilaWang/Mistral-7B-Instruct-v0.2-gad-slianogram3-merged

Model

Model Card for Model ID

MilaWang/Mistral-7B-Instruct-v0.2-gad-cp8-merged

Model

Model Card for Model ID

MilaWang/Mistral-7B-Instruct-v0.2-gad-bv4nogram3-merged

Model

Model Card for Model ID

HINT-lab/mistral-7b-hermes-rm-skywork

Model

Model Card for Model ID

HINT-lab/mistral-7b-hermes-cdpo-v0.2

Model

Model Card for Model ID

HINT-lab/mistral-7b-hermes-dpo-v0.2

Model

Model Card for Model ID

HINT-lab/mistral-7b-ppo-clean-hermes

Model

Model Card for Model ID

Linked in READMEs

aicrumb/aicrumb

README badge