teknium/airoboros-mistral2.2-7b

Model

stars

commits

Oct 3, 2023

updated

alpaca

distillation

endpoints_compatible

finetune

gpt4

instruct

llama-2

mistral

pytorch

synthetic data

text-generation

text-generation-inference

transformers

Browse cluster: Instruction-tuned LLM model distillation →

README

Mistral trained with the airoboros dataset!

image/png

Actual dataset is airoboros 2.2, but it seems to have been replaced on hf with 2.2.1.

Prompt Format:

USER: <prompt>
ASSISTANT:

TruthfulQA:

hf-causal-experimental (pretrained=/home/teknium/dakota/lm-evaluation-harness/airoboros2.2-mistral/,dtype=float16), limit: None, provide_description: False, num_fewshot: 0, batch_size: 8
|    Task     |Version|Metric|Value |   |Stderr|
|-------------|------:|------|-----:|---|-----:|
|truthfulqa_mc|      1|mc1   |0.3562|±  |0.0168|
|             |       |mc2   |0.5217|±  |0.0156|

Wandb training charts: https://wandb.ai/teknium1/airoboros-mistral-7b/runs/airoboros-mistral-1?workspace=user-teknium1

More info to come

Contributors

teknium

6 commits

Similar repos

teknium/CollectiveCognition-v1.1-Mistral-7B

Model

shared topics

teknium/CollectiveCognition-v1-Mistral-7B

Model

shared topics

teknium/OpenHermes-7B

Model

shared topics

teknium/OpenHermes-2-Mistral-7B

Model

255

shared topics

teknium/OpenHermes-2.5-Mistral-7B

Model

905

shared topics

teknium/OpenHermes-7B-adapter

Model

shared topics

teknium/Hermes-Trismegistus-Mistral-7B

Model

shared topics

teknium/Mistral-Trismegistus-7B

Model

242

shared topics

teknium/airoboros-mistral2.2-7b

Model

stars

commits

Oct 3, 2023

updated

alpaca

distillation

endpoints_compatible

finetune

gpt4

instruct

llama-2

mistral

pytorch

synthetic data

text-generation

text-generation-inference

transformers

Browse cluster: Instruction-tuned LLM model distillation →

README

Mistral trained with the airoboros dataset!

image/png

Actual dataset is airoboros 2.2, but it seems to have been replaced on hf with 2.2.1.

Prompt Format:

USER: <prompt>
ASSISTANT:

TruthfulQA:

hf-causal-experimental (pretrained=/home/teknium/dakota/lm-evaluation-harness/airoboros2.2-mistral/,dtype=float16), limit: None, provide_description: False, num_fewshot: 0, batch_size: 8
|    Task     |Version|Metric|Value |   |Stderr|
|-------------|------:|------|-----:|---|-----:|
|truthfulqa_mc|      1|mc1   |0.3562|±  |0.0168|
|             |       |mc2   |0.5217|±  |0.0156|

Wandb training charts: https://wandb.ai/teknium1/airoboros-mistral-7b/runs/airoboros-mistral-1?workspace=user-teknium1

More info to come