Shadowgraph
TrendingBuzzResearch
Chrome extension
Sign in

Troyanovsky/Local-LLM-Comparison-Colab-UI

Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.

Jupyter Notebook

1,107

418 commits

updated Jan 13, 2026

See the code

README

Local LLM Comparison & Colab Links (WIP)

(Update Nov. 27, 2023) The original goal of the repo was to compare some smaller models (7B and 13B) that can be run on consumer hardware so every model had a score for a set of questions from GPT-4. But I realized that as there are many more capable models appearing, the evaluation and comparison process may not suffice.

Therefore, I'm only putting Colab WebUI links for the newer models and you can try them out yourselves with a few clicks - after all, the effectiveness of a language model relies heavily on its suitability for your specific use case. By trying out the models firsthand, you can assess their performance and determine which one best fits your needs.

Newer Models without Scores:

These models can be run on consumer hardware and are generally good (from Reddit suggestions and my own experience). Try them out yourselves (tap the 'Open in Colab' button)!

ModelColab LinkDate AddedNotesLink
OpenHermes-2.5-Mistral-7B-GGUFOpen In Colab2023-11-27Roleplay good, not censoredhttps://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF
zephyr-7B-beta-GGUFOpen In Colab2023-11-27Roleplay okay, not censoredhttps://huggingface.co/TheBloke/zephyr-7B-beta-GGUF
dolphin-2.2.1-mistral-7B-GGUFOpen In Colab2023-11-27Roleplay okay, not censoredhttps://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF
neural-chat-7B-v3-1-GGUFOpen In Colab2023-11-27Roleplay okay, not censored; some logic flawshttps://huggingface.co/TheBloke/neural-chat-7B-v3-1-GGUF
openchat_3.5-16k-GGUFOpen In Colab2023-11-27Censoredhttps://huggingface.co/TheBloke/openchat_3.5-16k-GGUF
Starling-LM-7B-alpha-GGUFOpen In Colab2023-11-29Censored; highly rated on Reddithttps://huggingface.co/TheBloke/Starling-LM-7B-alpha-GGUF
Orca-2-7B-GGUFOpen In Colab2023-11-29Censoredhttps://huggingface.co/TheBloke/Orca-2-7B-GGUF
Orca-2-13B-GGUF Open In Colab2023-11-29Censored, some weird logic flaws worse than 7B versionhttps://huggingface.co/TheBloke/Orca-2-13B-GGUF
MythoMist-7B-GGUFOpen In Colab2023-11-29Roleplay okay, not censored; some logic flawshttps://huggingface.co/TheBloke/MythoMist-7B-GGUF
NeuralHermes-2.5-Mistral-7B-GGUFOpen In Colab2023-12-05Roleplay good, not censoredhttps://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GGUF
stablelm-zephyr-3b-GGUFOpen In Colab2023-12-113B; roleplay ok; not censored; some logic flawshttps://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF
deepseek-llm-7B-chat-GGUFOpen In Colab2023-12-11Censoredhttps://huggingface.co/TheBloke/deepseek-llm-7B-chat-GGUF
Mistral-7B-Instruct-v0.2-GGUFOpen In Colab2023-12-13Partially censored; role play ok; highly rated on Reddithttps://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF
Mixtral-8x7B-Instruct-v0.1-GGUFOpen In Colab2023-12-13MOE model; partially censored; role play okhttps://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF
deepsex-34b-GGUFOpen In Colab2023-12-1434B; NSFW modelhttps://huggingface.co/TheBloke/deepsex-34b-GGUF
phi-2Open In Colab2023-12-152.7B; Base model;https://huggingface.co/microsoft/phi-2
Xwin-MLewd-13B-v0.2-GGUFOpen In Colab2023-12-1513B; NSFW modelhttps://huggingface.co/TheBloke/Xwin-MLewd-13B-v0.2-GGUF
MythoMax-L2-13B-GGUFOpen In Colab2023-12-1513B; Censored; role play okhttps://huggingface.co/TheBloke/MythoMax-L2-13B-GGUF
LLaMA2-13B-Tiefighter-GGUFOpen In Colab2023-12-1513B; role play goodhttps://huggingface.co/TheBloke/LLaMA2-13B-Tiefighter-GGUF
LLaMA2-13B-Psyfighter2-GGUFOpen In Colab2023-12-1513B; Partially censored; role play ok; recommended on Reddithttps://huggingface.co/TheBloke/LLaMA2-13B-Psyfighter2-GGUF
Noromaid-13B-v0.1.1-GGUFOpen In Colab2023-12-1513B; NSFW model; roleplay goodhttps://huggingface.co/TheBloke/Noromaid-13B-v0.1.1-GGUF
dolphin-2.5-mixtral-8x7b-GGUFOpen In Colab2023-12-20MOE model; claims to be uncensored but denies some requests; role play not usablehttps://huggingface.co/TheBloke/dolphin-2.5-mixtral-8x7b-GGUF
SOLAR-10.7B-Instruct-v1.0-GGUFOpen In Colab2023-12-2110.7B; Censored; role play okhttps://huggingface.co/TheBloke/SOLAR-10.7B-Instruct-v1.0-GGUF
Nous-Hermes-2-SOLAR-10.7B-GGUFOpen In Colab2024-01-0810.7B; Partially Censored; role play goodhttps://huggingface.co/TheBloke/Nous-Hermes-2-SOLAR-10.7B-GGUF
openchat-3.5-0106-GGUFOpen In Colab2024-01-12Role play and creative writing good; not censoredhttps://huggingface.co/TheBloke/openchat-3.5-0106-GGUF
Mistral-7B-Instruct-v0.2-code-ft-GGUFOpen In Colab2024-01-12Coding modelhttps://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-code-ft-GGUF
Panda-7B-v0.1-GGUFOpen In Colab2024-01-12Role play ok; creative writing good; partially censoredhttps://huggingface.co/TheBloke/Panda-7B-v0.1-GGUF
LLaMA-Pro-8B-GGUFOpen In Colab2024-01-16Claims to be good at math/coding; quality so sohttps://huggingface.co/TheBloke/LLaMA-Pro-8B-GGUF
dolphin-2.6-mistral-7B-dpo-laser-GGUFOpen In Colab2024-01-16Uncensored; Good quality; role play goodhttps://huggingface.co/TheBloke/dolphin-2.6-mistral-7B-dpo-laser-GGUF
laserxtral-GGUFOpen In Colab2024-01-174x7B MOE model; half memory than Mixtral; not censored; role play goodhttps://huggingface.co/dagbs/laserxtral-GGUF
Mixtral_7Bx2_MoE-GGUFOpen In Colab2024-01-232x7B MOE model; uncensored; role play okhttps://huggingface.co/TheBloke/Mixtral_7Bx2_MoE-GGUF
CapybaraHermes-2.5-Mistral-7B-GGUFOpen In Colab2024-02-06Censored;https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF
stable-code-3b-GGUFOpen In Colab2024-02-063B; Coding model; quality meh probably too smallhttps://huggingface.co/TheBloke/stable-code-3b-GGUF/
EstopianMaid-13B-GGUFOpen In Colab2024-02-0613B; Roleplay modelhttps://huggingface.co/TheBloke/EstopianMaid-13B-GGUF
gemma-7b-it-GGUFOpen In Colab2024-02-287B; Censoredhttps://huggingface.co/mlabonne/gemma-7b-it-GGUF/
StarCoder2-15B-GGUFOpen In Colab2024-03-2015B; coding model; highly voted on r/LocalLLaMAhttps://huggingface.co/second-state/StarCoder2-15B-GGUF
Qwen1.5-7B-Chat-GGUFOpen In Colab2024-03-207B; Censored; highly voted on r/LocalLLaMAhttps://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF
Qwen1.5-14B-Chat-GGUFOpen In Colab2024-03-2014B; Censored; highly voted on r/LocalLLaMAhttps://huggingface.co/Qwen/Qwen1.5-14B-Chat-GGUF
Hermes-2-Pro-Mistral-7B-GGUFOpen In Colab2024-03-227B; Not censored; role play ok; highly ranked on Huggingfacehttps://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF
Nous-Hermes-2-Mistral-7B-DPO-GGUFOpen In Colab2024-03-227B; Partially censored; role play good; highly ranked on Huggingfacehttps://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO-GGUF
stable-code-instruct-3b-GGUFOpen In Colab2024-03-273B; instruction tuned code generation modelhttps://huggingface.co/bartowski/stable-code-instruct-3b-GGUF
Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4Open In Colab2024-04-03MOE; Small footprint; some logical errorshttps://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4
Octopus-v2Open In Colab2024-04-072B; non-quantized; optimized for on-device Android APIshttps://huggingface.co/NexaAIDev/Octopus-v2/
codegemma-7b-it-GGUFOpen In Colab2024-04-187B; coding modelhttps://huggingface.co/lmstudio-community/codegemma-7b-it-GGUF
CodeQwen1.5-7B-Chat-GGUFOpen In Colab2024-04-187B; coding modelhttps://huggingface.co/Qwen/CodeQwen1.5-7B-Chat-GGUF
WizardLM-2-7B-GGUFOpen In Colab2024-04-187B; Censoredhttps://huggingface.co/MaziyarPanahi/WizardLM-2-7B-GGUF
Meta-Llama-3-8B-Instruct-GGUFOpen In Colab2024-04-198B; Censoredhttps://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF
dolphin-2.9-llama3-8b-gguffOpen In Colab2024-04-228B; Uncensored; logic seems degraded from Llama-3-8Bhttps://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b-gguff
Lexi-Llama-3-8B-Uncensored-GGUFOpen In Colab2024-04-248B; Uncensoredhttps://huggingface.co/Orenguteng/Lexi-Llama-3-8B-Uncensored-GGUF
Llama3-8B-Chinese-Chat-GGUFOpen In Colab2024-04-248B; Chinesehttps://huggingface.co/QuantFactory/Llama3-8B-Chinese-Chat-GGUF
Phi-3-mini-4k-instruct-ggufOpen In Colab2024-04-243.8B; Censored; Fasthttps://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf
Llama-3-8B-Instruct-32k-v0.1-GGUFOpen In Colab2024-04-258B; 32K context; good for summarization of long texthttps://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-32k-v0.1-GGUF
starcoder2-15b-instruct-v0.1-GGUFOpen In Colab2024-05-0615B; coding modelhttps://huggingface.co/bartowski/starcoder2-15b-instruct-v0.1-GGUF
Hermes-2-Pro-Llama-3-8B-GGUFOpen In Colab2024-05-068B; Partially censored; JSON, Tool Use, etc.https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF
Llama-3-ChatQA-1.5-8B-GGUFOpen In Colab2024-05-158B; Uncensoredhttps://huggingface.co/bartowski/Llama-3-ChatQA-1.5-8B-GGUFF
Hermes-2-Theta-Llama-3-8B-GGUFOpen In Colab2024-05-178B; Censored; JSON, Tool Use, etchttps://huggingface.co/NousResearch/Hermes-2-Theta-Llama-3-8B-GGUF
blossom-v5.1-9b-GGUFOpen In Colab2024-05-179B; Wizard/Orca/Math/Chinese/English mixed datasethttps://huggingface.co/bartowski/blossom-v5.1-9b-GGUF
falcon-11B-GGUFOpen In Colab2024-05-1711B; base raw pretrained modelhttps://huggingface.co/bartowski/falcon-11B-GGUF
llama-3-cat-8b-instruct-v1-GGUFFOpen In Colab2024-05-208B; partially censored; system instruction fidelity; Role playhttps://huggingface.co/bartowski/llama-3-cat-8b-instruct-v1-GGUFF
Yi-1.5-9B-Chat-GGUFOpen In Colab2024-05-209B; Censoredhttps://huggingface.co/bartowski/Yi-1.5-9B-Chat-GGUF
SFR-Iterative-DPO-LLaMA-3-8B-R-GGUFOpen In Colab2024-05-228B; Partially censoredhttps://huggingface.co/bartowski/SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF
Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-ImatrixOpen In Colab2024-05-228B; Roleplay; Trained to not decline requests; Seems to be trained on Quora datahttps://huggingface.co/Lewdiculous/Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix
Mistral-7B-Instruct-v0.3-GGUFOpen In Colab2024-05-237B; Censoredhttps://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF
L3-8B-Stheno-v3.1-GGUF-IQ-ImatrixOpen In Colab2024-05-308B; trained for 1 on 1 uncensored roleplayhttps://huggingface.co/Lewdiculous/L3-8B-Stheno-v3.1-GGUF-IQ-Imatrix
aya-23-8B-GGUFOpen In Colab2024-05-308B; Censoredhttps://huggingface.co/bartowski/aya-23-8B-GGUF
LLaMA3-iterative-DPO-final-GGUFOpen In Colab2024-05-308B; Censoredhttps://huggingface.co/bartowski/LLaMA3-iterative-DPO-final-GGUF
openchat-3.6-8b-20240522-GGUFOpen In Colab2024-06-048B; Partially censoredhttps://huggingface.co/bartowski/openchat-3.6-8b-20240522-GGUF
Meta-Llama-3-8B-Instruct-abliterated-v3-GGUFOpen In Colab2024-06-048B; Uncensored;https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF
NeuralDaredevil-8B-abliterated-GGUFOpen In Colab2024-06-198B; Uncensored;https://huggingface.co/QuantFactory/NeuralDaredevil-8B-abliterated-GGUF
Qwen2-7B-Instruct-GGUFOpen In Colab2024-06-247B; Censoredhttps://huggingface.co/Qwen/Qwen2-7B-Instruct-GGUF
DeepSeek-Coder-V2-Lite-Instruct-GGUFOpen In Colab2024-06-2716B; Instruct coding modelhttps://huggingface.co/lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF
internlm2_5-7b-chat-ggufOpen In Colab2024-07-117B; Censored; Long context; Reasoning and Tool Use;https://huggingface.co/internlm/internlm2_5-7b-chat-gguf
gemma-2-9b-it-GGUFOpen In Colab2024-07-119B; Censoredhttps://huggingface.co/bartowski/gemma-2-9b-it-GGUF
Smegmma-Deluxe-9B-v1-GGUFOpen In Colab2024-07-129B; Less censored gemma (still refuses some requests); Role playhttps://huggingface.co/bartowski/Smegmma-Deluxe-9B-v1-GGUF
h2o-danube3-4b-chat-GGUFOpen In Colab2024-07-174B; Partially censoredhttps://huggingface.co/h2oai/h2o-danube3-4b-chat-GGUF
Tiger-Gemma-9B-v1-GGUFOpen In Colab2024-07-179B; Uncensored; but appears to have some flaws in logichttps://huggingface.co/bartowski/Tiger-Gemma-9B-v1-GGUF
Gemmasutra-9B-v1-GGUFOpen In Colab2024-07-249B; Roleplay with some censorshiphttps://huggingface.co/TheDrummer/Gemmasutra-9B-v1-GGUF
Meta-Llama-3.1-8B-Instruct-GGUFOpen In Colab2024-07-258B; Censoredhttps://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF
Mistral-Nemo-Instruct-2407-GGUFOpen In Colab2024-07-2512.2B; Partially Uncensored; Recommended by Redditorshttps://huggingface.co/second-state/Mistral-Nemo-Instruct-2407-GGUF
Celeste-12B-V1.6_iMatrix_GGUFOpen In Colab2024-07-3012B; Roleplay & story writing model, Uncensoredhttps://huggingface.co/MarsupialAI/Celeste-12B-V1.6_iMatrix_GGUF
Hermes-3-Llama-3.1-8B-GGUFOpen In Colab2024-08-198B; Uncensoredhttps://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF
Gemma-2-9B-It-SPPO-Iter3-GGUFOpen In Colab2024-08-199B; Recommended on Reddit for summarizationhttps://huggingface.co/bartowski/Gemma-2-9B-It-SPPO-Iter3-GGUF
Llama-3.1-Storm-8B-GGUFOpen In Colab2024-08-208B; Censoredhttps://huggingface.co/bartowski/Llama-3.1-Storm-8B-GGUF
Phi-3.5-mini-instruct-GGUFOpen In Colab2024-08-213.5B; Censored, small & fasthttps://huggingface.co/lmstudio-community/Phi-3.5-mini-instruct-GGUF
Phi-3.5-mini-instruct_Uncensored-GGUFOpen In Colab2024-08-263.5B; Uncensored, small & fast, but has logic flaws & inconsistent resultshttps://huggingface.co/bartowski/Phi-3.5-mini-instruct_Uncensored-GGUF
NemoMix-Unleashed-12B-GGUFOpen In Colab2024-08-2612B; Partially censored, Roleplaying & story tellinghttps://huggingface.co/bartowski/NemoMix-Unleashed-12B-GGUF
Mistral-NeMo-Minitron-8B-Base-GGUFOpen In Colab2024-08-268B; Base modelhttps://huggingface.co/QuantFactory/Mistral-NeMo-Minitron-8B-Base-GGUF
Yi-Coder-9B-Chat-GGUFOpen In Colab2024-09-059B; Instruction coding modelhttps://huggingface.co/bartowski/Yi-Coder-9B-Chat-GGUF
Llama-3.2-3B-Instruct-Q8_0-GGUFOpen In Colab2024-10-013B; Small censored modelhttps://huggingface.co/hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF
Qwen2.5-7B-Instruct-GGUFOpen In Colab2024-10-017B; Censored, Recommended on r/LocalLLaMAhttps://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF
Qwen2.5-Coder-7B-Instruct-GGUFOpen In Colab2024-10-017B; Instruction coding model, recommended on r/LocalLLaMAhttps://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF
Llama-3.1-8B-Lexi-Uncensored-V2-GGUFOpen In Colab2024-10-158B; Partially censoredhttps://huggingface.co/Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF
SuperNova-Medius-GGUFOpen In Colab2024-10-1514B; Censored, Recommended on r/LocalLLaMAhttps://huggingface.co/arcee-ai/SuperNova-Medius-GGUF
Llama-3.2-3B-Instruct-uncensored-GGUFOpen In Colab2024-10-153B; Small, uncensoredhttps://huggingface.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF
Ministral-8B-Instruct-2410-HF-GGUF-TESTOpen In Colab2024-10-218B; Partially censoredhttps://huggingface.co/bartowski/Ministral-8B-Instruct-2410-HF-GGUF-TEST
granite-3.0-8b-instruct-GGUFOpen In Colab2024-10-288B; Censoredhttps://huggingface.co/bartowski/granite-3.0-8b-instruct-GGUF
aya-expanse-8b-GGUFOpen In Colab2024-10-288B; Censoredhttps://huggingface.co/bartowski/aya-expanse-8b-GGUF
Qwen2.5-Coder-14B-Instruct-GGUFOpen In Colab2024-11-1214B; Coding model, good at its sizehttps://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct-GGUF
Marco-o1-GGUFOpen In Colab2024-11-257B; CoT model to mimic o1 for reasoninghttps://huggingface.co/bartowski/Marco-o1-GGUF
Llama-Sentient-3.2-3B-Instruct-GGUFOpen In Colab2024-11-253B; Censoredhttps://huggingface.co/prithivMLmods/Llama-Sentient-3.2-3B-Instruct-GGUF
Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-GGUFOpen In Colab2024-12-0510B; Uncensored story writing modelhttps://huggingface.co/DavidAU/Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-GGUF
INTELLECT-1-Instruct-GGUFOpen In Colab2024-12-0510B; Censoredhttps://huggingface.co/lmstudio-community/INTELLECT-1-Instruct-GGUF
L3.1-RP-Hero-BigTalker-8B-GGUFOpen In Colab2024-12-058B; Uncensored NSFW roleplay modelhttps://huggingface.co/DavidAU/L3.1-RP-Hero-BigTalker-8B-GGUF
EXAONE-3.5-7.8B-Instruct-GGUFOpen In Colab2024-12-097.8B; Censored, English and Korean, context up to 32Khttps://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-GGUF
Virtuoso-Small-GGUFOpen In Colab2024-12-1014B; Censored, good for general tasks, recommended on Reddithttps://huggingface.co/bartowski/Virtuoso-Small-GGUF
phi-4Open In Colab2024-12-1614B; Censoredhttps://huggingface.co/matteogeniaccio/phi-4
Hermes-3-Llama-3.2-3B-GGUFOpen In Colab2024-12-213B; Partially censoredhttps://huggingface.co/NousResearch/Hermes-3-Llama-3.2-3B-GGUF
Falcon3-Mamba-7B-Instruct-GGUFOpen In Colab2024-12-217B; Mamba1 architecture, 32K contexthttps://huggingface.co/tiiuae/Falcon3-Mamba-7B-Instruct-GGUF
QwQ-LCoT-7B-Instruct-GGUFOpen In Colab2024-12-217B; finetuned on CoThttps://huggingface.co/prithivMLmods/QwQ-LCoT-7B-Instruct-GGUF
granite-3.1-8b-instruct-GGUFOpen In Colab2024-12-248B; multilingual, 128K contexthttps://huggingface.co/bartowski/granite-3.1-8b-instruct-GGUF
granite-3.1-3b-a800m-instruct-GGUFOpen In Colab2024-12-243B; MoE, 128K contexthttps://huggingface.co/bartowski/granite-3.1-3b-a800m-instruct-GGUF
Falcon3-10B-Instruct-GGUFOpen In Colab2025-01-2110B; 32K context, English/French/Spanish/Portuguesehttps://huggingface.co/tiiuae/Falcon3-10B-Instruct-GGUF
DeepSeek-R1-Distill-Llama-8B-GGUFOpen In Colab2025-01-218B; Reasoning model, 128K contexthttps://huggingface.co/bartowski/DeepSeek-R1-Distill-Llama-8B-GGUF
DeepSeek-R1-Distill-Qwen-7B-GGUFOpen In Colab2025-02-067B; Reasoning model, 128K context, censoredhttps://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF
DeepSeek-R1-Distill-Qwen-14B-GGUFOpen In Colab2025-02-0614B; Reasoning model, 128K context, censoredhttps://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF
Xwen-7B-Chat-GGUFOpen In Colab2025-02-067B; Uncensoredhttps://huggingface.co/mradermacher/Xwen-7B-Chat-GGUF
Virtuoso-Small-v2-GGUFOpen In Colab2025-02-0614B; Censoredhttps://huggingface.co/bartowski/arcee-ai_Virtuoso-Small-v2-GGUF
DeepHermes-3-Llama-3-8B-Preview-GGUFOpen In Colab2025-02-148B; Reasoning model, partially censoredhttps://huggingface.co/NousResearch/DeepHermes-3-Llama-3-8B-Preview-GGUF
Reverb-7b-Q4_K_M-GGUFOpen In Colab2025-02-207B; Censoredhttps://huggingface.co/ozone-ai/Reverb-7b-Q4_K_M-GGUF
Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUFOpen In Colab2025-03-038x3B; MoE, Uncensored NSFW, roleplay, writinghttps://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF
google_gemma-3-12b-it-GGUFOpen In Colab2025-03-1712B; Censoredhttps://huggingface.co/bartowski/google_gemma-3-12b-it-GGUF
microsoft_Phi-4-mini-instruct-GGUFOpen In Colab2025-03-173.8B; Censored, 128 K contexthttps://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF
RekaAI_reka-flash-3-GGUFOpen In Colab2025-03-1821B; Reasoning model, censoredhttps://huggingface.co/bartowski/RekaAI_reka-flash-3-GGUF
mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUFOpen In Colab2025-03-1924B; Censored, unlike previous mistral modelshttps://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF
agentica-org_DeepCoder-14B-Preview-GGUFOpen In Colab2025-04-0914B; Reasoning coding modelhttps://huggingface.co/bartowski/agentica-org_DeepCoder-14B-Preview-GGUF
deepcogito_cogito-v1-preview-qwen-14B-GGUFOpen In Colab2025-04-0914B; Censoredhttps://huggingface.co/bartowski/deepcogito_cogito-v1-preview-qwen-14B-GGUF
Qwen_Qwen3-14B-GGUFOpen In Colab2025-04-2914B; Censoredhttps://huggingface.co/bartowski/Qwen_Qwen3-14B-GGUF
ReadyArt_Amoral-Fallen-Omega-Gemma3-12B-GGUFOpen In Colab2025-04-2912B; Uncensored NSFW roleplay, but may deny non-sexual illegal requestshttps://huggingface.co/bartowski/ReadyArt_Amoral-Fallen-Omega-Gemma3-12B-GGUF
soob3123_Veritas-12B-GGUFOpen In Colab2025-04-2912B; Partially censoredhttps://huggingface.co/bartowski/soob3123_Veritas-12B-GGUF
THUDM_GLM-4-9B-0414-GGUFOpen In Colab2025-04-299B; Censoredhttps://huggingface.co/bartowski/THUDM_GLM-4-9B-0414-GGUF
microsoft_Phi-4-reasoning-plus-GGUFOpen In Colab2025-05-0614B; Reasoning model, Censoredhttps://huggingface.co/bartowski/microsoft_Phi-4-reasoning-plus-GGUF
Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-GGUFOpen In Colab2025-05-068B; Uncensored modelhttps://huggingface.co/bartowski/Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-GGUF
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUFOpen In Colab2025-06-098B; Reasoning model, Censoredhttps://huggingface.co/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
open-thoughts_OpenThinker3-7B-GGUFOpen In Colab2025-06-097B; Reasoning model, Uncensored, tends to think a lot before answeringhttps://huggingface.co/bartowski/open-thoughts_OpenThinker3-7B-GGUF
Menlo_Jan-nano-128k-GGUFOpen In Colab2025-06-264B; Long-context model optimized for MCP use and deep researchhttps://huggingface.co/bartowski/Menlo_Jan-nano-128k-GGUF
google_gemma-3n-E4B-it-GGUFOpen In Colab2025-07-108B with 4B activated; Censored, 32K context, efficient parameter managementhttps://huggingface.co/bartowski/google_gemma-3n-E4B-it-GGUF
janhq_Jan-v1-4B-GGUFOpen In Colab2025-08-154B; Optimized for search and deep researchhttps://huggingface.co/bartowski/janhq_Jan-v1-4B-GGUF
Qwen3-Coder-30B-A3B-Instruct-GGUFOpen In Colab2025-08-1530B with 3 B activated; Coding modelhttps://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF
Qwen3-30B-A3B-Instruct-2507-GGUFOpen In Colab2025-08-1530B with 3 B activated; Censoredhttps://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF
Qwen3-4B-Instruct-2507-GGUFOpen In Colab2025-08-154B; Censoredhttps://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF
NousResearch_Hermes-4-14B-GGUFOpen In Colab2025-09-0914B; Partially censoredhttps://huggingface.co/bartowski/NousResearch_Hermes-4-14B-GGUF
VibeThinker-1.5B-GGUFFOpen In Colab2025-11-241.5B; Math and Algo Coding modelhttps://huggingface.co/MaziyarPanahi/VibeThinker-1.5B-GGUFF
Ministral-3-14B-Instruct-2512-GGUFOpen In Colab2025-12-0814B; Multi-modal, Agentic, Censored modelhttps://huggingface.co/unsloth/Ministral-3-14B-Instruct-2512-GGUF
Ministral-3-14B-Reasoning-2512-GGUFOpen In Colab2025-12-0814B; Reasoning, Multi-modal, Agentic, Censored modelhttps://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF
NousResearch_NousCoder-14B-GGUFOpen In Colab2026-01-1314B; Coding modelhttps://huggingface.co/bartowski/NousResearch_NousCoder-14B-GGUF
nvidia_Nemotron-Cascade-14B-Thinking-GGUFOpen In Colab2026-01-1315B; Coding, reasoning modelhttps://huggingface.co/bartowski/nvidia_Nemotron-Cascade-14B-Thinking-GGUF

Star History

Star History Chart

Share This Project

If you find this project helpful, please share it:

You may also be interested in

Check out awesome-TTS-Colab for a curated list of Google Colab notebooks featuring open source text-to-speech (TTS) models. These notebooks are easy to use and can also be run on your local machine!

Older Models with Scores

Older Models with Scores:

These models work better among the models I tested on my hardware (i5-12490F, 32GB RAM, RTX 3060 Ti GDDR6X 8GB VRAM): (Note: Because llama.cpp has made some breaking changes to the support of older ggml models. Some older ggml versions listed below may not work properly on current llama.cpp. But there should be GPTQ equivalents or newer ggml versions for the models.)

NOTE:

  • Major edit on June 30, 2023. Since I noticed GPT-4 started to give more granular scores and tends to give higher scores, I have decided to re-run the tests for all models so they can be compared (All evaluations done with GPT-4-0613 for consistency). The scores in the table below are the average of the scores from the latest tests. The scores in the table below are not comparable to the scores in the previous versions of this README.
Model_NameAvg_ScoreColab_LinkDate_AddedLink
Mistral-7B-OpenOrca (using oobabooga/text-generation-webui)10.00Open In Colab2023/10/08https://huggingface.co/TheBloke/Mistral-7B-OpenOrca-GGUF
Llama-2-13B-chat (using oobabooga/text-generation-webui)9.65Open In Colab2023/07/20https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML
wizard-vicuna-13B.ggml.q4_0 (using llama.cpp)9.63Open In Colab2023/05/07https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML
Nous-Capybara-7B (using oobabooga/text-generation-webui)9.56Open In Colab2023/10/08https://huggingface.co/TheBloke/Nous-Capybara-7B-GGUF
vicuna-13B-v1.5 (using oobabooga/text-generation-webui)9.53Open In Colab2023/08/09https://huggingface.co/TheBloke/vicuna-13B-v1.5-GGML
wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui)9.53Open In Colab2023/05/29https://huggingface.co/TheBloke/wizardLM-13B-1.0-GPTQ
airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui)9.50Open In Colab2023/06/30https://huggingface.co/TheBloke/airoboros-13B-gpt4-1.4-GPTQ
Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui)9.44Open In Colab2023/06/03https://huggingface.co/TheBloke/Nous-Hermes-13B-GPTQ/tree/main
Dolphin-Llama-13B (using oobabooga/text-generation-webui)9.38Open In Colab2023/07/24https://huggingface.co/TheBloke/Dolphin-Llama-13B-GGML
Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui)9.37Open In Colab2023/10/08https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF
OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui)9.37Open In Colab2023/08/15https://huggingface.co/TheBloke/OpenOrca-Platypus2-13B-GGML
airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui)9.34Open In Colab2023/08/01https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-2.0-GGML
Chronos-13B-v2 (using oobabooga/text-generation-webui)9.31Open In Colab2023/08/09https://huggingface.co/TheBloke/Chronos-13B-v2-GGML
vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui)9.31Open In Colab2023/06/29https://huggingface.co/TheBloke/vicuna-13b-v1.3.0-GPTQ
MythoLogic-13B (using oobabooga/text-generation-webui)9.31Open In Colab2023/07/20https://huggingface.co/TheBloke/MythoLogic-13B-GGML
Selfee-13B-GPTQ (using oobabooga/text-generation-webui)9.28Open In Colab2023/06/07https://huggingface.co/TheBloke/Selfee-13B-GPTQ
WizardLM-13B-V1.2 (using oobabooga/text-generation-webui)9.28Open In Colab2023/07/26https://huggingface.co/TheBloke/WizardLM-13B-V1.2-GGML
minotaur-13B-GPTQ (using oobabooga/text-generation-webui)9.28Open In Colab2023/06/09https://huggingface.co/TheBloke/minotaur-13B-GPTQ
Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui)9.20Open In Colab2023/09/21https://huggingface.co/TheBloke/Pygmalion-2-13B-SuperCOT2-GGUF
Athena-v1 (using oobabooga/text-generation-webui)9.19Open In Colab2023/08/31https://huggingface.co/TheBloke/Athena-v1-GGUF
PuddleJumper-13B (using oobabooga/text-generation-webui)9.1875Open In Colab2023/08/29https://huggingface.co/TheBloke/PuddleJumper-13B-GGUF
Nous-Hermes-Llama2 (using oobabooga/text-generation-webui)9.17Open In Colab2023/07/24https://huggingface.co/TheBloke/Nous-Hermes-Llama2-GGML
Luban-13B (using oobabooga/text-generation-webui)9.15Open In Colab2023/08/31https://huggingface.co/TheBloke/Luban-13B-GGUF
MythoBoros-13B (using oobabooga/text-generation-webui)9.15Open In Colab2023/07/24https://huggingface.co/TheBloke/MythoBoros-13B-GGML
13B-Ouroboros (using oobabooga/text-generation-webui)9.11Open In Colab2023/07/27https://huggingface.co/TheBloke/13B-Ouroboros-GGML
tulu-13B-GPTQ (using oobabooga/text-generation-webui)9.06Open In Colab2023/06/13https://huggingface.co/TheBloke/tulu-13B-GPTQ
AlpacaCielo-13B (using oobabooga/text-generation-webui)9.03125Open In Colab2023/07/27https://huggingface.co/TheBloke/AlpacaCielo-13B-GGML
StableBeluga-13B (using oobabooga/text-generation-webui)9Open In Colab2023/08/01https://huggingface.co/TheBloke/StableBeluga-13B-GGML
Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui)8.97Open In Colab2023/08/10https://huggingface.co/TheBloke/Chronos-Hermes-13B-v2-GGML
OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui)8.97Open In Colab2023/09/05https://huggingface.co/TheBloke/OpenBuddy-Llama2-13B-v11.1-GGUF
Camel-Platypus2-13B (using oobabooga/text-generation-webui)8.94Open In Colab2023/08/15https://huggingface.co/TheBloke/Camel-Platypus2-13B-GGML
airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui)8.94Open In Colab2023/09/21https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-m2.0-GGUF
UltraLM-13B-GPTQ (using oobabooga/text-generation-webui)8.89Open In Colab2023/06/30https://huggingface.co/TheBloke/UltraLM-13B-GPTQ
13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui)8.88Open In Colab2023/06/03https://huggingface.co/digitous/13B-HyperMantis_GPTQ_4bit-128g/
Stable-Platypus2-13B (using oobabooga/text-generation-webui)8.875Open In Colab2023/08/15https://huggingface.co/TheBloke/Stable-Platypus2-13B-GGML
Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui)8.84Open In Colab2023/05/25https://huggingface.co/TheBloke/airoboros-13B-GPTQ
Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui)8.84Open In Colab2023/09/21https://huggingface.co/TheBloke/Kuchiki-1.1-L2-7B-GGUF
WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui)8.80625Open In Colab2023/08/09https://huggingface.co/TheBloke/WizardLM-1.0-Uncensored-Llama2-13B-GGML
Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui)8.75Open In Colab2023/08/10https://huggingface.co/TheBloke/Chronos-Beluga-v2-13B-GGML
Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui)8.75Open In Colab2023/06/09https://huggingface.co/TheBloke/Vicuna-13B-CoT-GPTQ
wizardLM-7B.q4_2 (in GPT4All)8.75No2023/05/07https://gpt4all.io/models/ggml-wizardLM-7B.q4_2.bin
OpenChat_v3.2 (using oobabooga/text-generation-webui)8.71875Open In Colab2023/08/01https://huggingface.co/TheBloke/OpenChat_v3.2-GGML
Huginn-13B (using oobabooga/text-generation-webui)8.7125Open In Colab2023/08/10https://huggingface.co/TheBloke/Huginn-13B-GGML
WizardLM-13B-V1.1 (using oobabooga/text-generation-webui)8.66Open In Colab2023/07/17https://huggingface.co/TheBloke/WizardLM-13B-V1.1-GGML
robin-13B-v2-GPTQ (using oobabooga/text-generation-webui)8.66Open In Colab2023/06/19https://huggingface.co/TheBloke/robin-13B-v2-GPTQ
llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui)8.625Open In Colab2023/07/21https://huggingface.co/TheBloke/llama-2-13B-Guanaco-QLoRA-GGML
mpt-7b-chat (in GPT4All)8.53No2023/05/11https://gpt4all.io/models/ggml-mpt-7b-chat.bin
chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui)8.48125Open In Colab2023/06/16https://huggingface.co/TheBloke/chronos-hermes-13B-GPTQ
Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui)8.46875Open In Colab2023/07/20https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGML
stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui)8.25Open In Colab2023/05/12https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ
manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui)8.21875Open In Colab2023/05/24https://huggingface.co/TheBloke/manticore-13b-chat-pyg-GPTQ
CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui)8.09375Open In Colab2023/06/10https://huggingface.co/TheBloke/CAMEL-13B-Combined-Data-GPTQ
WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui)8.09375Open In Colab2023/06/02https://huggingface.co/TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ
llama-13b-supercot-GGML (using oobabooga/text-generation-webui)8.01Open In Colab2023/07/05https://huggingface.co/TheBloke/llama-13b-supercot-GGML
Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui)7.96875Open In Colab2023/05/24https://huggingface.co/TheBloke/Project-Baize-v2-13B-GPTQ
koala-13B-4bit-128g.GGML (using llama.cpp)7.9375No2023/05/07https://huggingface.co/TheBloke/koala-13B-GPTQ-4bit-128g-GGML
wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui)7.90625Open In Colab2023/05/19https://huggingface.co/4bit/WizardLM-13B-Uncensored-4bit-128g
vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui)7.875Open In Colab2023/06/29https://huggingface.co/TheBloke/vicuna-7B-v1.3-GPTQ
Manticore-13B-GPTQ (using oobabooga/text-generation-webui)7.78125Open In Colab2023/05/23https://huggingface.co/TheBloke/Manticore-13B-GPTQ
vicuna-13b-1.1-q4_2 (in GPT4All)7.75No2023/05/07https://gpt4all.io/models/ggml-vicuna-13b-1.1-q4_2.bin
falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui)7.625Open In Colab2023/06/02https://huggingface.co/TheBloke/falcon-7b-instruct-GPTQ
guanaco-13B-GPTQ (using oobabooga/text-generation-webui)7.5625Open In Colab2023/05/26https://huggingface.co/TheBloke/guanaco-13B-GPTQ
Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui)7.31Open In Colab2023/08/31https://huggingface.co/TheBloke/Mythical-Destroyer-V2-L2-13B-GGUF
Kimiko-v2-13B (using oobabooga/text-generation-webui)7.25 Open In Colab2023/08/31https://huggingface.co/TheBloke/Kimiko-v2-13B-GGUF
orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui)7.0875Open In Colab2023/06/28https://huggingface.co/TheBloke/orca_mini_13B-GGML
Platypus2-13B (using oobabooga/text-generation-webui)7.03125Open In Colab2023/08/15https://huggingface.co/TheBloke/Platypus2-13B-GGML
Redmond-Puffin-13B (using oobabooga/text-generation-webui)7.03125Open In Colab2023/07/20https://huggingface.co/TheBloke/Redmond-Puffin-13B-GGML
13B-BlueMethod (using oobabooga/text-generation-webui)7.025Open In Colab2023/07/24https://huggingface.co/TheBloke/13B-BlueMethod-GGML
mpt-7b-instruct6.6875No2023/05/12https://huggingface.co/TheBloke/MPT-7B-Instruct-GGML
Kimiko-13B (using oobabooga/text-generation-webui)6.46875Open In Colab2023/08/01https://huggingface.co/TheBloke/Kimiko-13B-GGML
gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp)6.0625No2023/05/07https://huggingface.co/Bradarr/gpt4-x-alpaca-13b-native-ggml-model-q4_0
minotaur-15B-GPTQ (using oobabooga/text-generation-webui)5.9375Open In Colab2023/06/26https://huggingface.co/TheBloke/minotaur-15B-GPTQ
baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui)5.90625Open In Colab2023/07/05https://huggingface.co/TheBloke/baichuan-vicuna-7B-GGML
gpt4all-j-v1.3-groovy (in GPT4All)5.6875No2023/05/07https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin

Many thanks to:
❤️ GPT4ALl: https://github.com/nomic-ai/gpt4all-chat
❤️ llama.cpp: https://github.com/ggerganov/llama.cpp
❤️ oobabooga text generation webui: https://github.com/oobabooga/text-generation-webui
❤️ Colab webui inspired by camenduru: https://github.com/camenduru/text-generation-webui-colab/tree/main
❤️ The Bloke for quantization of the models: https://huggingface.co/TheBloke

Coding models tested & average scores:

(All scores are from GPT-4-0613.)

Model_NameAvg_ScoresColab_LinkDate_AddedLink
CodeLlama-13B-oasst-sft-v10 (using oobabooga/text-generation-webui)9.8Open In Colab2023/08/28https://huggingface.co/TheBloke/CodeLlama-13B-oasst-sft-v10-GGUF
WizardCoder-Python-13B-V1.0 (using oobabooga/text-generation-webui)9.5Open In Colab2023/08/28https://huggingface.co/TheBloke/WizardCoder-Python-13B-V1.0-GGUF
Redmond-Hermes-Coder-GPTQ (using oobabooga/text-generation-webui)8.4Open In Colab2023/07/03https://huggingface.co/TheBloke/Redmond-Hermes-Coder-GPTQ
CodeUp-Alpha-13B-HF (using oobabooga/text-generation-webui)7.9Open In Colab2023/08/15https://huggingface.co/TheBloke/CodeUp-Alpha-13B-HF-GGML
starchat-beta-GPTQ (using oobabooga/text-generation-webui)7.6Open In Colab2023/07/04https://huggingface.co/TheBloke/starchat-beta-GPTQ
wizard-vicuna-13B-GPTQ (using oobabooga/text-generation-webui)7.3Open In Colab2023/07/03https://huggingface.co/TheBloke/wizard-vicuna-13B-GPTQ
WizardCoder-Guanaco-15B-V1.1 (using oobabooga/text-generation-webui)7.1Open In Colab2023/07/21https://huggingface.co/TheBloke/WizardCoder-Guanaco-15B-V1.1-GPTQ
CodeLlama-13B-Instruct (using oobabooga/text-generation-webui)7Open In Colab2023/08/28https://huggingface.co/TheBloke/CodeLlama-13B-Instruct-GGUF
CodeUp-Llama-2-13B-Chat-HF (using oobabooga/text-generation-webui)6Open In Colab2023/08/03https://huggingface.co/TheBloke/CodeUp-Llama-2-13B-Chat-HF-GGML
WizardCoder-15B-1.0-GPTQ (using oobabooga/text-generation-webui)5.9Open In Colab2023/07/03https://huggingface.co/TheBloke/WizardCoder-15B-1.0-GPTQ
wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui)5.9Open In Colab2023/05/29https://huggingface.co/TheBloke/wizardLM-13B-1.0-GPTQ

Questions and scores

Original responses can be found at: https://docs.google.com/spreadsheets/d/1ogDXUiaBx3t7EpMo44aaA6U6kLXX0x2tGRgLg8CISGs/edit?usp=sharing

Question 1: Translate the following English text into French: "The sun rises in the east and sets in the west."

Task Domain: Translation
Expected Good Response: "Le soleil se lève à l'est et se couche à l'ouest."
Explanation: This task tests the model's ability to understand and accurately translate text between languages.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 1
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 8
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 7
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 1
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 1
  • mpt-7b-instruct : 1
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 7
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 8
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 1
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 8
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 8
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 1
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 7
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 8
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 1
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 1
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 1
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 8
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 7
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 8
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 1
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 8.5
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 6
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 7.5
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 4
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 9
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 7.5
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 7.5
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 9.5
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 9.5
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 9.5
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 9.5

Question 2: Summarize the following text: "The water cycle is a natural process that involves the continuous movement of water on, above, and below the Earth's surface. It includes various stages like evaporation, condensation, precipitation, and runoff. This cycle plays a crucial role in maintaining Earth's water balance and supporting life."

Task Domain: Summary
Expected Good Response: "The water cycle is the continuous movement of water on Earth, crucial for maintaining water balance and supporting life."
Explanation: This task evaluates the model's ability to extract the main points from a given text and generate a concise summary.

  • wizardLM-7B.q4_2 (in GPT4All) : 5
  • gpt4all-j-v1.3-groovy (in GPT4All) : 7.5
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 9
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 4
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 7.5
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 1
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 8.5
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 7.5
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 9
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 9
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 5
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 9
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 9
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 8
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 1
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 9
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 9
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 8
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 7.5
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 1
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 1
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 7
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 7.5
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 9
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 7.5
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 7.5
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 9
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 5
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 7.5
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 9
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 9.5
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 9
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 5
  • Athena-v1 (using oobabooga/text-generation-webui) : 9
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 9.5
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 9
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 3: I want you to act as a senior software developer with deep knowledge in system design, frontend programming, and backend programming. Provide a high level design of a mental health journal app. Include the frontend and backend components. Do not write code.

Task Domain: App Design
Explanation: This task evaluates the model's ability to closely follow user's instruction for a complex task.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 9
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 7.5
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 6.5
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 8
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 9
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 9
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 9.5
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 3
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 9.5
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 8.5
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 7.4
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 4: What are the main causes of the French Revolution according to this passage: "The French Revolution, which took place between 1789 and 1799, was rooted in financial crises, social inequality, and Enlightenment ideas."

Task Domain: Abstractive Question Answering
Expected Good Response: Financial crises, social inequality, and Enlightenment ideas.
Explanation: This task tests the model's ability to understand the context and generate an answer in its own words.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 10
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 1
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 6
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 4
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 9.5
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 9
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 9.5
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 7.5
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 8
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 9
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 5: In the following text, identify the two artists mentioned: "During the Renaissance, Leonardo da Vinci and Michelangelo were two of the most influential artists who created masterpieces that continue to inspire people today."

Task Domain: Extractive Question Answering
Expected Good Response: Leonardo da Vinci, Michelangelo
Explanation: This task assesses the model's ability to extract specific information from a given text.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 10
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 9
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 9.5
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 8.5
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 6: Determine the sentiment of this customer review: "I had a fantastic experience at this restaurant. The food was delicious, and the service was outstanding."

Task Domain: Sentiment Analysis
Expected Good Response: Positive
Explanation: This task evaluates the model's ability to analyze text and identify the sentiment expressed.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 10
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 1
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 4
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 1
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 8.5
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 5
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 7: Classify the following text into one of these categories: Sports, Technology, Health, or Politics: "Apple recently unveiled its latest iPhone, featuring a faster processor and improved battery life."

Task Domain: Zero-shot Classification
Expected Good Response: Technology
Explanation: This task tests the model's ability to classify text into predefined categories without any prior training on the specific categories.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 5
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 1
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 2
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 4
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 8: Complete the following sentence by filling in the blank: "The capital city of France is _______."

Task Domain: Mask Filling
Expected Good Response: Paris
Explanation: This task assesses the model's ability to understand context and generate appropriate words to fill in missing information.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 10
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 5
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 1
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 5
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 9: Write a rhyming couplet about nature.

Task Domain: Poetry Generation
Expected Good Response: "In nature's beauty, we find respite and grace, A symphony of colors that time cannot erase."
Explanation: This task tests the model's ability to generate creative and coherent text that adheres to specific constraints, such as rhyme and theme.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 7.5
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 1
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 6.5
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 6
  • mpt-7b-instruct : 2
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 8
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 6.5
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 4
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 6
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 7
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 7.5
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 7
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 4
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 4
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 8
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 9
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 4
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 7
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 7.5
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 4
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 8
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 7
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 8
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 7.5
  • Kimiko-13B (using oobabooga/text-generation-webui) : 6.5
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 8.5
  • Platypus2-13B (using oobabooga/text-generation-webui) : 6
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 7.5
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 4
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 3
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 7.4
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 8
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 6.5
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 9

Question 10: Based on the following statement, determine if the author's opinion is for or against nuclear energy: "Nuclear energy is a powerful source, but the potential risks and radioactive waste management issues make it a dangerous choice."

Task Domain: Opinion Detection
Expected Good Response: Against
Explanation: This task evaluates the model's ability to understand and identify the author's stance or opinion on a specific topic.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 4
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 9.5
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 6
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 8.5
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 7.5
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 9.5
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 9.5
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 4
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 4
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 6.5
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 7
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 8.5
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 8.5
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 3
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 8.5
  • Platypus2-13B (using oobabooga/text-generation-webui) : 9
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 4
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 9.5
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 11: Rewrite the following sentence in passive voice: "The dog chased the cat."

Task Domain: Text Rewriting
Expected Good Response: "The cat was chased by the dog."
Explanation: This task tests the model's ability to manipulate and rewrite text according to specific grammatical requirements.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 3
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 6.5
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 9.5
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 7
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 7.34
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 12: Use the retrieved context to answer a question. Context does not contain the answer. (Prompt too long. See spreadsheet for original prompt)

Task Domain: Document Question Answering
Expected Good Response: "I don't know."
Explanation: This task tests the model's ability to understand context and answer questions based on the information provided in the context.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 1
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 1
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 1
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 1
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 1
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 3
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 1
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 1
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 1
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 1
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 1
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 1
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 7
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 1
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 1
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 1
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 1
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 1
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 1
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 2
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 1
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 13: Use the retrieved context to answer a question. Context contains the answer. (Prompt too long. See spreadsheet for original prompt)

Task Domain: Document Question Answering
Expected Good Response: "["Semantic text search", "Generative question-answering", "Hybrid search", "Image similarity search", "Product recommendations"]"
Explanation: This task tests the model's ability to understand context and answer questions based on the information provided in the context.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 1
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 1
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 4
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 5.5
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 4
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 1
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 5
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 2
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 5
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 1
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 7
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 8
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 5
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 1
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 8.5
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 5.7
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 6.5
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 1
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 7.4
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 5
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 1
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 7.7
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 5.6875
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 1
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 1
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 7.5
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 7.4
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 5.6875
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 7.4
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 7.34
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 1
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 1
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 3
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 7.5
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 1
  • Huginn-13B (using oobabooga/text-generation-webui) : 8.5
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 7.5
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 5
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 1
  • Platypus2-13B (using oobabooga/text-generation-webui) : 8
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 7.4
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 4
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 2
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 2
  • Athena-v1 (using oobabooga/text-generation-webui) : 3
  • Luban-13B (using oobabooga/text-generation-webui) : 1
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 1
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 3
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 14: What is the square root of banana?

Task Domain: Mathematical Reasoning
Expected Good Response: "The question is nonsensical, as square roots can only be calculated for numbers, not for objects or words like 'banana'."
Explanation: This task tests the model's ability to recognize questions that are illogical or nonsensical and respond accordingly.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 1
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 1
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 1
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 1
  • wizard-lm-uncensored-13b

Truncated — view the full README on GitHub.

ai
gpt
llama
llm

Significant stargazers

Govind Kamtamneni

152 followers · starred Nov 2023

oobabooga

5,031 followers · starred Aug 2023

Virat Singh

4,063 followers · starred Feb 2024

Rafał Studnicki

80 followers · starred Apr 2024

Similar repos

0xSojalSec/LLMs-local

list of awesome platforms, tools, and resources run for LLMs locally

1,196

rafska/awesome-local-llm

A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally

2,981

LudwigStumpp/llm-leaderboard

A joint community effort to create one central leaderboard for LLMs.

306

BarnacleLabs/awesome-llm-list

An overview of Large Language Model (LLM) options

75

janhq/awesome-local-ai

An awesome repository of local AI tools

2,039

camenduru/text-generation-webui-colab

A colab gradio web UI for running Large Language Models

2,089

ludwigstumpp/llm-leaderboard

Space

🏆 LLM-Leaderboard

18

jakemanger/whatmodel

Find the best AI models you can run

33

Troyanovsky/Local-LLM-Comparison-Colab-UI

Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.

Jupyter Notebook

1,107

418 commits

updated Jan 13, 2026

See the code

README

Local LLM Comparison & Colab Links (WIP)

(Update Nov. 27, 2023) The original goal of the repo was to compare some smaller models (7B and 13B) that can be run on consumer hardware so every model had a score for a set of questions from GPT-4. But I realized that as there are many more capable models appearing, the evaluation and comparison process may not suffice.

Therefore, I'm only putting Colab WebUI links for the newer models and you can try them out yourselves with a few clicks - after all, the effectiveness of a language model relies heavily on its suitability for your specific use case. By trying out the models firsthand, you can assess their performance and determine which one best fits your needs.

Newer Models without Scores:

These models can be run on consumer hardware and are generally good (from Reddit suggestions and my own experience). Try them out yourselves (tap the 'Open in Colab' button)!

ModelColab LinkDate AddedNotesLink
OpenHermes-2.5-Mistral-7B-GGUFOpen In Colab2023-11-27Roleplay good, not censoredhttps://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF
zephyr-7B-beta-GGUFOpen In Colab2023-11-27Roleplay okay, not censoredhttps://huggingface.co/TheBloke/zephyr-7B-beta-GGUF
dolphin-2.2.1-mistral-7B-GGUFOpen In Colab2023-11-27Roleplay okay, not censoredhttps://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF
neural-chat-7B-v3-1-GGUFOpen In Colab2023-11-27Roleplay okay, not censored; some logic flawshttps://huggingface.co/TheBloke/neural-chat-7B-v3-1-GGUF
openchat_3.5-16k-GGUFOpen In Colab2023-11-27Censoredhttps://huggingface.co/TheBloke/openchat_3.5-16k-GGUF
Starling-LM-7B-alpha-GGUFOpen In Colab2023-11-29Censored; highly rated on Reddithttps://huggingface.co/TheBloke/Starling-LM-7B-alpha-GGUF
Orca-2-7B-GGUFOpen In Colab2023-11-29Censoredhttps://huggingface.co/TheBloke/Orca-2-7B-GGUF
Orca-2-13B-GGUF Open In Colab2023-11-29Censored, some weird logic flaws worse than 7B versionhttps://huggingface.co/TheBloke/Orca-2-13B-GGUF
MythoMist-7B-GGUFOpen In Colab2023-11-29Roleplay okay, not censored; some logic flawshttps://huggingface.co/TheBloke/MythoMist-7B-GGUF
NeuralHermes-2.5-Mistral-7B-GGUFOpen In Colab2023-12-05Roleplay good, not censoredhttps://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GGUF
stablelm-zephyr-3b-GGUFOpen In Colab2023-12-113B; roleplay ok; not censored; some logic flawshttps://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF
deepseek-llm-7B-chat-GGUFOpen In Colab2023-12-11Censoredhttps://huggingface.co/TheBloke/deepseek-llm-7B-chat-GGUF
Mistral-7B-Instruct-v0.2-GGUFOpen In Colab2023-12-13Partially censored; role play ok; highly rated on Reddithttps://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF
Mixtral-8x7B-Instruct-v0.1-GGUFOpen In Colab2023-12-13MOE model; partially censored; role play okhttps://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF
deepsex-34b-GGUFOpen In Colab2023-12-1434B; NSFW modelhttps://huggingface.co/TheBloke/deepsex-34b-GGUF
phi-2Open In Colab2023-12-152.7B; Base model;https://huggingface.co/microsoft/phi-2
Xwin-MLewd-13B-v0.2-GGUFOpen In Colab2023-12-1513B; NSFW modelhttps://huggingface.co/TheBloke/Xwin-MLewd-13B-v0.2-GGUF
MythoMax-L2-13B-GGUFOpen In Colab2023-12-1513B; Censored; role play okhttps://huggingface.co/TheBloke/MythoMax-L2-13B-GGUF
LLaMA2-13B-Tiefighter-GGUFOpen In Colab2023-12-1513B; role play goodhttps://huggingface.co/TheBloke/LLaMA2-13B-Tiefighter-GGUF
LLaMA2-13B-Psyfighter2-GGUFOpen In Colab2023-12-1513B; Partially censored; role play ok; recommended on Reddithttps://huggingface.co/TheBloke/LLaMA2-13B-Psyfighter2-GGUF
Noromaid-13B-v0.1.1-GGUFOpen In Colab2023-12-1513B; NSFW model; roleplay goodhttps://huggingface.co/TheBloke/Noromaid-13B-v0.1.1-GGUF
dolphin-2.5-mixtral-8x7b-GGUFOpen In Colab2023-12-20MOE model; claims to be uncensored but denies some requests; role play not usablehttps://huggingface.co/TheBloke/dolphin-2.5-mixtral-8x7b-GGUF
SOLAR-10.7B-Instruct-v1.0-GGUFOpen In Colab2023-12-2110.7B; Censored; role play okhttps://huggingface.co/TheBloke/SOLAR-10.7B-Instruct-v1.0-GGUF
Nous-Hermes-2-SOLAR-10.7B-GGUFOpen In Colab2024-01-0810.7B; Partially Censored; role play goodhttps://huggingface.co/TheBloke/Nous-Hermes-2-SOLAR-10.7B-GGUF
openchat-3.5-0106-GGUFOpen In Colab2024-01-12Role play and creative writing good; not censoredhttps://huggingface.co/TheBloke/openchat-3.5-0106-GGUF
Mistral-7B-Instruct-v0.2-code-ft-GGUFOpen In Colab2024-01-12Coding modelhttps://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-code-ft-GGUF
Panda-7B-v0.1-GGUFOpen In Colab2024-01-12Role play ok; creative writing good; partially censoredhttps://huggingface.co/TheBloke/Panda-7B-v0.1-GGUF
LLaMA-Pro-8B-GGUFOpen In Colab2024-01-16Claims to be good at math/coding; quality so sohttps://huggingface.co/TheBloke/LLaMA-Pro-8B-GGUF
dolphin-2.6-mistral-7B-dpo-laser-GGUFOpen In Colab2024-01-16Uncensored; Good quality; role play goodhttps://huggingface.co/TheBloke/dolphin-2.6-mistral-7B-dpo-laser-GGUF
laserxtral-GGUFOpen In Colab2024-01-174x7B MOE model; half memory than Mixtral; not censored; role play goodhttps://huggingface.co/dagbs/laserxtral-GGUF
Mixtral_7Bx2_MoE-GGUFOpen In Colab2024-01-232x7B MOE model; uncensored; role play okhttps://huggingface.co/TheBloke/Mixtral_7Bx2_MoE-GGUF
CapybaraHermes-2.5-Mistral-7B-GGUFOpen In Colab2024-02-06Censored;https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF
stable-code-3b-GGUFOpen In Colab2024-02-063B; Coding model; quality meh probably too smallhttps://huggingface.co/TheBloke/stable-code-3b-GGUF/
EstopianMaid-13B-GGUFOpen In Colab2024-02-0613B; Roleplay modelhttps://huggingface.co/TheBloke/EstopianMaid-13B-GGUF
gemma-7b-it-GGUFOpen In Colab2024-02-287B; Censoredhttps://huggingface.co/mlabonne/gemma-7b-it-GGUF/
StarCoder2-15B-GGUFOpen In Colab2024-03-2015B; coding model; highly voted on r/LocalLLaMAhttps://huggingface.co/second-state/StarCoder2-15B-GGUF
Qwen1.5-7B-Chat-GGUFOpen In Colab2024-03-207B; Censored; highly voted on r/LocalLLaMAhttps://huggingface.co/Qwen/Qwen1.5-7B-Chat-GGUF
Qwen1.5-14B-Chat-GGUFOpen In Colab2024-03-2014B; Censored; highly voted on r/LocalLLaMAhttps://huggingface.co/Qwen/Qwen1.5-14B-Chat-GGUF
Hermes-2-Pro-Mistral-7B-GGUFOpen In Colab2024-03-227B; Not censored; role play ok; highly ranked on Huggingfacehttps://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF
Nous-Hermes-2-Mistral-7B-DPO-GGUFOpen In Colab2024-03-227B; Partially censored; role play good; highly ranked on Huggingfacehttps://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO-GGUF
stable-code-instruct-3b-GGUFOpen In Colab2024-03-273B; instruction tuned code generation modelhttps://huggingface.co/bartowski/stable-code-instruct-3b-GGUF
Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4Open In Colab2024-04-03MOE; Small footprint; some logical errorshttps://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4
Octopus-v2Open In Colab2024-04-072B; non-quantized; optimized for on-device Android APIshttps://huggingface.co/NexaAIDev/Octopus-v2/
codegemma-7b-it-GGUFOpen In Colab2024-04-187B; coding modelhttps://huggingface.co/lmstudio-community/codegemma-7b-it-GGUF
CodeQwen1.5-7B-Chat-GGUFOpen In Colab2024-04-187B; coding modelhttps://huggingface.co/Qwen/CodeQwen1.5-7B-Chat-GGUF
WizardLM-2-7B-GGUFOpen In Colab2024-04-187B; Censoredhttps://huggingface.co/MaziyarPanahi/WizardLM-2-7B-GGUF
Meta-Llama-3-8B-Instruct-GGUFOpen In Colab2024-04-198B; Censoredhttps://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF
dolphin-2.9-llama3-8b-gguffOpen In Colab2024-04-228B; Uncensored; logic seems degraded from Llama-3-8Bhttps://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b-gguff
Lexi-Llama-3-8B-Uncensored-GGUFOpen In Colab2024-04-248B; Uncensoredhttps://huggingface.co/Orenguteng/Lexi-Llama-3-8B-Uncensored-GGUF
Llama3-8B-Chinese-Chat-GGUFOpen In Colab2024-04-248B; Chinesehttps://huggingface.co/QuantFactory/Llama3-8B-Chinese-Chat-GGUF
Phi-3-mini-4k-instruct-ggufOpen In Colab2024-04-243.8B; Censored; Fasthttps://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf
Llama-3-8B-Instruct-32k-v0.1-GGUFOpen In Colab2024-04-258B; 32K context; good for summarization of long texthttps://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-32k-v0.1-GGUF
starcoder2-15b-instruct-v0.1-GGUFOpen In Colab2024-05-0615B; coding modelhttps://huggingface.co/bartowski/starcoder2-15b-instruct-v0.1-GGUF
Hermes-2-Pro-Llama-3-8B-GGUFOpen In Colab2024-05-068B; Partially censored; JSON, Tool Use, etc.https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF
Llama-3-ChatQA-1.5-8B-GGUFOpen In Colab2024-05-158B; Uncensoredhttps://huggingface.co/bartowski/Llama-3-ChatQA-1.5-8B-GGUFF
Hermes-2-Theta-Llama-3-8B-GGUFOpen In Colab2024-05-178B; Censored; JSON, Tool Use, etchttps://huggingface.co/NousResearch/Hermes-2-Theta-Llama-3-8B-GGUF
blossom-v5.1-9b-GGUFOpen In Colab2024-05-179B; Wizard/Orca/Math/Chinese/English mixed datasethttps://huggingface.co/bartowski/blossom-v5.1-9b-GGUF
falcon-11B-GGUFOpen In Colab2024-05-1711B; base raw pretrained modelhttps://huggingface.co/bartowski/falcon-11B-GGUF
llama-3-cat-8b-instruct-v1-GGUFFOpen In Colab2024-05-208B; partially censored; system instruction fidelity; Role playhttps://huggingface.co/bartowski/llama-3-cat-8b-instruct-v1-GGUFF
Yi-1.5-9B-Chat-GGUFOpen In Colab2024-05-209B; Censoredhttps://huggingface.co/bartowski/Yi-1.5-9B-Chat-GGUF
SFR-Iterative-DPO-LLaMA-3-8B-R-GGUFOpen In Colab2024-05-228B; Partially censoredhttps://huggingface.co/bartowski/SFR-Iterative-DPO-LLaMA-3-8B-R-GGUF
Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-ImatrixOpen In Colab2024-05-228B; Roleplay; Trained to not decline requests; Seems to be trained on Quora datahttps://huggingface.co/Lewdiculous/Llama-3-Lumimaid-8B-v0.1-OAS-GGUF-IQ-Imatrix
Mistral-7B-Instruct-v0.3-GGUFOpen In Colab2024-05-237B; Censoredhttps://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF
L3-8B-Stheno-v3.1-GGUF-IQ-ImatrixOpen In Colab2024-05-308B; trained for 1 on 1 uncensored roleplayhttps://huggingface.co/Lewdiculous/L3-8B-Stheno-v3.1-GGUF-IQ-Imatrix
aya-23-8B-GGUFOpen In Colab2024-05-308B; Censoredhttps://huggingface.co/bartowski/aya-23-8B-GGUF
LLaMA3-iterative-DPO-final-GGUFOpen In Colab2024-05-308B; Censoredhttps://huggingface.co/bartowski/LLaMA3-iterative-DPO-final-GGUF
openchat-3.6-8b-20240522-GGUFOpen In Colab2024-06-048B; Partially censoredhttps://huggingface.co/bartowski/openchat-3.6-8b-20240522-GGUF
Meta-Llama-3-8B-Instruct-abliterated-v3-GGUFOpen In Colab2024-06-048B; Uncensored;https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF
NeuralDaredevil-8B-abliterated-GGUFOpen In Colab2024-06-198B; Uncensored;https://huggingface.co/QuantFactory/NeuralDaredevil-8B-abliterated-GGUF
Qwen2-7B-Instruct-GGUFOpen In Colab2024-06-247B; Censoredhttps://huggingface.co/Qwen/Qwen2-7B-Instruct-GGUF
DeepSeek-Coder-V2-Lite-Instruct-GGUFOpen In Colab2024-06-2716B; Instruct coding modelhttps://huggingface.co/lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF
internlm2_5-7b-chat-ggufOpen In Colab2024-07-117B; Censored; Long context; Reasoning and Tool Use;https://huggingface.co/internlm/internlm2_5-7b-chat-gguf
gemma-2-9b-it-GGUFOpen In Colab2024-07-119B; Censoredhttps://huggingface.co/bartowski/gemma-2-9b-it-GGUF
Smegmma-Deluxe-9B-v1-GGUFOpen In Colab2024-07-129B; Less censored gemma (still refuses some requests); Role playhttps://huggingface.co/bartowski/Smegmma-Deluxe-9B-v1-GGUF
h2o-danube3-4b-chat-GGUFOpen In Colab2024-07-174B; Partially censoredhttps://huggingface.co/h2oai/h2o-danube3-4b-chat-GGUF
Tiger-Gemma-9B-v1-GGUFOpen In Colab2024-07-179B; Uncensored; but appears to have some flaws in logichttps://huggingface.co/bartowski/Tiger-Gemma-9B-v1-GGUF
Gemmasutra-9B-v1-GGUFOpen In Colab2024-07-249B; Roleplay with some censorshiphttps://huggingface.co/TheDrummer/Gemmasutra-9B-v1-GGUF
Meta-Llama-3.1-8B-Instruct-GGUFOpen In Colab2024-07-258B; Censoredhttps://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF
Mistral-Nemo-Instruct-2407-GGUFOpen In Colab2024-07-2512.2B; Partially Uncensored; Recommended by Redditorshttps://huggingface.co/second-state/Mistral-Nemo-Instruct-2407-GGUF
Celeste-12B-V1.6_iMatrix_GGUFOpen In Colab2024-07-3012B; Roleplay & story writing model, Uncensoredhttps://huggingface.co/MarsupialAI/Celeste-12B-V1.6_iMatrix_GGUF
Hermes-3-Llama-3.1-8B-GGUFOpen In Colab2024-08-198B; Uncensoredhttps://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF
Gemma-2-9B-It-SPPO-Iter3-GGUFOpen In Colab2024-08-199B; Recommended on Reddit for summarizationhttps://huggingface.co/bartowski/Gemma-2-9B-It-SPPO-Iter3-GGUF
Llama-3.1-Storm-8B-GGUFOpen In Colab2024-08-208B; Censoredhttps://huggingface.co/bartowski/Llama-3.1-Storm-8B-GGUF
Phi-3.5-mini-instruct-GGUFOpen In Colab2024-08-213.5B; Censored, small & fasthttps://huggingface.co/lmstudio-community/Phi-3.5-mini-instruct-GGUF
Phi-3.5-mini-instruct_Uncensored-GGUFOpen In Colab2024-08-263.5B; Uncensored, small & fast, but has logic flaws & inconsistent resultshttps://huggingface.co/bartowski/Phi-3.5-mini-instruct_Uncensored-GGUF
NemoMix-Unleashed-12B-GGUFOpen In Colab2024-08-2612B; Partially censored, Roleplaying & story tellinghttps://huggingface.co/bartowski/NemoMix-Unleashed-12B-GGUF
Mistral-NeMo-Minitron-8B-Base-GGUFOpen In Colab2024-08-268B; Base modelhttps://huggingface.co/QuantFactory/Mistral-NeMo-Minitron-8B-Base-GGUF
Yi-Coder-9B-Chat-GGUFOpen In Colab2024-09-059B; Instruction coding modelhttps://huggingface.co/bartowski/Yi-Coder-9B-Chat-GGUF
Llama-3.2-3B-Instruct-Q8_0-GGUFOpen In Colab2024-10-013B; Small censored modelhttps://huggingface.co/hugging-quants/Llama-3.2-3B-Instruct-Q8_0-GGUF
Qwen2.5-7B-Instruct-GGUFOpen In Colab2024-10-017B; Censored, Recommended on r/LocalLLaMAhttps://huggingface.co/bartowski/Qwen2.5-7B-Instruct-GGUF
Qwen2.5-Coder-7B-Instruct-GGUFOpen In Colab2024-10-017B; Instruction coding model, recommended on r/LocalLLaMAhttps://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF
Llama-3.1-8B-Lexi-Uncensored-V2-GGUFOpen In Colab2024-10-158B; Partially censoredhttps://huggingface.co/Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2-GGUF
SuperNova-Medius-GGUFOpen In Colab2024-10-1514B; Censored, Recommended on r/LocalLLaMAhttps://huggingface.co/arcee-ai/SuperNova-Medius-GGUF
Llama-3.2-3B-Instruct-uncensored-GGUFOpen In Colab2024-10-153B; Small, uncensoredhttps://huggingface.co/mradermacher/Llama-3.2-3B-Instruct-uncensored-GGUF
Ministral-8B-Instruct-2410-HF-GGUF-TESTOpen In Colab2024-10-218B; Partially censoredhttps://huggingface.co/bartowski/Ministral-8B-Instruct-2410-HF-GGUF-TEST
granite-3.0-8b-instruct-GGUFOpen In Colab2024-10-288B; Censoredhttps://huggingface.co/bartowski/granite-3.0-8b-instruct-GGUF
aya-expanse-8b-GGUFOpen In Colab2024-10-288B; Censoredhttps://huggingface.co/bartowski/aya-expanse-8b-GGUF
Qwen2.5-Coder-14B-Instruct-GGUFOpen In Colab2024-11-1214B; Coding model, good at its sizehttps://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct-GGUF
Marco-o1-GGUFOpen In Colab2024-11-257B; CoT model to mimic o1 for reasoninghttps://huggingface.co/bartowski/Marco-o1-GGUF
Llama-Sentient-3.2-3B-Instruct-GGUFOpen In Colab2024-11-253B; Censoredhttps://huggingface.co/prithivMLmods/Llama-Sentient-3.2-3B-Instruct-GGUF
Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-GGUFOpen In Colab2024-12-0510B; Uncensored story writing modelhttps://huggingface.co/DavidAU/Gemma-The-Writer-N-Restless-Quill-10B-Uncensored-GGUF
INTELLECT-1-Instruct-GGUFOpen In Colab2024-12-0510B; Censoredhttps://huggingface.co/lmstudio-community/INTELLECT-1-Instruct-GGUF
L3.1-RP-Hero-BigTalker-8B-GGUFOpen In Colab2024-12-058B; Uncensored NSFW roleplay modelhttps://huggingface.co/DavidAU/L3.1-RP-Hero-BigTalker-8B-GGUF
EXAONE-3.5-7.8B-Instruct-GGUFOpen In Colab2024-12-097.8B; Censored, English and Korean, context up to 32Khttps://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-GGUF
Virtuoso-Small-GGUFOpen In Colab2024-12-1014B; Censored, good for general tasks, recommended on Reddithttps://huggingface.co/bartowski/Virtuoso-Small-GGUF
phi-4Open In Colab2024-12-1614B; Censoredhttps://huggingface.co/matteogeniaccio/phi-4
Hermes-3-Llama-3.2-3B-GGUFOpen In Colab2024-12-213B; Partially censoredhttps://huggingface.co/NousResearch/Hermes-3-Llama-3.2-3B-GGUF
Falcon3-Mamba-7B-Instruct-GGUFOpen In Colab2024-12-217B; Mamba1 architecture, 32K contexthttps://huggingface.co/tiiuae/Falcon3-Mamba-7B-Instruct-GGUF
QwQ-LCoT-7B-Instruct-GGUFOpen In Colab2024-12-217B; finetuned on CoThttps://huggingface.co/prithivMLmods/QwQ-LCoT-7B-Instruct-GGUF
granite-3.1-8b-instruct-GGUFOpen In Colab2024-12-248B; multilingual, 128K contexthttps://huggingface.co/bartowski/granite-3.1-8b-instruct-GGUF
granite-3.1-3b-a800m-instruct-GGUFOpen In Colab2024-12-243B; MoE, 128K contexthttps://huggingface.co/bartowski/granite-3.1-3b-a800m-instruct-GGUF
Falcon3-10B-Instruct-GGUFOpen In Colab2025-01-2110B; 32K context, English/French/Spanish/Portuguesehttps://huggingface.co/tiiuae/Falcon3-10B-Instruct-GGUF
DeepSeek-R1-Distill-Llama-8B-GGUFOpen In Colab2025-01-218B; Reasoning model, 128K contexthttps://huggingface.co/bartowski/DeepSeek-R1-Distill-Llama-8B-GGUF
DeepSeek-R1-Distill-Qwen-7B-GGUFOpen In Colab2025-02-067B; Reasoning model, 128K context, censoredhttps://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF
DeepSeek-R1-Distill-Qwen-14B-GGUFOpen In Colab2025-02-0614B; Reasoning model, 128K context, censoredhttps://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-14B-GGUF
Xwen-7B-Chat-GGUFOpen In Colab2025-02-067B; Uncensoredhttps://huggingface.co/mradermacher/Xwen-7B-Chat-GGUF
Virtuoso-Small-v2-GGUFOpen In Colab2025-02-0614B; Censoredhttps://huggingface.co/bartowski/arcee-ai_Virtuoso-Small-v2-GGUF
DeepHermes-3-Llama-3-8B-Preview-GGUFOpen In Colab2025-02-148B; Reasoning model, partially censoredhttps://huggingface.co/NousResearch/DeepHermes-3-Llama-3-8B-Preview-GGUF
Reverb-7b-Q4_K_M-GGUFOpen In Colab2025-02-207B; Censoredhttps://huggingface.co/ozone-ai/Reverb-7b-Q4_K_M-GGUF
Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUFOpen In Colab2025-03-038x3B; MoE, Uncensored NSFW, roleplay, writinghttps://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF
google_gemma-3-12b-it-GGUFOpen In Colab2025-03-1712B; Censoredhttps://huggingface.co/bartowski/google_gemma-3-12b-it-GGUF
microsoft_Phi-4-mini-instruct-GGUFOpen In Colab2025-03-173.8B; Censored, 128 K contexthttps://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF
RekaAI_reka-flash-3-GGUFOpen In Colab2025-03-1821B; Reasoning model, censoredhttps://huggingface.co/bartowski/RekaAI_reka-flash-3-GGUF
mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUFOpen In Colab2025-03-1924B; Censored, unlike previous mistral modelshttps://huggingface.co/bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF
agentica-org_DeepCoder-14B-Preview-GGUFOpen In Colab2025-04-0914B; Reasoning coding modelhttps://huggingface.co/bartowski/agentica-org_DeepCoder-14B-Preview-GGUF
deepcogito_cogito-v1-preview-qwen-14B-GGUFOpen In Colab2025-04-0914B; Censoredhttps://huggingface.co/bartowski/deepcogito_cogito-v1-preview-qwen-14B-GGUF
Qwen_Qwen3-14B-GGUFOpen In Colab2025-04-2914B; Censoredhttps://huggingface.co/bartowski/Qwen_Qwen3-14B-GGUF
ReadyArt_Amoral-Fallen-Omega-Gemma3-12B-GGUFOpen In Colab2025-04-2912B; Uncensored NSFW roleplay, but may deny non-sexual illegal requestshttps://huggingface.co/bartowski/ReadyArt_Amoral-Fallen-Omega-Gemma3-12B-GGUF
soob3123_Veritas-12B-GGUFOpen In Colab2025-04-2912B; Partially censoredhttps://huggingface.co/bartowski/soob3123_Veritas-12B-GGUF
THUDM_GLM-4-9B-0414-GGUFOpen In Colab2025-04-299B; Censoredhttps://huggingface.co/bartowski/THUDM_GLM-4-9B-0414-GGUF
microsoft_Phi-4-reasoning-plus-GGUFOpen In Colab2025-05-0614B; Reasoning model, Censoredhttps://huggingface.co/bartowski/microsoft_Phi-4-reasoning-plus-GGUF
Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-GGUFOpen In Colab2025-05-068B; Uncensored modelhttps://huggingface.co/bartowski/Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-GGUF
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUFOpen In Colab2025-06-098B; Reasoning model, Censoredhttps://huggingface.co/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
open-thoughts_OpenThinker3-7B-GGUFOpen In Colab2025-06-097B; Reasoning model, Uncensored, tends to think a lot before answeringhttps://huggingface.co/bartowski/open-thoughts_OpenThinker3-7B-GGUF
Menlo_Jan-nano-128k-GGUFOpen In Colab2025-06-264B; Long-context model optimized for MCP use and deep researchhttps://huggingface.co/bartowski/Menlo_Jan-nano-128k-GGUF
google_gemma-3n-E4B-it-GGUFOpen In Colab2025-07-108B with 4B activated; Censored, 32K context, efficient parameter managementhttps://huggingface.co/bartowski/google_gemma-3n-E4B-it-GGUF
janhq_Jan-v1-4B-GGUFOpen In Colab2025-08-154B; Optimized for search and deep researchhttps://huggingface.co/bartowski/janhq_Jan-v1-4B-GGUF
Qwen3-Coder-30B-A3B-Instruct-GGUFOpen In Colab2025-08-1530B with 3 B activated; Coding modelhttps://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF
Qwen3-30B-A3B-Instruct-2507-GGUFOpen In Colab2025-08-1530B with 3 B activated; Censoredhttps://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF
Qwen3-4B-Instruct-2507-GGUFOpen In Colab2025-08-154B; Censoredhttps://huggingface.co/unsloth/Qwen3-4B-Instruct-2507-GGUF
NousResearch_Hermes-4-14B-GGUFOpen In Colab2025-09-0914B; Partially censoredhttps://huggingface.co/bartowski/NousResearch_Hermes-4-14B-GGUF
VibeThinker-1.5B-GGUFFOpen In Colab2025-11-241.5B; Math and Algo Coding modelhttps://huggingface.co/MaziyarPanahi/VibeThinker-1.5B-GGUFF
Ministral-3-14B-Instruct-2512-GGUFOpen In Colab2025-12-0814B; Multi-modal, Agentic, Censored modelhttps://huggingface.co/unsloth/Ministral-3-14B-Instruct-2512-GGUF
Ministral-3-14B-Reasoning-2512-GGUFOpen In Colab2025-12-0814B; Reasoning, Multi-modal, Agentic, Censored modelhttps://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF
NousResearch_NousCoder-14B-GGUFOpen In Colab2026-01-1314B; Coding modelhttps://huggingface.co/bartowski/NousResearch_NousCoder-14B-GGUF
nvidia_Nemotron-Cascade-14B-Thinking-GGUFOpen In Colab2026-01-1315B; Coding, reasoning modelhttps://huggingface.co/bartowski/nvidia_Nemotron-Cascade-14B-Thinking-GGUF

Star History

Star History Chart

Share This Project

If you find this project helpful, please share it:

You may also be interested in

Check out awesome-TTS-Colab for a curated list of Google Colab notebooks featuring open source text-to-speech (TTS) models. These notebooks are easy to use and can also be run on your local machine!

Older Models with Scores

Older Models with Scores:

These models work better among the models I tested on my hardware (i5-12490F, 32GB RAM, RTX 3060 Ti GDDR6X 8GB VRAM): (Note: Because llama.cpp has made some breaking changes to the support of older ggml models. Some older ggml versions listed below may not work properly on current llama.cpp. But there should be GPTQ equivalents or newer ggml versions for the models.)

NOTE:

  • Major edit on June 30, 2023. Since I noticed GPT-4 started to give more granular scores and tends to give higher scores, I have decided to re-run the tests for all models so they can be compared (All evaluations done with GPT-4-0613 for consistency). The scores in the table below are the average of the scores from the latest tests. The scores in the table below are not comparable to the scores in the previous versions of this README.
Model_NameAvg_ScoreColab_LinkDate_AddedLink
Mistral-7B-OpenOrca (using oobabooga/text-generation-webui)10.00Open In Colab2023/10/08https://huggingface.co/TheBloke/Mistral-7B-OpenOrca-GGUF
Llama-2-13B-chat (using oobabooga/text-generation-webui)9.65Open In Colab2023/07/20https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML
wizard-vicuna-13B.ggml.q4_0 (using llama.cpp)9.63Open In Colab2023/05/07https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML
Nous-Capybara-7B (using oobabooga/text-generation-webui)9.56Open In Colab2023/10/08https://huggingface.co/TheBloke/Nous-Capybara-7B-GGUF
vicuna-13B-v1.5 (using oobabooga/text-generation-webui)9.53Open In Colab2023/08/09https://huggingface.co/TheBloke/vicuna-13B-v1.5-GGML
wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui)9.53Open In Colab2023/05/29https://huggingface.co/TheBloke/wizardLM-13B-1.0-GPTQ
airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui)9.50Open In Colab2023/06/30https://huggingface.co/TheBloke/airoboros-13B-gpt4-1.4-GPTQ
Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui)9.44Open In Colab2023/06/03https://huggingface.co/TheBloke/Nous-Hermes-13B-GPTQ/tree/main
Dolphin-Llama-13B (using oobabooga/text-generation-webui)9.38Open In Colab2023/07/24https://huggingface.co/TheBloke/Dolphin-Llama-13B-GGML
Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui)9.37Open In Colab2023/10/08https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF
OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui)9.37Open In Colab2023/08/15https://huggingface.co/TheBloke/OpenOrca-Platypus2-13B-GGML
airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui)9.34Open In Colab2023/08/01https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-2.0-GGML
Chronos-13B-v2 (using oobabooga/text-generation-webui)9.31Open In Colab2023/08/09https://huggingface.co/TheBloke/Chronos-13B-v2-GGML
vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui)9.31Open In Colab2023/06/29https://huggingface.co/TheBloke/vicuna-13b-v1.3.0-GPTQ
MythoLogic-13B (using oobabooga/text-generation-webui)9.31Open In Colab2023/07/20https://huggingface.co/TheBloke/MythoLogic-13B-GGML
Selfee-13B-GPTQ (using oobabooga/text-generation-webui)9.28Open In Colab2023/06/07https://huggingface.co/TheBloke/Selfee-13B-GPTQ
WizardLM-13B-V1.2 (using oobabooga/text-generation-webui)9.28Open In Colab2023/07/26https://huggingface.co/TheBloke/WizardLM-13B-V1.2-GGML
minotaur-13B-GPTQ (using oobabooga/text-generation-webui)9.28Open In Colab2023/06/09https://huggingface.co/TheBloke/minotaur-13B-GPTQ
Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui)9.20Open In Colab2023/09/21https://huggingface.co/TheBloke/Pygmalion-2-13B-SuperCOT2-GGUF
Athena-v1 (using oobabooga/text-generation-webui)9.19Open In Colab2023/08/31https://huggingface.co/TheBloke/Athena-v1-GGUF
PuddleJumper-13B (using oobabooga/text-generation-webui)9.1875Open In Colab2023/08/29https://huggingface.co/TheBloke/PuddleJumper-13B-GGUF
Nous-Hermes-Llama2 (using oobabooga/text-generation-webui)9.17Open In Colab2023/07/24https://huggingface.co/TheBloke/Nous-Hermes-Llama2-GGML
Luban-13B (using oobabooga/text-generation-webui)9.15Open In Colab2023/08/31https://huggingface.co/TheBloke/Luban-13B-GGUF
MythoBoros-13B (using oobabooga/text-generation-webui)9.15Open In Colab2023/07/24https://huggingface.co/TheBloke/MythoBoros-13B-GGML
13B-Ouroboros (using oobabooga/text-generation-webui)9.11Open In Colab2023/07/27https://huggingface.co/TheBloke/13B-Ouroboros-GGML
tulu-13B-GPTQ (using oobabooga/text-generation-webui)9.06Open In Colab2023/06/13https://huggingface.co/TheBloke/tulu-13B-GPTQ
AlpacaCielo-13B (using oobabooga/text-generation-webui)9.03125Open In Colab2023/07/27https://huggingface.co/TheBloke/AlpacaCielo-13B-GGML
StableBeluga-13B (using oobabooga/text-generation-webui)9Open In Colab2023/08/01https://huggingface.co/TheBloke/StableBeluga-13B-GGML
Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui)8.97Open In Colab2023/08/10https://huggingface.co/TheBloke/Chronos-Hermes-13B-v2-GGML
OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui)8.97Open In Colab2023/09/05https://huggingface.co/TheBloke/OpenBuddy-Llama2-13B-v11.1-GGUF
Camel-Platypus2-13B (using oobabooga/text-generation-webui)8.94Open In Colab2023/08/15https://huggingface.co/TheBloke/Camel-Platypus2-13B-GGML
airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui)8.94Open In Colab2023/09/21https://huggingface.co/TheBloke/airoboros-l2-13b-gpt4-m2.0-GGUF
UltraLM-13B-GPTQ (using oobabooga/text-generation-webui)8.89Open In Colab2023/06/30https://huggingface.co/TheBloke/UltraLM-13B-GPTQ
13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui)8.88Open In Colab2023/06/03https://huggingface.co/digitous/13B-HyperMantis_GPTQ_4bit-128g/
Stable-Platypus2-13B (using oobabooga/text-generation-webui)8.875Open In Colab2023/08/15https://huggingface.co/TheBloke/Stable-Platypus2-13B-GGML
Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui)8.84Open In Colab2023/05/25https://huggingface.co/TheBloke/airoboros-13B-GPTQ
Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui)8.84Open In Colab2023/09/21https://huggingface.co/TheBloke/Kuchiki-1.1-L2-7B-GGUF
WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui)8.80625Open In Colab2023/08/09https://huggingface.co/TheBloke/WizardLM-1.0-Uncensored-Llama2-13B-GGML
Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui)8.75Open In Colab2023/08/10https://huggingface.co/TheBloke/Chronos-Beluga-v2-13B-GGML
Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui)8.75Open In Colab2023/06/09https://huggingface.co/TheBloke/Vicuna-13B-CoT-GPTQ
wizardLM-7B.q4_2 (in GPT4All)8.75No2023/05/07https://gpt4all.io/models/ggml-wizardLM-7B.q4_2.bin
OpenChat_v3.2 (using oobabooga/text-generation-webui)8.71875Open In Colab2023/08/01https://huggingface.co/TheBloke/OpenChat_v3.2-GGML
Huginn-13B (using oobabooga/text-generation-webui)8.7125Open In Colab2023/08/10https://huggingface.co/TheBloke/Huginn-13B-GGML
WizardLM-13B-V1.1 (using oobabooga/text-generation-webui)8.66Open In Colab2023/07/17https://huggingface.co/TheBloke/WizardLM-13B-V1.1-GGML
robin-13B-v2-GPTQ (using oobabooga/text-generation-webui)8.66Open In Colab2023/06/19https://huggingface.co/TheBloke/robin-13B-v2-GPTQ
llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui)8.625Open In Colab2023/07/21https://huggingface.co/TheBloke/llama-2-13B-Guanaco-QLoRA-GGML
mpt-7b-chat (in GPT4All)8.53No2023/05/11https://gpt4all.io/models/ggml-mpt-7b-chat.bin
chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui)8.48125Open In Colab2023/06/16https://huggingface.co/TheBloke/chronos-hermes-13B-GPTQ
Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui)8.46875Open In Colab2023/07/20https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGML
stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui)8.25Open In Colab2023/05/12https://huggingface.co/TheBloke/stable-vicuna-13B-GPTQ
manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui)8.21875Open In Colab2023/05/24https://huggingface.co/TheBloke/manticore-13b-chat-pyg-GPTQ
CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui)8.09375Open In Colab2023/06/10https://huggingface.co/TheBloke/CAMEL-13B-Combined-Data-GPTQ
WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui)8.09375Open In Colab2023/06/02https://huggingface.co/TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ
llama-13b-supercot-GGML (using oobabooga/text-generation-webui)8.01Open In Colab2023/07/05https://huggingface.co/TheBloke/llama-13b-supercot-GGML
Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui)7.96875Open In Colab2023/05/24https://huggingface.co/TheBloke/Project-Baize-v2-13B-GPTQ
koala-13B-4bit-128g.GGML (using llama.cpp)7.9375No2023/05/07https://huggingface.co/TheBloke/koala-13B-GPTQ-4bit-128g-GGML
wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui)7.90625Open In Colab2023/05/19https://huggingface.co/4bit/WizardLM-13B-Uncensored-4bit-128g
vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui)7.875Open In Colab2023/06/29https://huggingface.co/TheBloke/vicuna-7B-v1.3-GPTQ
Manticore-13B-GPTQ (using oobabooga/text-generation-webui)7.78125Open In Colab2023/05/23https://huggingface.co/TheBloke/Manticore-13B-GPTQ
vicuna-13b-1.1-q4_2 (in GPT4All)7.75No2023/05/07https://gpt4all.io/models/ggml-vicuna-13b-1.1-q4_2.bin
falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui)7.625Open In Colab2023/06/02https://huggingface.co/TheBloke/falcon-7b-instruct-GPTQ
guanaco-13B-GPTQ (using oobabooga/text-generation-webui)7.5625Open In Colab2023/05/26https://huggingface.co/TheBloke/guanaco-13B-GPTQ
Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui)7.31Open In Colab2023/08/31https://huggingface.co/TheBloke/Mythical-Destroyer-V2-L2-13B-GGUF
Kimiko-v2-13B (using oobabooga/text-generation-webui)7.25 Open In Colab2023/08/31https://huggingface.co/TheBloke/Kimiko-v2-13B-GGUF
orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui)7.0875Open In Colab2023/06/28https://huggingface.co/TheBloke/orca_mini_13B-GGML
Platypus2-13B (using oobabooga/text-generation-webui)7.03125Open In Colab2023/08/15https://huggingface.co/TheBloke/Platypus2-13B-GGML
Redmond-Puffin-13B (using oobabooga/text-generation-webui)7.03125Open In Colab2023/07/20https://huggingface.co/TheBloke/Redmond-Puffin-13B-GGML
13B-BlueMethod (using oobabooga/text-generation-webui)7.025Open In Colab2023/07/24https://huggingface.co/TheBloke/13B-BlueMethod-GGML
mpt-7b-instruct6.6875No2023/05/12https://huggingface.co/TheBloke/MPT-7B-Instruct-GGML
Kimiko-13B (using oobabooga/text-generation-webui)6.46875Open In Colab2023/08/01https://huggingface.co/TheBloke/Kimiko-13B-GGML
gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp)6.0625No2023/05/07https://huggingface.co/Bradarr/gpt4-x-alpaca-13b-native-ggml-model-q4_0
minotaur-15B-GPTQ (using oobabooga/text-generation-webui)5.9375Open In Colab2023/06/26https://huggingface.co/TheBloke/minotaur-15B-GPTQ
baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui)5.90625Open In Colab2023/07/05https://huggingface.co/TheBloke/baichuan-vicuna-7B-GGML
gpt4all-j-v1.3-groovy (in GPT4All)5.6875No2023/05/07https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin

Many thanks to:
❤️ GPT4ALl: https://github.com/nomic-ai/gpt4all-chat
❤️ llama.cpp: https://github.com/ggerganov/llama.cpp
❤️ oobabooga text generation webui: https://github.com/oobabooga/text-generation-webui
❤️ Colab webui inspired by camenduru: https://github.com/camenduru/text-generation-webui-colab/tree/main
❤️ The Bloke for quantization of the models: https://huggingface.co/TheBloke

Coding models tested & average scores:

(All scores are from GPT-4-0613.)

Model_NameAvg_ScoresColab_LinkDate_AddedLink
CodeLlama-13B-oasst-sft-v10 (using oobabooga/text-generation-webui)9.8Open In Colab2023/08/28https://huggingface.co/TheBloke/CodeLlama-13B-oasst-sft-v10-GGUF
WizardCoder-Python-13B-V1.0 (using oobabooga/text-generation-webui)9.5Open In Colab2023/08/28https://huggingface.co/TheBloke/WizardCoder-Python-13B-V1.0-GGUF
Redmond-Hermes-Coder-GPTQ (using oobabooga/text-generation-webui)8.4Open In Colab2023/07/03https://huggingface.co/TheBloke/Redmond-Hermes-Coder-GPTQ
CodeUp-Alpha-13B-HF (using oobabooga/text-generation-webui)7.9Open In Colab2023/08/15https://huggingface.co/TheBloke/CodeUp-Alpha-13B-HF-GGML
starchat-beta-GPTQ (using oobabooga/text-generation-webui)7.6Open In Colab2023/07/04https://huggingface.co/TheBloke/starchat-beta-GPTQ
wizard-vicuna-13B-GPTQ (using oobabooga/text-generation-webui)7.3Open In Colab2023/07/03https://huggingface.co/TheBloke/wizard-vicuna-13B-GPTQ
WizardCoder-Guanaco-15B-V1.1 (using oobabooga/text-generation-webui)7.1Open In Colab2023/07/21https://huggingface.co/TheBloke/WizardCoder-Guanaco-15B-V1.1-GPTQ
CodeLlama-13B-Instruct (using oobabooga/text-generation-webui)7Open In Colab2023/08/28https://huggingface.co/TheBloke/CodeLlama-13B-Instruct-GGUF
CodeUp-Llama-2-13B-Chat-HF (using oobabooga/text-generation-webui)6Open In Colab2023/08/03https://huggingface.co/TheBloke/CodeUp-Llama-2-13B-Chat-HF-GGML
WizardCoder-15B-1.0-GPTQ (using oobabooga/text-generation-webui)5.9Open In Colab2023/07/03https://huggingface.co/TheBloke/WizardCoder-15B-1.0-GPTQ
wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui)5.9Open In Colab2023/05/29https://huggingface.co/TheBloke/wizardLM-13B-1.0-GPTQ

Questions and scores

Original responses can be found at: https://docs.google.com/spreadsheets/d/1ogDXUiaBx3t7EpMo44aaA6U6kLXX0x2tGRgLg8CISGs/edit?usp=sharing

Question 1: Translate the following English text into French: "The sun rises in the east and sets in the west."

Task Domain: Translation
Expected Good Response: "Le soleil se lève à l'est et se couche à l'ouest."
Explanation: This task tests the model's ability to understand and accurately translate text between languages.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 1
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 8
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 7
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 1
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 1
  • mpt-7b-instruct : 1
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 7
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 8
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 1
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 8
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 8
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 1
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 7
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 8
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 1
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 1
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 1
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 8
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 7
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 8
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 1
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 8.5
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 6
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 7.5
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 4
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 9
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 7.5
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 7.5
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 9.5
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 9.5
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 9.5
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 9.5

Question 2: Summarize the following text: "The water cycle is a natural process that involves the continuous movement of water on, above, and below the Earth's surface. It includes various stages like evaporation, condensation, precipitation, and runoff. This cycle plays a crucial role in maintaining Earth's water balance and supporting life."

Task Domain: Summary
Expected Good Response: "The water cycle is the continuous movement of water on Earth, crucial for maintaining water balance and supporting life."
Explanation: This task evaluates the model's ability to extract the main points from a given text and generate a concise summary.

  • wizardLM-7B.q4_2 (in GPT4All) : 5
  • gpt4all-j-v1.3-groovy (in GPT4All) : 7.5
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 9
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 4
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 7.5
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 1
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 8.5
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 7.5
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 9
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 9
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 5
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 9
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 9
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 8
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 1
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 9
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 9
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 8
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 7.5
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 1
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 1
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 7
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 7.5
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 9
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 7.5
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 7.5
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 9
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 5
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 7.5
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 9
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 9.5
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 9
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 5
  • Athena-v1 (using oobabooga/text-generation-webui) : 9
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 9.5
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 9
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 3: I want you to act as a senior software developer with deep knowledge in system design, frontend programming, and backend programming. Provide a high level design of a mental health journal app. Include the frontend and backend components. Do not write code.

Task Domain: App Design
Explanation: This task evaluates the model's ability to closely follow user's instruction for a complex task.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 9
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 7.5
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 6.5
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 8
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 9
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 9
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 9.5
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 3
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 9.5
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 8.5
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 7.4
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 4: What are the main causes of the French Revolution according to this passage: "The French Revolution, which took place between 1789 and 1799, was rooted in financial crises, social inequality, and Enlightenment ideas."

Task Domain: Abstractive Question Answering
Expected Good Response: Financial crises, social inequality, and Enlightenment ideas.
Explanation: This task tests the model's ability to understand the context and generate an answer in its own words.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 10
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 1
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 6
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 4
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 9.5
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 9
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 9.5
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 7.5
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 8
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 9
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 5: In the following text, identify the two artists mentioned: "During the Renaissance, Leonardo da Vinci and Michelangelo were two of the most influential artists who created masterpieces that continue to inspire people today."

Task Domain: Extractive Question Answering
Expected Good Response: Leonardo da Vinci, Michelangelo
Explanation: This task assesses the model's ability to extract specific information from a given text.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 10
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 9
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 9.5
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 8.5
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 6: Determine the sentiment of this customer review: "I had a fantastic experience at this restaurant. The food was delicious, and the service was outstanding."

Task Domain: Sentiment Analysis
Expected Good Response: Positive
Explanation: This task evaluates the model's ability to analyze text and identify the sentiment expressed.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 10
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 1
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 4
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 1
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 8.5
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 5
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 7: Classify the following text into one of these categories: Sports, Technology, Health, or Politics: "Apple recently unveiled its latest iPhone, featuring a faster processor and improved battery life."

Task Domain: Zero-shot Classification
Expected Good Response: Technology
Explanation: This task tests the model's ability to classify text into predefined categories without any prior training on the specific categories.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 5
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 1
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 2
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 4
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 10
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 8: Complete the following sentence by filling in the blank: "The capital city of France is _______."

Task Domain: Mask Filling
Expected Good Response: Paris
Explanation: This task assesses the model's ability to understand context and generate appropriate words to fill in missing information.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 10
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 5
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 1
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 5
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 9: Write a rhyming couplet about nature.

Task Domain: Poetry Generation
Expected Good Response: "In nature's beauty, we find respite and grace, A symphony of colors that time cannot erase."
Explanation: This task tests the model's ability to generate creative and coherent text that adheres to specific constraints, such as rhyme and theme.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 7.5
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 1
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 6.5
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 6
  • mpt-7b-instruct : 2
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 8
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 6.5
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 9
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 4
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 6
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 7
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 7.5
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 7
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 4
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 4
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 8
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 9
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 4
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 7
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 7.5
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 4
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 8
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 7
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 8
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 7.5
  • Kimiko-13B (using oobabooga/text-generation-webui) : 6.5
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 8.5
  • Platypus2-13B (using oobabooga/text-generation-webui) : 6
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 7.5
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 4
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 3
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 7.4
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 8
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 6.5
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 9

Question 10: Based on the following statement, determine if the author's opinion is for or against nuclear energy: "Nuclear energy is a powerful source, but the potential risks and radioactive waste management issues make it a dangerous choice."

Task Domain: Opinion Detection
Expected Good Response: Against
Explanation: This task evaluates the model's ability to understand and identify the author's stance or opinion on a specific topic.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 4
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 9.5
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 6
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 8.5
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 7.5
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 9.5
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 9.5
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 4
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 4
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 6.5
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 7
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 8.5
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 8.5
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 3
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 8.5
  • Platypus2-13B (using oobabooga/text-generation-webui) : 9
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 4
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 10
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 9.5
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 11: Rewrite the following sentence in passive voice: "The dog chased the cat."

Task Domain: Text Rewriting
Expected Good Response: "The cat was chased by the dog."
Explanation: This task tests the model's ability to manipulate and rewrite text according to specific grammatical requirements.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 3
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 10
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 10
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 10
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 10
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 10
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 6.5
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 10
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 10
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 9.5
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 7
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 7.34
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 12: Use the retrieved context to answer a question. Context does not contain the answer. (Prompt too long. See spreadsheet for original prompt)

Task Domain: Document Question Answering
Expected Good Response: "I don't know."
Explanation: This task tests the model's ability to understand context and answer questions based on the information provided in the context.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 1
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 1
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 1
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 10
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 1
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 1
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 3
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 10
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 1
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 1
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 1
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 10
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 1
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 1
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 10
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 10
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 10
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 1
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 7
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 10
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 1
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 10
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 1
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 10
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 1
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 10
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 10
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 1
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 10
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 10
  • Huginn-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 10
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 10
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • Platypus2-13B (using oobabooga/text-generation-webui) : 1
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 1
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 2
  • Athena-v1 (using oobabooga/text-generation-webui) : 10
  • Luban-13B (using oobabooga/text-generation-webui) : 10
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 10
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 1
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 10
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 13: Use the retrieved context to answer a question. Context contains the answer. (Prompt too long. See spreadsheet for original prompt)

Task Domain: Document Question Answering
Expected Good Response: "["Semantic text search", "Generative question-answering", "Hybrid search", "Image similarity search", "Product recommendations"]"
Explanation: This task tests the model's ability to understand context and answer questions based on the information provided in the context.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 1
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 1
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 4
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 5.5
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 4
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 1
  • mpt-7b-instruct : 10
  • wizard-lm-uncensored-13b-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 5
  • Manticore-13B-GPTQ (using oobabooga/text-generation-webui) : 2
  • manticore_13b_chat_pyg_GPTQ (using oobabooga/text-generation-webui) : 5
  • Project-Baize-v2-13B-GPTQ (using oobabooga/text-generation-webui) : 1
  • Airoboros-13B-GPTQ-4bit (using oobabooga/text-generation-webui) : 7
  • guanaco-13B-GPTQ (using oobabooga/text-generation-webui) : 8
  • wizardLM-13B-1.0-GPTQ (using oobabooga/text-generation-webui) : 8.5
  • falcon-7b-instruct-GPTQ (using oobabooga/text-generation-webui) : 10
  • WizardLM-Uncensored-Falcon-7B-GPTQ (using oobabooga/text-generation-webui) : 7.5
  • Nous-Hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 5
  • 13B-HyperMantis_GPTQ (using oobabooga/text-generation-webui) : 10
  • Selfee-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • minotaur-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • Vicuna-13B-CoT-GPTQ (using oobabooga/text-generation-webui) : 1
  • CAMEL_13B_Combined_Data_GPTQ (using oobabooga/text-generation-webui) : 8.5
  • tulu-13B-GPTQ (using oobabooga/text-generation-webui) : 10
  • chronos-hermes-13B-GPTQ (using oobabooga/text-generation-webui) : 5.7
  • robin-13B-v2-GPTQ (using oobabooga/text-generation-webui) : 6.5
  • minotaur-15B-GPTQ (using oobabooga/text-generation-webui) : 1
  • orca-mini-13b.ggmlv3.q5_K_M (using oobabooga/text-generation-webui) : 7.4
  • vicuna-13b-v1.3.0-GPTQ (using oobabooga/text-generation-webui) : 5
  • vicuna-7B-v1.3-GPTQ (using oobabooga/text-generation-webui) : 1
  • UltraLM-13B-GPTQ (using oobabooga/text-generation-webui) : 7.7
  • airoboros-13B-gpt4-1.4-GPTQ (using oobabooga/text-generation-webui) : 10
  • llama-13b-supercot-GGML (using oobabooga/text-generation-webui) : 5.6875
  • baichuan-vicuna-7B-GGML (using oobabooga/text-generation-webui) : 1
  • WizardLM-13B-V1.1 (using oobabooga/text-generation-webui) : 1
  • Llama-2-13B-chat (using oobabooga/text-generation-webui) : 10
  • Luna-AI-Llama2-Uncensored (using oobabooga/text-generation-webui) : 7.5
  • MythoLogic-13B (using oobabooga/text-generation-webui) : 7.4
  • Redmond-Puffin-13B (using oobabooga/text-generation-webui) : 10
  • llama-2-13B-Guanaco-QLoRA (using oobabooga/text-generation-webui) : 10
  • Dolphin-Llama-13B (using oobabooga/text-generation-webui) : 10
  • Nous-Hermes-Llama2 (using oobabooga/text-generation-webui) : 5.6875
  • 13B-BlueMethod (using oobabooga/text-generation-webui) : 7.4
  • MythoBoros-13B (using oobabooga/text-generation-webui) : 7.34
  • WizardLM-13B-V1.2 (using oobabooga/text-generation-webui) : 1
  • 13B-Ouroboros (using oobabooga/text-generation-webui) : 10
  • AlpacaCielo-13B (using oobabooga/text-generation-webui) : 10
  • OpenChat_v3.2 (using oobabooga/text-generation-webui) : 1
  • airoboros-l2-13b-gpt4-2.0 (using oobabooga/text-generation-webui) : 10
  • StableBeluga-13B (using oobabooga/text-generation-webui) : 10
  • Kimiko-13B (using oobabooga/text-generation-webui) : 3
  • Chronos-13B-v2 (using oobabooga/text-generation-webui) : 7.5
  • WizardLM-1.0-Uncensored-Llama2-13B (using oobabooga/text-generation-webui) : 1
  • Huginn-13B (using oobabooga/text-generation-webui) : 8.5
  • Chronos-Beluga-v2-13B (using oobabooga/text-generation-webui) : 7.5
  • Chronos-Hermes-13B-v2 (using oobabooga/text-generation-webui) : 5
  • vicuna-13B-v1.5 (using oobabooga/text-generation-webui) : 10
  • Stable-Platypus2-13B (using oobabooga/text-generation-webui) : 1
  • Platypus2-13B (using oobabooga/text-generation-webui) : 8
  • Camel-Platypus2-13B (using oobabooga/text-generation-webui) : 10
  • OpenOrca-Platypus2-13B (using oobabooga/text-generation-webui) : 7.4
  • PuddleJumper-13B (using oobabooga/text-generation-webui) : 4
  • Kimiko-v2-13B (using oobabooga/text-generation-webui) : 2
  • Mythical-Destroyer-V2-L2-13B (using oobabooga/text-generation-webui) : 2
  • Athena-v1 (using oobabooga/text-generation-webui) : 3
  • Luban-13B (using oobabooga/text-generation-webui) : 1
  • OpenBuddy-Llama2-13B-v11.1 (using oobabooga/text-generation-webui) : 1
  • Kuchiki-1.1-L2-7B (using oobabooga/text-generation-webui) : 10
  • Pygmalion-2-13B-SuperCOT2 (using oobabooga/text-generation-webui) : 3
  • airoboros-l2-13b-gpt4-m2.0 (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-OpenOrca (using oobabooga/text-generation-webui) : 10
  • Nous-Capybara-7B (using oobabooga/text-generation-webui) : 10
  • Mistral-7B-Instruct-v0.1 (using oobabooga/text-generation-webui) : 10

Question 14: What is the square root of banana?

Task Domain: Mathematical Reasoning
Expected Good Response: "The question is nonsensical, as square roots can only be calculated for numbers, not for objects or words like 'banana'."
Explanation: This task tests the model's ability to recognize questions that are illogical or nonsensical and respond accordingly.

  • wizardLM-7B.q4_2 (in GPT4All) : 10
  • gpt4all-j-v1.3-groovy (in GPT4All) : 1
  • vicuna-13b-1.1-q4_2 (in GPT4All) : 10
  • gpt4-x-alpaca-13b-ggml-q4_0 (using llama.cpp) : 1
  • koala-13B-4bit-128g.GGML (using llama.cpp) : 1
  • wizard-vicuna-13B.ggml.q4_0 (using llama.cpp) : 10
  • mpt-7b-chat (in GPT4All) : 10
  • stable-vicuna-13B-GPTQ-4bit-128g (using oobabooga/text-generation-webui) : 10
  • mpt-7b-instruct : 1
  • wizard-lm-uncensored-13b

Truncated — view the full README on GitHub.

ai
gpt
llama
llm

Significant stargazers

Govind Kamtamneni

152 followers · starred Nov 2023

oobabooga

5,031 followers · starred Aug 2023

Virat Singh

4,063 followers · starred Feb 2024

Rafał Studnicki

80 followers · starred Apr 2024

Similar repos

0xSojalSec/LLMs-local

list of awesome platforms, tools, and resources run for LLMs locally

1,196

rafska/awesome-local-llm

A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally

2,981

LudwigStumpp/llm-leaderboard

A joint community effort to create one central leaderboard for LLMs.

306

BarnacleLabs/awesome-llm-list

An overview of Large Language Model (LLM) options

75

janhq/awesome-local-ai

An awesome repository of local AI tools

2,039

camenduru/text-generation-webui-colab

A colab gradio web UI for running Large Language Models

2,089

ludwigstumpp/llm-leaderboard

Space

🏆 LLM-Leaderboard

18

jakemanger/whatmodel

Find the best AI models you can run

33