README
generic llama.cpp quants of Minthy/ToriiGate-0.5 | Using llama.cpp release b8720
You can find imatrix, K_L quants (--token-embedding-type & --output-tensor-type are in Q8_0) here:
- https://huggingface.co/SleepVeryHard/ToriiGate-0.5_GGUF
The README mentions that
GGUFs have been updated to include the correct chat template--reasoning off/-rea off should be added as llama.cpp cli arg to workaround an issue where the actual output is just in the reasoning part - I found that to be the case too and disabling reasoning helped.
Original model + more info like prompts/formats, examples, stats:
- https://huggingface.co/Minthy/ToriiGate-0.5
Tested the Q4KM variant with Q8 vision encoder:
Contributors
14 commits
Similar repos
README
generic llama.cpp quants of Minthy/ToriiGate-0.5 | Using llama.cpp release b8720
You can find imatrix, K_L quants (--token-embedding-type & --output-tensor-type are in Q8_0) here:
- https://huggingface.co/SleepVeryHard/ToriiGate-0.5_GGUF
The README mentions that
GGUFs have been updated to include the correct chat template--reasoning off/-rea off should be added as llama.cpp cli arg to workaround an issue where the actual output is just in the reasoning part - I found that to be the case too and disabling reasoning helped.
Original model + more info like prompts/formats, examples, stats:
- https://huggingface.co/Minthy/ToriiGate-0.5
Tested the Q4KM variant with Q8 vision encoder:
Similar repos
Linked in READMEs
Contributors
14 commits