K2-Horizon-0.9B-GGUF

[!NOTE] This repository contains GGUF versions of the IFM/K2-Horizon-0.9B for use with llama.cpp.

Multiple precision and quantization variants are available, including BF16, Q8_0, Q6_K, Q5_K_M, Q5_0, and Q4_K_M. All GGUF files include tokenizer metadata and a llama.cpp-compatible chat template.

[!WARNING] 💡 Important Deployment Notes: These models require a version of llama.cpp containing K2 Horizon architecture support. PR to llama.cpp is in progress. MBZUAI-IFM fork of llama.cpp is in https://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon

K2-Horizon-0.9B is the compact dense member of the K2-Horizon family: a 0.9B-class decoder-only model with a 128K context window.

K2-Horizon-0.9B Highlights

• Compact reasoning model. A 0.9B-class dense model evaluated across mathematics, coding, science, and tool-use benchmarks.
• 128K context. Supports up to 131,072 tokens with YaRN RoPE scaling.
• Multi-teacher distillation. Trained with domain teachers for math and code, STEM, and instruction following.

• Fully open. Training data/recipe and the training code will be made public.

Benchmark Results

The chart at the top of this card shows K2-Horizon-0.9B against selected reference models. The table below lists every comparison model used in the figure.

Full Results

Scores in %. Bold highlights K2-Horizon-0.9B; Qwen3.5-2B is included as a larger reference model. Protocol and provenance details are in the Technical Appendix.

GGUF BF16 vs. Quantized

The evaluation context length is set to 65,536 tokens. Unless otherwise specified, all tasks are evaluated in a 0-shot setting. The GGUF models have currently been evaluated only on non-agent tasks. Results for agent tasks will be released later.

Quickstart

Serving

vLLM (source at PR #53806, commit d9fd5f11):

vllm serve IFM/K2-Horizon-0.9B \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --hf-overrides '{"rope_parameters":{"rope_type":"yarn","factor":16.0,"original_max_position_embeddings":8192}}' \
  --gpu-memory-utilization 0.85 \
  --tensor-parallel-size 1 \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice \
  --tool-call-parser k2_horizon

SGLang, from a source checkout that includes sgl-project/sglang#37654. This is the recipe validated in the SGLang K2 Horizon cookbook:

sglang serve \
  --model-path IFM/K2-Horizon-0.9B \
  --revision 9b9ec1f7e17f62ed218df542687a144116219d84 \
  --tp 1 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --reasoning-parser k2_horizon \
  --host 0.0.0.0 \
  --port 30000

API Usage

[!Tip] Recommended settings: reasoning_effort="high", temperature=0.6, top_p=0.95, and at least 32,768 output tokens. Reasoning depth is selected per request through chat_template_kwargs. Thinking is returned in reasoning_content and the answer in content.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="IFM/K2-Horizon-0.9B",
    messages=[{"role": "user", "content": "Explain the result step by step."}],
    temperature=0.6,
    top_p=0.95,
    max_tokens=32768,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high", "tool_call_format": "xml"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Our model supports multiple tool calls formats, which can be changed with chat_template_kwargs. The supported values are json, xml, and xml_typed . The default is xml. Keep --tool-call-parser k2_horizon enabled to parse the selected format.

Transformers

Validated with Transformers 5.15.0, PyTorch 2.13.0, Safetensors 0.8.0.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "IFM/K2-Horizon-0.9B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
)

inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Best Practices

1. Reasoning effort: always high. All reported results use high reasoning effort. Pass {"chat_template_kwargs": {"reasoning_effort": "high"}} on every request; medium and low trade accuracy for speed and are not recommended for evaluation.
2. Sampling parameters. temperature=0.6, top_p=0.95.
3. Output length. Allow at least 32,768 output tokens so reasoning is never cut off. Truncated reasoning is a failed response, not a shorter one.
4. Serving. Use the validated SGLang recipe above: BF16, TP=1, FlashAttention-3. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in the SGLang cookbook.
5. Parsers. Enable the k2_horizon reasoning parser for chat, and add the k2_horizon tool-call parser for agent use. Leave both off for plain completion-style generation.
6. Revisions. main is the MOPD release checkpoint; mid1_75k and mid2_47k preserve the context-extension stages.

Citation

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}