🧰 1. Files & comparison

Calibrated imatrix-quants, each with the MTP head bundled at Q8_0. Plain Q2_K (no imatrix) is the no-calibration anchor. FP16 reference: 50.90 GiB (not included; fetch from Jackrong/Qwopus3.6-27B-Coder).

SWE-rebench Results

The agentic coding capabilities of each quant were evaluated on 10 real-world coding issues from the nebius/SWE-rebench using the OpenAI Agents SDK pointed at a local llama-server. For each nebius/SWE-rebench issue, the agent gets the problem statement and a live bash tool that shells into a dedicated Docker container with the repo pre-checked out at the failing commit. It iterates by reading files, running tests, editing code until it produces a git diff or hits the step limit. The patch is then graded by actually running the repo's FAIL_TO_PASS test suite inside the container, so pass/fail is real execution, not fuzzy matching. We tried using mini SWE-Agent but it wasn't adequately resolving issues despite have a similar patch rate.

Sampling Parameters: temperature=0.25, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0, max_tokens=32768, ctx=131072, thinking=true, mtp=true, mtp_draft_n_max=2. Tested on 4060Ti (16Gb)

Definitions:

• patched - how many of the 10 issues did the agent produce a patch for (even if it didn't resolve)?
• resolved - how many of the 10 issues had patches that passed all FAIL_TO_PASS tests?
• max_turns - how many of the 10 issues hit the 100-step cap without resolving?
• mean_steps - average number of agentic steps taken (shelling into Docker, reading files,editing code counts as steps)
• mean_tokens - average number of tokens generated across the entire agentic episode
• tool_err_rate - how often the agent produced an invalid shell command that couldn't be executed (syntax errors, wrong file paths, etc.)
• mean_wall - average wall-clock time per episode (capped at 2 hours for those that hit the step limit)

Overall, the IQ2_M quant achieves a strong 63% pass rate on this agentic coding benchmark, which is impressive for a 2-bit model. The high patch rate across all quants suggests that even the weaker ones can still generate plausible patches, but the lower pass rates and higher max turn rates indicate that many of those patches aren't actually resolving the issues. The IQ2_M quant behaves as good as the Q5_K_M albiet with ~20% more steps and tokens, however those additional steps and iterations look to be effective ones that are helping it self-correct and resolve more issues, rather than just looping. When the quant has a high number of mean tokens in combination with a high max turn rate that usually indicates the agent is stuck in a loop. It's worth pointing out that Q5KM never hits its max turn (100) when solving these issues. We recommend running these quants with a repetition penalty of >1 to break it out of loops. Given the variation induced from sampling, we run a few repetitions of each quant and report the mean ± standard deviation across those runs.

🔬 2. How they were made

🚀 3. Usage

Quick start with Ollama

Each quant is exposed as a tag (the filename's quant suffix):

ollama run hf.co/pearsonkyle/Qwopus3.6-27B-Coder-2bit-MTP-GGUF:IQ2_M
# also: :IQ4_XS  ·  :IQ3_M  ·  :Q2_K_S  ·  :IQ2_XS  ·  :Q2_K

Building llama.cpp from source (GPU)

apt-get update && apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON   # -DGGML_CUDA=OFF for CPU/Metal
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-server
cp llama.cpp/build/bin/llama-* llama.cpp/

MTP needs a recent llama.cpp — --spec-type draft-mtp support was merged in 2026-06. Build from current master.

Running the server with MTP speculative decoding

 ./llama-server \
    --model Qwopus3.6-27B-Coder-IQ2_M.gguf \
    --ctx-size 16384 \
    --n-gpu-layers 999 \
    --spec-type draft-mtp \
    --spec-draft-n-max 1 \
    --flash-attn on \
    --cache-type-k q8_0 --cache-type-v q8_0 \
    --host 0.0.0.0 --port 1234

Drop --spec-type draft-mtp --spec-draft-n-max 1 to run without MTP.

Querying via the OpenAI-compatible API

import json, urllib.request

def ask(content, max_tokens=256):
    body = {
        "messages": [{"role": "user", "content": content}],
        "max_tokens": max_tokens,
        # Coder variant emits <think> reasoning. Set enable_thinking False
        # (or raise max_tokens) so the answer lands in "content".
        "chat_template_kwargs": {"enable_thinking": False},
    }
    req = urllib.request.Request("http://127.0.0.1:1234/v1/chat/completions",
                                 json.dumps(body).encode(),
                                 {"Content-Type": "application/json"})
    return json.loads(urllib.request.urlopen(req).read())["choices"][0]["message"]["content"]

print(ask("Write a Python function that reverses a linked list."))

🪪 4. License & attribution

• Inherits its license from the base model Jackrong/Qwopus3.6-27B-Coder. Confirm the exact terms and update the frontmatter license: before publishing.
• Base weights: Jackrong/Qwopus3.6-27B-Coder (full finetune of Qwen3.6-27B, ships its own MTP head).
• Calibration + quantization performed locally with Quant-Tuner; vendored llama.cpp at commit 32782998 (2-bit rows: Q2_K, IQ2_XS, IQ2_M, Q2_K_S) and f3e1828 (IQ3_M, IQ4_XS). All six share the same hybrid imatrix + calibration corpora.
• Calibration data (usage logs) scraped using LogMiner.