Chaty · Qwen3.5-4B Design (GGUF)
1
3 commits
4 linked in READMEs
updated Jun 21, 2026
README
Chaty · Qwen3.5-4B Design (GGUF)
A small, on-device model fine-tuned to give Chaty — a private, fully-local desktop chat app — stronger single-file web/HTML design on weak hardware, while self-identifying as Chaty and following Chaty's retrieval-grounding contract.
- Base model:
Qwen/Qwen3.5-4B(Apache-2.0) - Created by: Fangyuan Lin (author of Chaty)
- Format: GGUF,
Q4_K_Mquant — runs in llama.cpp, LM Studio, or Chaty - Languages: English + 简体中文
What it's for
Chaty runs entirely on the user's own device (no cloud). This model is tuned so that weak-hardware users get better-looking, leaner web pages, faster — and so Chaty's assistant behaves consistently:
- Web design — concise, self-contained single-file HTML pages (premium typography, dark/editorial styles), ~20–30% fewer tokens than the base ⇒ faster generation on weak hardware.
- Identity — reliably self-identifies as Chaty (a fine-tuned Qwen3.5-4B running locally), created by Fangyuan Lin; uses no version number.
- RAG grounding — when given numbered passages, cites with 【n】 and refuses ("not mentioned") when the answer is absent.
How it was made
LoRA fine-tune (Unsloth, bf16) of Qwen/Qwen3.5-4B,
then exported to GGUF and quantized to Q4_K_M. Training data (≈760 examples):
- Web design distilled from a local Qwen3.6-35B-A3B teacher (concise-but-complete pages, rendered & QC-filtered with a headless-Chrome / CDP probe).
- RAG grounding contracts (【n】 citations + graceful refusals).
- Identity distilled from the same 35B teacher under a Chaty persona (bilingual).
Evaluation (honest)
Held-out sets; base = Qwen3.5-4B, heuristic-scored (no LLM judge).
| Dimension | base | this model |
|---|---|---|
| Self-identifies as Chaty | 0% | 94% |
| RAG 【n】 citation compliance | 100% | 100% |
| RAG refusal when answer absent | 100% | 100% |
| Design: clean render (0 JS errors + real content) | 56% | 59% |
| Design: self-contained single file | 89% | 97% |
| Avg tokens / page (↓ = faster on weak HW) | ~6000 | ~4700 |
Trade-off: the gain is a more refined, leaner design style + consistent Chaty behavior; the cost is occasional content-fidelity slips on data-heavy interactive widgets (e.g. decimal-price formatting, complex JS logic). It is a 4B model specialized for Chaty's workflow, not a general chat assistant.
Run it
llama.cpp:
llama-cli -m chaty-qwen3.5-4b-design-Q4_K_M.gguf -p "Design a pricing page for a meditation app."
Or load the GGUF in LM Studio, or drop it into Chaty's models folder.
License
Apache-2.0, inherited from the base model Qwen/Qwen3.5-4B.
Contributors
3 commits
Similar repos
Linked in READMEs
Chaty · Qwen3.5-4B Design (GGUF)
1
3 commits
4 linked in READMEs
updated Jun 21, 2026
README
Chaty · Qwen3.5-4B Design (GGUF)
A small, on-device model fine-tuned to give Chaty — a private, fully-local desktop chat app — stronger single-file web/HTML design on weak hardware, while self-identifying as Chaty and following Chaty's retrieval-grounding contract.
- Base model:
Qwen/Qwen3.5-4B(Apache-2.0) - Created by: Fangyuan Lin (author of Chaty)
- Format: GGUF,
Q4_K_Mquant — runs in llama.cpp, LM Studio, or Chaty - Languages: English + 简体中文
What it's for
Chaty runs entirely on the user's own device (no cloud). This model is tuned so that weak-hardware users get better-looking, leaner web pages, faster — and so Chaty's assistant behaves consistently:
- Web design — concise, self-contained single-file HTML pages (premium typography, dark/editorial styles), ~20–30% fewer tokens than the base ⇒ faster generation on weak hardware.
- Identity — reliably self-identifies as Chaty (a fine-tuned Qwen3.5-4B running locally), created by Fangyuan Lin; uses no version number.
- RAG grounding — when given numbered passages, cites with 【n】 and refuses ("not mentioned") when the answer is absent.
How it was made
LoRA fine-tune (Unsloth, bf16) of Qwen/Qwen3.5-4B,
then exported to GGUF and quantized to Q4_K_M. Training data (≈760 examples):
- Web design distilled from a local Qwen3.6-35B-A3B teacher (concise-but-complete pages, rendered & QC-filtered with a headless-Chrome / CDP probe).
- RAG grounding contracts (【n】 citations + graceful refusals).
- Identity distilled from the same 35B teacher under a Chaty persona (bilingual).
Evaluation (honest)
Held-out sets; base = Qwen3.5-4B, heuristic-scored (no LLM judge).
| Dimension | base | this model |
|---|---|---|
| Self-identifies as Chaty | 0% | 94% |
| RAG 【n】 citation compliance | 100% | 100% |
| RAG refusal when answer absent | 100% | 100% |
| Design: clean render (0 JS errors + real content) | 56% | 59% |
| Design: self-contained single file | 89% | 97% |
| Avg tokens / page (↓ = faster on weak HW) | ~6000 | ~4700 |
Trade-off: the gain is a more refined, leaner design style + consistent Chaty behavior; the cost is occasional content-fidelity slips on data-heavy interactive widgets (e.g. decimal-price formatting, complex JS logic). It is a 4B model specialized for Chaty's workflow, not a general chat assistant.
Run it
llama.cpp:
llama-cli -m chaty-qwen3.5-4b-design-Q4_K_M.gguf -p "Design a pricing page for a meditation app."
Or load the GGUF in LM Studio, or drop it into Chaty's models folder.
License
Apache-2.0, inherited from the base model Qwen/Qwen3.5-4B.
Similar repos
Linked in READMEs
Contributors
3 commits