GLM-4.7-Flash-Claude-Opus-4.5-High-Reasoning-Distill EOQ Q5 (Compressed)

EOQ (Entropy-Optimal Quantization) Q5 compressed version of TeichAI/GLM-4.7-Flash-Claude-Opus-4.5-High-Reasoning-Distill.

GLM-4.7-Flash is a 30B parameter MoE model (Glm4MoeLite architecture / DeepSeek2), distilled from Claude Opus 4.5 reasoning traces. 262K native context.

Verified Benchmark Results

All benchmarks on NVIDIA RTX PRO 6000 Blackwell (96 GB VRAM), verified on Google Colab Pro G4.

• Bits: 5
• Block size: 128
• Architecture: Glm4MoeLite (MoE, DeepSeek2-based)
• Note: High base PPL is expected -- this model is optimized for chat/reasoning with a specific template, not raw text completion. WikiText-2 is not the ideal benchmark for this model type.

Usage

Method 1: Using eoq_loader.py (recommended)

from huggingface_hub import snapshot_download
import sys
local = snapshot_download("caiovicentino1/GLM-4.7-Flash-Claude-Opus-4.5-High-Reasoning-Distill-EOQ-Q5-compressed")
sys.path.insert(0, local)
from eoq_loader import load_eoq_model
model, tokenizer = load_eoq_model("caiovicentino1/GLM-4.7-Flash-Claude-Opus-4.5-High-Reasoning-Distill-EOQ-Q5-compressed")

inputs = tokenizer("Hello!", return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Method 2: Manual loading

import torch, json, torch.nn.functional as F
from safetensors.torch import load_file
from huggingface_hub import snapshot_download

local = snapshot_download("caiovicentino1/GLM-4.7-Flash-Claude-Opus-4.5-High-Reasoning-Distill-EOQ-Q5-compressed")

# Load metadata and compressed weights, then dequantize
# See eoq_loader.py for full decompression logic

Links

• GitHub: https://github.com/caiovicentino/eoq-quantization
• Base model: TeichAI/GLM-4.7-Flash-Claude-Opus-4.5-High-Reasoning-Distill
• GGUF version: TeichAI/GLM-4.7-Flash-Claude-Opus-4.5-High-Reasoning-Distill-GGUF