A hands-on evaluation of Qwen 3.6 27B quantizations (from 8-bit down to 2-bit) using statistical measures (KL divergence, top-1 accuracy, collision cross-entropy, perplexity), SVG generation tasks (pelican and gears), and two benchmarks (Terminal-Bench 2.1 and AIME-120). The key finding: 4-bit quantizations (Q4_K_M, UD-Q4_K_XL) are essentially indistinguishable from the full BF16 model in quality, while 3-bit results vary significantly by variant and 2-bit models show only mild degradation. Quantization reduces model size ~3x with little quality loss but does not meaningfully improve inference speed on Apple Silicon. The author spent 37 hours of local compute and $1,430 in cloud GPU costs to produce these results.

8m read timeFrom quesma.com
Post cover image
Table of contents
QuantizationsStatistical measuresLooking at SVGsBenchmarksAnd how about speedConclusion
398 Impressions