<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/do-qwen-3-6-27b-quantizations-break-the-pelican--gnbqr3e7n" -->

---
title: Do Qwen 3.6 27B quantizations break the pelican? | daily.dev
description: A hands-on evaluation of Qwen 3.6 27B quantizations (from 8-bit down to 2-bit) using statistical measures (KL divergence, top-1 accuracy, collision...
canonical: https://daily.dev/posts/do-qwen-3-6-27b-quantizations-break-the-pelican--gnbqr3e7n
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Do Qwen 3.6 27B quantizations break the pelican? | daily.dev
og:description: A hands-on evaluation of Qwen 3.6 27B quantizations (from 8-bit down to 2-bit) using statistical measures (KL divergence, top-1 accuracy, collision...
og:url: https://daily.dev/posts/do-qwen-3-6-27b-quantizations-break-the-pelican--gnbqr3e7n
og:image: https://api.daily.dev/og/posts/gnBqr3e7n.png
og:image:alt: Do Qwen 3.6 27B quantizations break the pelican?
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Do Qwen 3.6 27B quantizations break the pelican?

**[Quesma](https://daily.dev/sources/quesma)** · 8 min read · 0 upvotes · 0 comments

## Summary

A hands-on evaluation of Qwen 3.6 27B quantizations (from 8-bit down to 2-bit) using statistical measures (KL divergence, top-1 accuracy, collision cross-entropy, perplexity), SVG generation tasks (pelican and gears), and two benchmarks (Terminal-Bench 2.1 and AIME-120). The key finding: 4-bit quantizations (Q4_K_M, UD-Q4_K_XL) are essentially indistinguishable from the full BF16 model in quality, while 3-bit results vary significantly by variant and 2-bit models show only mild degradation. Quantization reduces model size ~3x with little quality loss but does not meaningfully improve inference speed on Apple Silicon. The author spent 37 hours of local compute and $1,430 in cloud GPU costs to produce these results.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://quesma.com/blog/qwen-quantization-quality>

## Questions this post answers

### Does quantizing Qwen 3.6 27B to 4-bit hurt output quality compared to the full BF16 model?

No, 4-bit quantizations like Q4_K_M and UD-Q4_K_XL performed indistinguishably from the full BF16 model across statistical measures, SVG generation, and benchmarks like AIME-120 and Terminal-Bench 2.1. These formats are about 3x smaller than the full model, making them the recommended choice when downloading a quantized version for practical use.

_Anyone picking a quantized model size for local inference can track findings like this on daily.dev._

### Why do some 3-bit quantized LLMs perform much worse than others of the same bit-width?

Bit-width alone does not predict quality reliably; Q3_K_M scored 73.3% on AIME-120, matching the full model, while Q3_K_S, only 1.2 GB smaller, dropped to 54.2%. Kullback-Leibler divergence better predicted this split: variants with mean KL divergence below 0.05 stayed close to the BF16 baseline, while those above 0.08 degraded sharply.

_Developers comparing quantization formats beyond raw bit count can follow this kind of analysis on daily.dev._

### Does quantizing a large language model make inference faster on Apple Silicon or GPUs?

No, quantization mainly reduces memory footprint rather than speeding up inference, because matrix multiplications still run on 16-bit floats after unpacking the compressed weights. On an M5 Max MacBook Pro with 128GB RAM running UD-Q2_K_XL, throughput was 30 tokens per second, versus 94 tok/s on an L40S GPU and 112 tok/s on an H100.

_Engineers weighing memory savings against speed when deploying quantized models can follow benchmarks like this on daily.dev._

## Similar posts on daily.dev

- [I dropped Qwen and Gemma for a 27B model that runs smoothly on my 8GB graphics card](https://daily.dev/posts/i-dropped-qwen-and-gemma-for-a-27b-model-that-runs-smoothly-on-my-8gb-graphics-card-h5gi3rwyo) · XDA Developers · 0 upvotes · 0 comments
- [Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7](https://daily.dev/posts/qwen3-6-35b-a3b-on-my-laptop-drew-me-a-better-pelican-than-claude-opus-4-7-htpbdbpvw) · Simon Willison · 8 upvotes · 0 comments
- [Measuring Model Quantisation Quality with KL Divergence](https://daily.dev/posts/measuring-model-quantisation-quality-with-kl-divergence-chfxwppvm) · Sam McLeod · 1 upvotes · 0 comments

---

Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#llm](https://daily.dev/tags/llm), [#qwen](https://daily.dev/tags/qwen), [#llama-cpp](https://daily.dev/tags/llama-cpp)

[View this post on daily.dev](https://daily.dev/posts/do-qwen-3-6-27b-quantizations-break-the-pelican--gnbqr3e7n)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Do Qwen 3.6 27B quantizations break the pelican?","url":"https://daily.dev/posts/do-qwen-3-6-27b-quantizations-break-the-pelican--gnbqr3e7n","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/do-qwen-3-6-27b-quantizations-break-the-pelican--gnbqr3e7n"},"datePublished":"2026-07-27T11:45:17.491Z","dateModified":"2026-09-14T08:54:40.402Z","description":"A hands-on evaluation of Qwen 3.6 27B quantizations (from 8-bit down to 2-bit) using statistical measures (KL divergence, top-1 accuracy, collision...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/cf9df3cd40129ef58d8903f941c36f3d?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/cf9df3cd40129ef58d8903f941c36f3d?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Quesma","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Quesma","logo":"https://media.daily.dev/image/upload/s--I-Be0YJY--/f_auto,q_auto/v1774964372/logos/quesma?_a=BAMAMiWQ0","url":"https://daily.dev/sources/quesma"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/do-qwen-3-6-27b-quantizations-break-the-pelican--gnbqr3e7n","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,llm,qwen,llama-cpp","timeRequired":"PT8M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Quesma","item":"https://daily.dev/sources/quesma"},{"@type":"ListItem","position":3,"name":"Do Qwen 3.6 27B quantizations break the pelican?"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/do-qwen-3-6-27b-quantizations-break-the-pelican--gnbqr3e7n#faq","mainEntity":[{"@type":"Question","name":"Does quantizing Qwen 3.6 27B to 4-bit hurt output quality compared to the full BF16 model?","acceptedAnswer":{"@type":"Answer","text":"No, 4-bit quantizations like Q4_K_M and UD-Q4_K_XL performed indistinguishably from the full BF16 model across statistical measures, SVG generation, and benchmarks like AIME-120 and Terminal-Bench 2.1. These formats are about 3x smaller than the full model, making them the recommended choice when downloading a quantized version for practical use. Anyone picking a quantized model size for local inference can track findings like this on daily.dev."}},{"@type":"Question","name":"Why do some 3-bit quantized LLMs perform much worse than others of the same bit-width?","acceptedAnswer":{"@type":"Answer","text":"Bit-width alone does not predict quality reliably; Q3_K_M scored 73.3% on AIME-120, matching the full model, while Q3_K_S, only 1.2 GB smaller, dropped to 54.2%. Kullback-Leibler divergence better predicted this split: variants with mean KL divergence below 0.05 stayed close to the BF16 baseline, while those above 0.08 degraded sharply. Developers comparing quantization formats beyond raw bit count can follow this kind of analysis on daily.dev."}},{"@type":"Question","name":"Does quantizing a large language model make inference faster on Apple Silicon or GPUs?","acceptedAnswer":{"@type":"Answer","text":"No, quantization mainly reduces memory footprint rather than speeding up inference, because matrix multiplications still run on 16-bit floats after unpacking the compressed weights. On an M5 Max MacBook Pro with 128GB RAM running UD-Q2_K_XL, throughput was 30 tokens per second, versus 94 tok/s on an L40S GPU and 112 tok/s on an H100. Engineers weighing memory savings against speed when deploying quantized models can follow benchmarks like this on daily.dev."}}]}
```