<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk" -->

---
title: Quantization hurts knowledge nonlinearly | daily.dev
description: Quantization of LLMs compresses model weights lossily, and factual knowledge turns out to be the first casualty. Testing 55 quantizations of Qwen3.6 27B using...
canonical: https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Quantization hurts knowledge nonlinearly | daily.dev
og:description: Quantization of LLMs compresses model weights lossily, and factual knowledge turns out to be the first casualty. Testing 55 quantizations of Qwen3.6 27B using...
og:url: https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk
og:image: https://api.daily.dev/og/posts/1k1ALe5qK.png
og:image:alt: Quantization hurts knowledge nonlinearly
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Quantization hurts knowledge nonlinearly

**[Quesma](https://daily.dev/sources/quesma)** · 5 min read · 1 upvotes · 1 comments

## Summary

Quantization of LLMs compresses model weights lossily, and factual knowledge turns out to be the first casualty. Testing 55 quantizations of Qwen3.6 27B using GGUF formats from Unsloth, Bartowski, and llama.cpp against the Incompressible Knowledge Probes (IKP) benchmark reveals a nonlinear degradation pattern. Models quantized to 5-bit or above (>20 GB) retain near-original quality, while 3-bit and 2-bit variants show steep drops — especially for obscure trivia tiers. Crucially, the degradation correlates linearly with KL divergence from the original BF16 model, not with model size alone. The key insight: quantization is blind compression that doesn't optimize for training data, unlike training a smaller model from scratch. Practical takeaways: Q8_0 is a safe bet, Q4_K_M is generally acceptable, and for knowledge-intensive tasks (like navigating a large Python package ecosystem), model size and quantization level matter significantly.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://quesma.com/blog/quantization-hurts-knowledge>

## Community discussion

Top comments from developers on daily.dev.

**@agustinbarrientos** · 0 upvotes

> Have you tried the same KL-divergence check on code completion, where losing package trivia could show up before the usual benchmarks move?

## Similar posts on daily.dev

- [Quantization's Real Tradeoff: Where FP16, INT8, and GGUF Actually Diverge in Production by Model Size](https://daily.dev/posts/quantization-s-real-tradeoff-where-fp16-int8-and-gguf-actually-diverge-in-production-by-model-siz-gorxyymj9) · DigitalOcean Community · 0 upvotes · 0 comments
- [Measuring Model Quantisation Quality with KL Divergence](https://daily.dev/posts/measuring-model-quantisation-quality-with-kl-divergence-chfxwppvm) · Sam McLeod · 1 upvotes · 0 comments
- [Q4 vs Q6 vs Q8: The Quantization Decision Framework for Local LLMs](https://daily.dev/posts/q4-vs-q6-vs-q8-the-quantization-decision-framework-for-local-llms-a0umzfovm) · SitePoint · 0 upvotes · 0 comments

---

Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#data-science](https://daily.dev/tags/data-science), [#llm](https://daily.dev/tags/llm), [#llama-cpp](https://daily.dev/tags/llama-cpp)

[View this post on daily.dev](https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Quantization hurts knowledge nonlinearly","url":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk"},"datePublished":"2026-08-03T14:35:30.590Z","dateModified":"2026-08-03T18:18:18.444Z","description":"Quantization of LLMs compresses model weights lossily, and factual knowledge turns out to be the first casualty. Testing 55 quantizations of Qwen3.6 27B using...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/af054a3e648b611301247796155be96a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/af054a3e648b611301247796155be96a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Quesma","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Quesma","logo":"https://media.daily.dev/image/upload/s--I-Be0YJY--/f_auto,q_auto/v1774964372/logos/quesma?_a=BAMAMiWQ0","url":"https://daily.dev/sources/quesma"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"machine-learning,data-science,llm,llama-cpp","timeRequired":"PT5M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Quesma","item":"https://daily.dev/sources/quesma"},{"@type":"ListItem","position":3,"name":"Quantization hurts knowledge nonlinearly"}]}
{"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk","comment":[{"@type":"Comment","text":"Have you tried the same KL-divergence check on code completion, where losing package trivia could show up before the usual benchmarks move?","datePublished":"2026-08-03T17:26:42.215Z","url":"https://daily.dev/posts/1k1ALe5qK#c-2MgQCzl0G","author":{"@type":"Person","name":"Agustin Barrientos","url":"https://daily.dev/agustinbarrientos","image":"https://media.daily.dev/image/upload/s--5ayxQnqn--/f_auto/v1788281802/avatars/avatar_wQYYVe5Tbj0NJ7C7qPoa8?_a=BAMAMicg0"}}]}
```