--- title: Quantization hurts knowledge nonlinearly | daily.dev description: Quantization of LLMs compresses model weights lossily, and factual knowledge turns out to be the first casualty. Testing 55 quantizations of Qwen3.6 27B using... canonical: https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Quantization hurts knowledge nonlinearly | daily.dev og:description: Quantization of LLMs compresses model weights lossily, and factual knowledge turns out to be the first casualty. Testing 55 quantizations of Qwen3.6 27B using... og:url: https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk og:image: https://api.daily.dev/og/posts/1k1ALe5qK.png og:image:alt: Quantization hurts knowledge nonlinearly og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Quantization hurts knowledge nonlinearly **[Quesma](https://daily.dev/sources/quesma)** · 5 min read · 1 upvotes · 1 comments ## Summary Quantization of LLMs compresses model weights lossily, and factual knowledge turns out to be the first casualty. Testing 55 quantizations of Qwen3.6 27B using GGUF formats from Unsloth, Bartowski, and llama.cpp against the Incompressible Knowledge Probes (IKP) benchmark reveals a nonlinear degradation pattern. Models quantized to 5-bit or above (>20 GB) retain near-original quality, while 3-bit and 2-bit variants show steep drops — especially for obscure trivia tiers. Crucially, the degradation correlates linearly with KL divergence from the original BF16 model, not with model size alone. The key insight: quantization is blind compression that doesn't optimize for training data, unlike training a smaller model from scratch. Practical takeaways: Q8_0 is a safe bet, Q4_K_M is generally acceptable, and for knowledge-intensive tasks (like navigating a large Python package ecosystem), model size and quantization level matter significantly. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much does GGUF quantization affect factual knowledge in Qwen3.6 27B? Quantizations above 20 GB (5-bit or more) show the same factual knowledge accuracy as the original BF16 model, measured on the Incompressible Knowledge Probes benchmark. Below that, quality drops: a noticeable decline appears at 3-bit, and it becomes much steeper at 2-bit. The degradation is nonlinear, unlike the near-linear relationship between model size and knowledge across different model scales. _daily.dev surfaces benchmarks like this for developers deciding which quantization level to run locally._ ### What quantization level should I use for Qwen3.6 27B to avoid losing factual knowledge? An 8-bit quantization such as Q8_0 from Unsloth is a safe bet for retaining factual knowledge, and a good 4-bit quant like Q4_K_M from Bartowski is also likely fine. Below that, especially at 2-bit, knowledge degrades sharply, though tasks like code generation or SVG drawing show much smaller quality loss at the same bit depths. _Developers picking model sizes for local inference can track findings like these on daily.dev before committing to a quantization._ ### Why does quantization hurt factual knowledge more than other capabilities in large language models? Quantization is a blind, lossy compression that optimizes for closeness to the original probability distribution rather than for any particular task or training data, unlike training a smaller model from scratch which still optimizes for the data. Kullback-Leibler divergence from the original BF16 model correlates linearly with IKP benchmark accuracy loss, showing early compression removes noise for free before the distribution starts breaking down. _daily.dev helps developers reasoning about model compression trade-offs keep up with findings like this KL-divergence explanation._ ## Community discussion Top comments from developers on daily.dev. **@agustinbarrientos** · 0 upvotes > Have you tried the same KL-divergence check on code completion, where losing package trivia could show up before the usual benchmarks move? ## Similar posts on daily.dev - [Quantization's Real Tradeoff: Where FP16, INT8, and GGUF Actually Diverge in Production by Model Size](https://daily.dev/posts/quantization-s-real-tradeoff-where-fp16-int8-and-gguf-actually-diverge-in-production-by-model-siz-gorxyymj9) · DigitalOcean Community · 0 upvotes · 0 comments - [Measuring Model Quantisation Quality with KL Divergence](https://daily.dev/posts/measuring-model-quantisation-quality-with-kl-divergence-chfxwppvm) · Sam McLeod · 1 upvotes · 0 comments - [Q4 vs Q6 vs Q8: The Quantization Decision Framework for Local LLMs](https://daily.dev/posts/q4-vs-q6-vs-q8-the-quantization-decision-framework-for-local-llms-a0umzfovm) · SitePoint · 0 upvotes · 0 comments - [Quantization from the ground up](https://daily.dev/posts/quantization-from-the-ground-up-h60w1pagu) · Hacker News · 3 upvotes · 1 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#data-science](https://daily.dev/tags/data-science), [#llm](https://daily.dev/tags/llm), [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Quantization hurts knowledge nonlinearly","url":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk"},"datePublished":"2026-08-03T14:35:30.590Z","dateModified":"2026-09-14T06:28:47.315Z","description":"Quantization of LLMs compresses model weights lossily, and factual knowledge turns out to be the first casualty. Testing 55 quantizations of Qwen3.6 27B using...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/af054a3e648b611301247796155be96a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/af054a3e648b611301247796155be96a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Quesma","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Quesma","logo":"https://media.daily.dev/image/upload/s--I-Be0YJY--/f_auto,q_auto/v1774964372/logos/quesma?_a=BAMAMiWQ0","url":"https://daily.dev/sources/quesma"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"machine-learning,data-science,llm,llama-cpp","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Quesma","item":"https://daily.dev/sources/quesma"},{"@type":"ListItem","position":3,"name":"Quantization hurts knowledge nonlinearly"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk","comment":[{"@type":"Comment","text":"Have you tried the same KL-divergence check on code completion, where losing package trivia could show up before the usual benchmarks move?","datePublished":"2026-08-03T17:26:42.215Z","url":"https://daily.dev/posts/1k1ALe5qK#c-2MgQCzl0G","author":{"@type":"Person","name":"Agustin Barrientos","url":"https://daily.dev/agustinbarrientos","image":"https://media.daily.dev/image/upload/s--5ayxQnqn--/f_auto/v1788281802/avatars/avatar_wQYYVe5Tbj0NJ7C7qPoa8?_a=BAMAMicg0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/quantization-hurts-knowledge-nonlinearly-1k1ale5qk#faq","mainEntity":[{"@type":"Question","name":"How much does GGUF quantization affect factual knowledge in Qwen3.6 27B?","acceptedAnswer":{"@type":"Answer","text":"Quantizations above 20 GB (5-bit or more) show the same factual knowledge accuracy as the original BF16 model, measured on the Incompressible Knowledge Probes benchmark. Below that, quality drops: a noticeable decline appears at 3-bit, and it becomes much steeper at 2-bit. The degradation is nonlinear, unlike the near-linear relationship between model size and knowledge across different model scales. daily.dev surfaces benchmarks like this for developers deciding which quantization level to run locally."}},{"@type":"Question","name":"What quantization level should I use for Qwen3.6 27B to avoid losing factual knowledge?","acceptedAnswer":{"@type":"Answer","text":"An 8-bit quantization such as Q8_0 from Unsloth is a safe bet for retaining factual knowledge, and a good 4-bit quant like Q4_K_M from Bartowski is also likely fine. Below that, especially at 2-bit, knowledge degrades sharply, though tasks like code generation or SVG drawing show much smaller quality loss at the same bit depths. Developers picking model sizes for local inference can track findings like these on daily.dev before committing to a quantization."}},{"@type":"Question","name":"Why does quantization hurt factual knowledge more than other capabilities in large language models?","acceptedAnswer":{"@type":"Answer","text":"Quantization is a blind, lossy compression that optimizes for closeness to the original probability distribution rather than for any particular task or training data, unlike training a smaller model from scratch which still optimizes for the data. Kullback-Leibler divergence from the original BF16 model correlates linearly with IKP benchmark accuracy loss, showing early compression removes noise for free before the distribution starts breaking down. daily.dev helps developers reasoning about model compression trade-offs keep up with findings like this KL-divergence explanation."}}]} ```