--- title: MLA-Whisper across 11 languages, and what the... description: An extension of the Whisper-MLA paper that converts Whisper's decoder self-attention from multi-head to multi-head latent attention (DeepSeek-style KV cache... canonical: https://daily.dev/posts/mla-whisper-across-11-languages-and-what-the-compression-actually-costs-mi0ucdz07 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: MLA-Whisper across 11 languages, and what the compression actually costs | daily.dev og:description: An extension of the Whisper-MLA paper that converts Whisper's decoder self-attention from multi-head to multi-head latent attention (DeepSeek-style KV cache... og:url: https://daily.dev/posts/mla-whisper-across-11-languages-and-what-the-compression-actually-costs-mi0ucdz07 og:image: https://api.daily.dev/og/posts/mI0UcDz07.png og:image:alt: MLA-Whisper across 11 languages, and what the compression actually costs og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # MLA-Whisper across 11 languages, and what the compression actually costs **[Medium](https://daily.dev/sources/medium_js)** · 8 min read · 0 upvotes · 0 comments ## Summary An extension of the Whisper-MLA paper that converts Whisper's decoder self-attention from multi-head to multi-head latent attention (DeepSeek-style KV cache compression) across 11 languages and six scripts. The author releases all checkpoints plus matched controls, enabling reproducible measurement of the true compression cost. Key findings: the 62.5% decode self-attention cache reduction costs near-zero WER at large-v3 but ~+1.9 WER at tiny, meaning smaller models pay the highest price for compression. The saving is a memory win enabling more concurrent streams, not a latency improvement — per-step decoding is 2–18% slower. Cross-attention cache (covering ~1500 audio frames) is not compressed in the default release. Georgian is an outlier with unusable quality at base/small tiers. Models require trust_remote_code and transformers 4.46.x and do not work with whisper.cpp or faster-whisper. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Build DeepSeek-V3: Multi-Head Latent Attention \(MLA\) Architecture](https://daily.dev/posts/build-deepseek-v3-multi-head-latent-attention-mla-architecture-krgk8duit) · PyImageSearch · 1 upvotes · 0 comments - [DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0](https://daily.dev/posts/deepseek-v4-1-flash-on-vllm-5x-agentic-throughput-since-day-0-js5lna7tm) · vLLM · 0 upvotes · 0 comments - [Bytes Speak All Languages: Cross-Script Name Retrieval via Contrastive Learning](https://daily.dev/posts/bytes-speak-all-languages-cross-script-name-retrieval-via-contrastive-learning-nexqauuco) · Towards Data Science · 2 upvotes · 0 comments - [Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention](https://daily.dev/posts/recent-developments-in-llm-architectures-kv-sharing-mhc-and-compressed-attention-qexkx5v1i) · Sebastian Raschka · 55 upvotes · 1 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#machine-learning](https://daily.dev/tags/machine-learning), [#speech-recognition](https://daily.dev/tags/speech-recognition), [#whisper](https://daily.dev/tags/whisper) [View this post on daily.dev](https://daily.dev/posts/mla-whisper-across-11-languages-and-what-the-compression-actually-costs-mi0ucdz07) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"MLA-Whisper across 11 languages, and what the compression actually costs","url":"https://daily.dev/posts/mla-whisper-across-11-languages-and-what-the-compression-actually-costs-mi0ucdz07","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/mla-whisper-across-11-languages-and-what-the-compression-actually-costs-mi0ucdz07"},"datePublished":"2026-07-22T17:53:22.911Z","dateModified":"2026-10-15T02:20:25.492Z","description":"An extension of the Whisper-MLA paper that converts Whisper's decoder self-attention from multi-head to multi-head latent attention (DeepSeek-style KV cache...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d6ab0a9a223ee01bc82d647f54feb2d2?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d6ab0a9a223ee01bc82d647f54feb2d2?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/mla-whisper-across-11-languages-and-what-the-compression-actually-costs-mi0ucdz07","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,machine-learning,speech-recognition,whisper","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"MLA-Whisper across 11 languages, and what the compression actually costs"}]} ```