--- title: Which Routing Does Each MoE Model Use? DeepSeek, Qwen,... description: Ten open Mixture-of-Experts models disagree on nearly every routing detail despite sharing the same basic idea. Expert counts range from 8 in Mixtral to 384 in... canonical: https://daily.dev/posts/which-routing-does-each-moe-model-use-deepseek-qwen-mixtral-llama-4-gpt-oss-compared-kp4pr047p twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared | daily.dev og:description: Ten open Mixture-of-Experts models disagree on nearly every routing detail despite sharing the same basic idea. Expert counts range from 8 in Mixtral to 384 in... og:url: https://daily.dev/posts/which-routing-does-each-moe-model-use-deepseek-qwen-mixtral-llama-4-gpt-oss-compared-kp4pr047p og:image: https://api.daily.dev/og/posts/Kp4Pr047p.png og:image:alt: Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared **[Machine Learning News](https://daily.dev/sources/mlnews)** · 4 min read · 0 upvotes · 0 comments ## Summary Ten open Mixture-of-Experts models disagree on nearly every routing detail despite sharing the same basic idea. Expert counts range from 8 in Mixtral to 384 in Kimi K2 and DeepSeek-V4-Pro, and top-k ranges from 1 in Llama 4 to 8 in DeepSeek-V3, Qwen3, Kimi K2 and GLM-4.5. Seven of ten models keep one always-on shared expert; Mixtral, gpt-oss and Qwen3 drop it. DeepSeek-V4-Pro raises routed experts to 384 but lowers top-k from 8 to 6 and switches its scoring function to sqrtsoftplus. Kimi K2's report found that higher sparsity lowered loss at fixed active compute. Llama 4 Maverick and gpt-oss do not publish their gating or balancing method in their configs. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How many routed experts and what top-k does DeepSeek-V4-Pro use compared to DeepSeek-V3? DeepSeek-V4-Pro uses 384 routed experts with top-k lowered to 6, compared to DeepSeek-V3's 256 experts with top-k 8. V4-Pro also switches its scoring function from sigmoid to sqrtsoftplus, while keeping bias-based noaux_tc load balancing. Its first 3 MoE layers use fixed hash routing instead of learned routing. _Comparing MoE architectures before picking a model to serve? daily.dev surfaces routing deep dives like this one._ ### Does Llama 4 Maverick use a shared expert in its MoE layers? Yes, Llama 4 Maverick routes each token to 1 shared expert plus 1 of 128 routed experts per MoE layer, a top-1 design similar to the Switch Transformer. It alternates dense and MoE layers and keeps 17B active parameters out of 400B total. Its model card does not publish the exact gating or load-balancing method. _Developers sizing GPUs for Llama 4 can track architecture details like this via daily.dev._ ### How does gpt-oss-120b's MoE routing let it fit on a single 80GB GPU? gpt-oss-120b uses a linear router over 128 experts, selecting the top 4 per token, with no shared expert listed in its config. Its MoE weights ship in MXFP4 quantization, which lets the 117B-parameter model fit on one 80GB GPU. The config does not publish its load-balancing method. _Teams planning MoE deployments can follow quantization and routing tradeoffs on daily.dev._ ## Similar posts on daily.dev - [From Mixtral to Kimi K3: How Mixture-of-Experts Models Evolved](https://daily.dev/posts/from-mixtral-to-kimi-k3-how-mixture-of-experts-models-evolved-zxggx56pu) · freeCodeCamp · 6 upvotes · 0 comments - [The Architecture Behind Open-Source LLMs](https://daily.dev/posts/the-architecture-behind-open-source-llms-vqh5xymdw) · ByteByteGo · 2 upvotes · 0 comments - [7 MoE routing strategies in production, clearly explained](https://daily.dev/posts/7-moe-routing-strategies-in-production-clearly-explained-gwu8u0sgm) · Daily Dose of Data Science \| Avi Chawla \| Substack · 51 upvotes · 1 comments - [DeepSeek-V3 from Scratch: Mixture of Experts \(MoE\)](https://daily.dev/posts/deepseek-v3-from-scratch-mixture-of-experts-moe--dexrfaga5) · PyImageSearch · 0 upvotes · 0 comments - [MoE inference engineering, clearly explained](https://daily.dev/posts/moe-inference-engineering-clearly-explained-pkujcfe50) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#deepseek](https://daily.dev/tags/deepseek), [#qwen](https://daily.dev/tags/qwen), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts), [#llama](https://daily.dev/tags/llama) [View this post on daily.dev](https://daily.dev/posts/which-routing-does-each-moe-model-use-deepseek-qwen-mixtral-llama-4-gpt-oss-compared-kp4pr047p) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared","url":"https://daily.dev/posts/which-routing-does-each-moe-model-use-deepseek-qwen-mixtral-llama-4-gpt-oss-compared-kp4pr047p","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/which-routing-does-each-moe-model-use-deepseek-qwen-mixtral-llama-4-gpt-oss-compared-kp4pr047p"},"datePublished":"2026-11-08T10:32:20.606Z","dateModified":"2026-11-08T10:47:15.747Z","description":"Ten open Mixture-of-Experts models disagree on nearly every routing detail despite sharing the same basic idea. Expert counts range from 8 in Mixtral to 384 in...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/de863da113559ae315af5c073b12770a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/de863da113559ae315af5c073b12770a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Machine Learning News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Machine Learning News","logo":"https://media.daily.dev/image/upload/s--0PoEtdkd--/f_auto/v1750409401/squads/marktechpost?_a=BAMClqZW0","url":"https://daily.dev/squads/mlnews"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/which-routing-does-each-moe-model-use-deepseek-qwen-mixtral-llama-4-gpt-oss-compared-kp4pr047p","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,deepseek,qwen,mixture-of-experts,llama","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Machine Learning News","item":"https://daily.dev/squads/mlnews"},{"@type":"ListItem","position":3,"name":"Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss Compared"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/which-routing-does-each-moe-model-use-deepseek-qwen-mixtral-llama-4-gpt-oss-compared-kp4pr047p#faq","mainEntity":[{"@type":"Question","name":"How many routed experts and what top-k does DeepSeek-V4-Pro use compared to DeepSeek-V3?","acceptedAnswer":{"@type":"Answer","text":"DeepSeek-V4-Pro uses 384 routed experts with top-k lowered to 6, compared to DeepSeek-V3's 256 experts with top-k 8. V4-Pro also switches its scoring function from sigmoid to sqrtsoftplus, while keeping bias-based noaux_tc load balancing. Its first 3 MoE layers use fixed hash routing instead of learned routing. Comparing MoE architectures before picking a model to serve? daily.dev surfaces routing deep dives like this one."}},{"@type":"Question","name":"Does Llama 4 Maverick use a shared expert in its MoE layers?","acceptedAnswer":{"@type":"Answer","text":"Yes, Llama 4 Maverick routes each token to 1 shared expert plus 1 of 128 routed experts per MoE layer, a top-1 design similar to the Switch Transformer. It alternates dense and MoE layers and keeps 17B active parameters out of 400B total. Its model card does not publish the exact gating or load-balancing method. Developers sizing GPUs for Llama 4 can track architecture details like this via daily.dev."}},{"@type":"Question","name":"How does gpt-oss-120b's MoE routing let it fit on a single 80GB GPU?","acceptedAnswer":{"@type":"Answer","text":"gpt-oss-120b uses a linear router over 128 experts, selecting the top 4 per token, with no shared expert listed in its config. Its MoE weights ship in MXFP4 quantization, which lets the 117B-parameter model fit on one 80GB GPU. The config does not publish its load-balancing method. Teams planning MoE deployments can follow quantization and routing tradeoffs on daily.dev."}}]} ```