--- title: Mistral 8x7B 32k model stats | daily.dev description: The Mistral 8x7B 32k model is a Mixture of Experts (MoE) model with 995 tensors, including token embedding, output norm, and output tensors. The model has 32... canonical: https://daily.dev/posts/mistral-8x7b-32k-model-stats-qxstnduo9 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Mistral 8x7B 32k model stats | daily.dev og:description: The Mistral 8x7B 32k model is a Mixture of Experts (MoE) model with 995 tensors, including token embedding, output norm, and output tensors. The model has 32... og:url: https://daily.dev/posts/mistral-8x7b-32k-model-stats-qxstnduo9 og:image: https://api.daily.dev/og/posts/qxStndUo9.png og:image:alt: Mistral 8x7B 32k model stats og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Mistral 8x7B 32k model stats **[GoPenAI](https://daily.dev/sources/gopenai)** · 3 min read · 1 upvotes · 0 comments ## Summary The Mistral 8x7B 32k model is a Mixture of Experts (MoE) model with 995 tensors, including token embedding, output norm, and output tensors. The model has 32 blocks of attention and ffn. During inference, two experts are used per token, resulting in a faster speed as if using a 12B model. The model has 47B parameters because the FFN layers are treated as individual experts. The model can be run on CPU if there is not enough VRAM on the GPU. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: --- Tags: [#llama](https://daily.dev/tags/llama), [#llama-cpp](https://daily.dev/tags/llama-cpp), [#llm](https://daily.dev/tags/llm), [#local-ai](https://daily.dev/tags/local-ai), [#machine-learning](https://daily.dev/tags/machine-learning), [#mistral-ai](https://daily.dev/tags/mistral-ai), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts) [View this post on daily.dev](https://daily.dev/posts/mistral-8x7b-32k-model-stats-qxstnduo9) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Mistral 8x7B 32k model stats","url":"https://daily.dev/posts/mistral-8x7b-32k-model-stats-qxstnduo9","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/mistral-8x7b-32k-model-stats-qxstnduo9"},"datePublished":"2023-12-14T12:56:54.342Z","dateModified":"2026-05-14T02:14:39.311Z","description":"The Mistral 8x7B 32k model is a Mixture of Experts (MoE) model with 995 tensors, including token embedding, output norm, and output tensors. The model has 32...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/631e5ec78319964ae62ef040d5b92d21?_a=AQAEufR","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/631e5ec78319964ae62ef040d5b92d21?_a=AQAEufR","isAccessibleForFree":true,"articleSection":"GoPenAI","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"GoPenAI","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/f34dfd0c312c4a59b897eb64ad28d895","url":"https://daily.dev/sources/gopenai"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/mistral-8x7b-32k-model-stats-qxstnduo9","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llama,llama-cpp,llm,local-ai,machine-learning,mistral-ai,mixture-of-experts","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"GoPenAI","item":"https://daily.dev/sources/gopenai"},{"@type":"ListItem","position":3,"name":"Mistral 8x7B 32k model stats"}]} ```