--- title: Create Mixtures of Experts with MergeKit | daily.dev description: This post introduces the Mixture of Experts (MoE) architecture and explains how frankenMoEs can be created using the MergeKit library. It explores the benefits... canonical: https://daily.dev/posts/create-mixtures-of-experts-with-mergekit-qx7fhuo1o twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Create Mixtures of Experts with MergeKit | daily.dev og:description: This post introduces the Mixture of Experts (MoE) architecture and explains how frankenMoEs can be created using the MergeKit library. It explores the benefits... og:url: https://daily.dev/posts/create-mixtures-of-experts-with-mergekit-qx7fhuo1o og:image: https://api.daily.dev/og/posts/Qx7fHuO1O.png og:image:alt: Create Mixtures of Experts with MergeKit og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Create Mixtures of Experts with MergeKit **[Towards Data Science](https://daily.dev/sources/tds)** · 7 min read · 0 upvotes · 0 comments ## Summary This post introduces the Mixture of Experts (MoE) architecture and explains how frankenMoEs can be created using the MergeKit library. It explores the benefits and challenges of MoEs and provides a step-by-step guide for creating a frankenMoE. The post also highlights the performance of a specific frankenMoE model called Beyonder-4x7B-v3. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts) [View this post on daily.dev](https://daily.dev/posts/create-mixtures-of-experts-with-mergekit-qx7fhuo1o) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Create Mixtures of Experts with MergeKit","url":"https://daily.dev/posts/create-mixtures-of-experts-with-mergekit-qx7fhuo1o","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/create-mixtures-of-experts-with-mergekit-qx7fhuo1o"},"datePublished":"2024-03-27T14:17:56.835Z","dateModified":"2026-04-02T02:13:35.412Z","description":"This post introduces the Mixture of Experts (MoE) architecture and explains how frankenMoEs can be created using the MergeKit library. It explores the benefits...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/0166de48623b910ec496b3a7572403ea?_a=AQAEufR","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/0166de48623b910ec496b3a7572403ea?_a=AQAEufR","isAccessibleForFree":true,"articleSection":"Towards Data Science","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Towards Data Science","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/tds","url":"https://daily.dev/sources/tds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/create-mixtures-of-experts-with-mergekit-qx7fhuo1o","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,mixture-of-experts","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Towards Data Science","item":"https://daily.dev/sources/tds"},{"@type":"ListItem","position":3,"name":"Create Mixtures of Experts with MergeKit"}]} ```