---
title: "Create Mixtures of Experts with MergeKit"
url: https://daily.dev/posts/create-mixtures-of-experts-with-mergekit-qx7fhuo1o
source_url: https://towardsdatascience.com/create-mixtures-of-experts-with-mergekit-11b318c99562
type: article
source: "Towards Data Science"
published: 2024-03-27T14:17:56.835Z
updated: 2026-04-02T02:13:35.412Z
tags: ["machine-learning", "mixture-of-experts"]
reading_time: 7
upvotes: 0
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Create Mixtures of Experts with MergeKit

**[Towards Data Science](https://daily.dev/sources/tds)** · 7 min read · 0 upvotes · 0 comments

## Summary

This post introduces the Mixture of Experts (MoE) architecture and explains how frankenMoEs can be created using the MergeKit library. It explores the benefits and challenges of MoEs and provides a step-by-step guide for creating a frankenMoE. The post also highlights the performance of a specific frankenMoE model called Beyonder-4x7B-v3.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://towardsdatascience.com/create-mixtures-of-experts-with-mergekit-11b318c99562>

## Similar posts on daily.dev

- [Don’t just attend KubeCon \+ CloudNativeCon, Merge Forward your experience\!](https://daily.dev/posts/don-t-just-attend-kubecon-cloudnativecon-merge-forward-your-experience--l0rpp73x8) · CNCF · 0 upvotes · 0 comments
- [Announcing H2 2026 KCDs](https://daily.dev/posts/announcing-h2-2026-kcds-m96goajm1) · CNCF · 1 upvotes · 0 comments
- [Two months of Open Community Groups](https://daily.dev/posts/two-months-of-open-community-groups-asf52zhbs) · CNCF · 0 upvotes · 0 comments
- [CNCF Unveils Schedule for KubeCon \+ CloudNativeCon Europe 2026](https://daily.dev/posts/cncf-unveils-schedule-for-kubecon-cloudnativecon-europe-2026-ikhcoa5cb) · CNCF · 2 upvotes · 0 comments
- [CNCF Debuts KubeCon \+ CloudNativeCon Japan 2026 Schedule](https://daily.dev/posts/cncf-debuts-kubecon-cloudnativecon-japan-2026-schedule-xp5pyudub) · CNCF · 1 upvotes · 0 comments

---

Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts)

[View this post on daily.dev](https://daily.dev/posts/create-mixtures-of-experts-with-mergekit-qx7fhuo1o)