--- title: FreeToken Just Shipped. Is It Better Than Ollama and... description: A hands-on benchmark pits the newly released FreeToken MoE serving engine against Ollama and llama.cpp on a 6GB VRAM laptop running gpt-oss-20b. Ollama and... canonical: https://daily.dev/posts/freetoken-just-shipped-is-it-better-than-ollama-and-llama-cpp--dsvmi8hqy twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: FreeToken Just Shipped. Is It Better Than Ollama and llama.cpp? | daily.dev og:description: A hands-on benchmark pits the newly released FreeToken MoE serving engine against Ollama and llama.cpp on a 6GB VRAM laptop running gpt-oss-20b. Ollama and... og:url: https://daily.dev/posts/freetoken-just-shipped-is-it-better-than-ollama-and-llama-cpp--dsvmi8hqy og:image: https://api.daily.dev/og/posts/DSvmi8Hqy.png og:image:alt: FreeToken Just Shipped. Is It Better Than Ollama and llama.cpp? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # FreeToken Just Shipped. Is It Better Than Ollama and llama.cpp? **[Medium](https://daily.dev/sources/medium_js)** · 8 min read · 0 upvotes · 0 comments ## Summary A hands-on benchmark pits the newly released FreeToken MoE serving engine against Ollama and llama.cpp on a 6GB VRAM laptop running gpt-oss-20b. Ollama and llama.cpp use a fixed CPU/GPU layer split decided at load time, while FreeToken dynamically caches experts in a VRAM LRU cache and adaptively decides whether to fetch weights over PCIe or compute on CPU. On this small 20B model, that dynamic machinery adds pure overhead: Ollama answered fastest (0.38s time-to-first-token), llama.cpp close behind (0.5s), and FreeToken lagged at 1.17s with no throughput benefit. The author argues FreeToken's design targets larger models and changing agent workloads on workstation-class RAM, not a 20B model on a 6GB laptop, and that this test couldn't validate or refute its claims at scale. Practical setup notes are included: Windows Memory Integrity blocking unsigned llama.cpp DLLs, and a FreeToken cache memory_ratio setting that needed raising from 0.9 to 0.95. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Is FreeToken faster than Ollama or llama.cpp for running a 20B MoE model on a laptop with 6GB VRAM? No, FreeToken was slower in this scenario. Running gpt-oss-20b in MXFP4 on an RTX 3050 laptop with 6GB VRAM, Ollama had a time-to-first-token of 0.38 seconds, llama.cpp took 0.5 seconds, while FreeToken took 1.17 seconds due to its continuous cache-checking and bandwidth-adaptive routing overhead, which has little benefit at this model size. _Weighing local inference engines for your hardware? daily.dev surfaces benchmarks like this as new serving engines emerge._ ### How does FreeToken's approach to running MoE models differ from Ollama and llama.cpp? FreeToken keeps the full set of experts in CPU RAM as the source of truth and turns spare GPU memory into an LRU cache for recently used experts, deciding on the fly whether to pull weights over PCIe or compute them on CPU based on measured bandwidth. Ollama and llama.cpp instead split model layers once at load time between GPU and CPU RAM and leave that split fixed during generation. _Understanding these architecture trade-offs helps when choosing a local LLM serving engine on daily.dev._ ### Why did llama.cpp fail to load on Windows with a Bad Image error? Windows Memory Integrity blocked the llama.cpp DLLs because the binaries are not code-signed, producing a Bad Image error with no output in a normal terminal. Running the terminal as administrator revealed the real error, and disabling Memory Integrity in Windows Security resolved the load failure. _Developers troubleshooting local LLM tooling setups can track fixes like this on daily.dev._ --- Tags: [#gpu](https://daily.dev/tags/gpu), [#ollama](https://daily.dev/tags/ollama), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts), [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/freetoken-just-shipped-is-it-better-than-ollama-and-llama-cpp--dsvmi8hqy) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"FreeToken Just Shipped. Is It Better Than Ollama and llama.cpp?","url":"https://daily.dev/posts/freetoken-just-shipped-is-it-better-than-ollama-and-llama-cpp--dsvmi8hqy","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/freetoken-just-shipped-is-it-better-than-ollama-and-llama-cpp--dsvmi8hqy"},"datePublished":"2026-08-31T14:24:22.724Z","dateModified":"2026-08-31T14:25:27.260Z","description":"A hands-on benchmark pits the newly released FreeToken MoE serving engine against Ollama and llama.cpp on a 6GB VRAM laptop running gpt-oss-20b. Ollama and...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d8d45988bf049261f1daa0d00a0bf596?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d8d45988bf049261f1daa0d00a0bf596?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/freetoken-just-shipped-is-it-better-than-ollama-and-llama-cpp--dsvmi8hqy","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"gpu,ollama,mixture-of-experts,llama-cpp","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"FreeToken Just Shipped. Is It Better Than Ollama and llama.cpp?"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/freetoken-just-shipped-is-it-better-than-ollama-and-llama-cpp--dsvmi8hqy#faq","mainEntity":[{"@type":"Question","name":"Is FreeToken faster than Ollama or llama.cpp for running a 20B MoE model on a laptop with 6GB VRAM?","acceptedAnswer":{"@type":"Answer","text":"No, FreeToken was slower in this scenario. Running gpt-oss-20b in MXFP4 on an RTX 3050 laptop with 6GB VRAM, Ollama had a time-to-first-token of 0.38 seconds, llama.cpp took 0.5 seconds, while FreeToken took 1.17 seconds due to its continuous cache-checking and bandwidth-adaptive routing overhead, which has little benefit at this model size. Weighing local inference engines for your hardware? daily.dev surfaces benchmarks like this as new serving engines emerge."}},{"@type":"Question","name":"How does FreeToken's approach to running MoE models differ from Ollama and llama.cpp?","acceptedAnswer":{"@type":"Answer","text":"FreeToken keeps the full set of experts in CPU RAM as the source of truth and turns spare GPU memory into an LRU cache for recently used experts, deciding on the fly whether to pull weights over PCIe or compute them on CPU based on measured bandwidth. Ollama and llama.cpp instead split model layers once at load time between GPU and CPU RAM and leave that split fixed during generation. Understanding these architecture trade-offs helps when choosing a local LLM serving engine on daily.dev."}},{"@type":"Question","name":"Why did llama.cpp fail to load on Windows with a Bad Image error?","acceptedAnswer":{"@type":"Answer","text":"Windows Memory Integrity blocked the llama.cpp DLLs because the binaries are not code-signed, producing a Bad Image error with no output in a normal terminal. Running the terminal as administrator revealed the real error, and disabling Memory Integrity in Windows Security resolved the load failure. Developers troubleshooting local LLM tooling setups can track fixes like this on daily.dev."}}]} ```