--- title: I ran the tiny Bonsai model on my tiny GPU. Here’s how... description: A hands-on test of PrismML's Bonsai 27B model, a 27-billion-parameter LLM whose 1-bit quantized version shrinks to 3.9GB and fits on a smartphone, run locally... canonical: https://daily.dev/posts/i-ran-the-tiny-bonsai-model-on-my-tiny-gpu-here-s-how-it-performed-dsp1h8gia twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: I ran the tiny Bonsai model on my tiny GPU. Here’s how it performed | daily.dev og:description: A hands-on test of PrismML's Bonsai 27B model, a 27-billion-parameter LLM whose 1-bit quantized version shrinks to 3.9GB and fits on a smartphone, run locally... og:url: https://daily.dev/posts/i-ran-the-tiny-bonsai-model-on-my-tiny-gpu-here-s-how-it-performed-dsp1h8gia og:image: https://api.daily.dev/og/posts/DSP1h8gia.png og:image:alt: I ran the tiny Bonsai model on my tiny GPU. Here’s how it performed og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # I ran the tiny Bonsai model on my tiny GPU. Here’s how it performed **[InfoWorld](https://daily.dev/sources/infoworld)** · 6 min read · 0 upvotes · 0 comments ## Summary A hands-on test of PrismML's Bonsai 27B model, a 27-billion-parameter LLM whose 1-bit quantized version shrinks to 3.9GB and fits on a smartphone, run locally via LM Studio 0.4.20 on a modest AMD Ryzen 5 3600 with an RTX 5060 (8GB VRAM). Despite its compact size, Bonsai's large parameter count made it slower than smaller models like Qwen 7B, topping out around 40 tokens/sec with averages of 10-20. Disabling 'thinking' sped up output but sometimes hurt accuracy (it incorrectly attributed Blood Meridian to William Gibson). Speculative decoding, a feature for faster token generation, required more VRAM than available. Coding tasks benefited from the massive 262K token context window, and overall code quality matched Gemma 4 and Qwen3.5. The model is available under Apache 2.0 with open weights on Hugging Face, and can also run in-browser via WebGPU. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How fast does the 1-bit quantized Bonsai 27B model run on a consumer GPU with only 8GB VRAM? On an NVIDIA RTX 5060 with 8GB VRAM and an AMD Ryzen 5 3600 CPU, using LM Studio 0.4.20, output speed peaked around 40 tokens per second with an average of 10-20 tokens per second. Complex coding prompts with thinking enabled took over six minutes just for the thinking phase and generated tokens as slowly as 4.1 per second, though disabling thinking raised throughput to 7.7 tokens per second at the cost of some quality. _Anyone sizing local models against their own GPU can compare real benchmark results like this on daily.dev._ ### What size is the 1-bit quantized version of the Bonsai 27B model compared to the original? The 1-bit quantized version of Bonsai 27B is about 3.9GB, down from the original model's 54GB footprint. It still supports a 262,144 token context window, speculative decoding, and flash attention, and is released with open weights under the Apache 2.0 license, making it small enough to run on a smartphone. _Developers weighing quantized model trade-offs can track results like these on daily.dev._ ### Why did disabling thinking mode cause the Bonsai 27B model to give an inaccurate answer? Disabling thinking mode sped up output and reduced time to first token but sometimes sacrificed accuracy and coherence. In one test, asking what William Gibson thought of Blade Runner produced a false claim that Gibson wrote Blood Meridian (actually written by Cormac McCarthy) when thinking was turned off, while the same question worked correctly with thinking enabled. _Engineers tuning reasoning-mode trade-offs in local LLMs can follow findings like this on daily.dev._ ## Similar posts on daily.dev - [PrismML — Introducing Ternary Bonsai: Top Intelligence at 1.58 Bits](https://daily.dev/posts/prismml-introducing-ternary-bonsai-top-intelligence-at-1-58-bits-klkxnwryn) · Hacker News · 1 upvotes · 0 comments - [AI On Your Gaming PC](https://daily.dev/posts/ai-on-your-gaming-pc-a7iadecz7) · Hackaday · 2 upvotes · 2 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#llm](https://daily.dev/tags/llm), [#gpu](https://daily.dev/tags/gpu), [#local-ai](https://daily.dev/tags/local-ai), [#lm-studio](https://daily.dev/tags/lm-studio) [View this post on daily.dev](https://daily.dev/posts/i-ran-the-tiny-bonsai-model-on-my-tiny-gpu-here-s-how-it-performed-dsp1h8gia) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"I ran the tiny Bonsai model on my tiny GPU. Here’s how it performed","url":"https://daily.dev/posts/i-ran-the-tiny-bonsai-model-on-my-tiny-gpu-here-s-how-it-performed-dsp1h8gia","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/i-ran-the-tiny-bonsai-model-on-my-tiny-gpu-here-s-how-it-performed-dsp1h8gia"},"datePublished":"2026-08-17T09:04:36.440Z","dateModified":"2026-08-17T09:45:08.793Z","description":"A hands-on test of PrismML's Bonsai 27B model, a 27-billion-parameter LLM whose 1-bit quantized version shrinks to 3.9GB and fits on a smartphone, run locally...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/cbcd8a22ec36da97412f9b77c056e3eb?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/cbcd8a22ec36da97412f9b77c056e3eb?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"InfoWorld","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"InfoWorld","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/bf6d68a999064029b0bb09aa6268f1f3","url":"https://daily.dev/sources/infoworld"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/i-ran-the-tiny-bonsai-model-on-my-tiny-gpu-here-s-how-it-performed-dsp1h8gia","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,llm,gpu,local-ai,lm-studio","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"InfoWorld","item":"https://daily.dev/sources/infoworld"},{"@type":"ListItem","position":3,"name":"I ran the tiny Bonsai model on my tiny GPU. Here’s how it performed"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/i-ran-the-tiny-bonsai-model-on-my-tiny-gpu-here-s-how-it-performed-dsp1h8gia#faq","mainEntity":[{"@type":"Question","name":"How fast does the 1-bit quantized Bonsai 27B model run on a consumer GPU with only 8GB VRAM?","acceptedAnswer":{"@type":"Answer","text":"On an NVIDIA RTX 5060 with 8GB VRAM and an AMD Ryzen 5 3600 CPU, using LM Studio 0.4.20, output speed peaked around 40 tokens per second with an average of 10-20 tokens per second. Complex coding prompts with thinking enabled took over six minutes just for the thinking phase and generated tokens as slowly as 4.1 per second, though disabling thinking raised throughput to 7.7 tokens per second at the cost of some quality. Anyone sizing local models against their own GPU can compare real benchmark results like this on daily.dev."}},{"@type":"Question","name":"What size is the 1-bit quantized version of the Bonsai 27B model compared to the original?","acceptedAnswer":{"@type":"Answer","text":"The 1-bit quantized version of Bonsai 27B is about 3.9GB, down from the original model's 54GB footprint. It still supports a 262,144 token context window, speculative decoding, and flash attention, and is released with open weights under the Apache 2.0 license, making it small enough to run on a smartphone. Developers weighing quantized model trade-offs can track results like these on daily.dev."}},{"@type":"Question","name":"Why did disabling thinking mode cause the Bonsai 27B model to give an inaccurate answer?","acceptedAnswer":{"@type":"Answer","text":"Disabling thinking mode sped up output and reduced time to first token but sometimes sacrificed accuracy and coherence. In one test, asking what William Gibson thought of Blade Runner produced a false claim that Gibson wrote Blood Meridian (actually written by Cormac McCarthy) when thinking was turned off, while the same question worked correctly with thinking enabled. Engineers tuning reasoning-mode trade-offs in local LLMs can follow findings like this on daily.dev."}}]} ```