--- title: cua/blog/gpu-passthrough-macos-vms.md at main · trycua/cua description: The Cua team released a Metal capability shim — a process-scoped compatibility layer — that intercepts Metal GPU capability queries inside macOS VMs running on... canonical: https://daily.dev/posts/cua-blog-gpu-passthrough-macos-vms-md-at-main-trycua-cua-3jsfxxx0k twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: cua/blog/gpu-passthrough-macos-vms.md at main · trycua/cua | daily.dev og:description: The Cua team released a Metal capability shim — a process-scoped compatibility layer — that intercepts Metal GPU capability queries inside macOS VMs running on... og:url: https://daily.dev/posts/cua-blog-gpu-passthrough-macos-vms-md-at-main-trycua-cua-3jsfxxx0k og:image: https://api.daily.dev/og/posts/3jSfxxx0k.png og:image:alt: cua/blog/gpu-passthrough-macos-vms.md at main · trycua/cua og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # cua/blog/gpu-passthrough-macos-vms.md at main · trycua/cua **[Hacker News](https://daily.dev/sources/hn)** · 9 min read · 1 upvotes · 0 comments ## Summary The Cua team released a Metal capability shim — a process-scoped compatibility layer — that intercepts Metal GPU capability queries inside macOS VMs running on Apple's Virtualization.framework. By reporting Apple family 9 (instead of ~5) and raising threadgroup memory from 32 KB to 64 KB, the shim enables llama.cpp to use newer SIMD-group matrix, SIMD-group reduction, and bfloat16 Metal paths. On an M1 Ultra, TinyLlama 1.1B prompt processing jumped from 432 to 4,787 tok/s (11.08× speedup, 98.25% of bare-metal), and token generation improved 16.36×. Gemma 4 12B saw 7.20× prompt and 14.54× generation speedups, reaching 99.59% and 94.82% of bare-metal respectively. MLX-LM showed no change since it already used fast paths. The shim is experimental, version-sensitive, and scoped to the injected process only. Source, build scripts, and raw benchmark logs are published under a permissive license. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much faster does llama.cpp run inside a macOS VM after applying the Lume Metal capability shim compared to a stock VM? On an M1 Ultra running macOS Tahoe, TinyLlama 1.1B prompt processing improved 11.08x (from 431.86 to 4,786.70 tok/s) and token generation improved 16.36x (from 12.63 to 206.60 tok/s) after the shim intercepted Metal capability queries. Prompt processing reached 98.25% of bare-metal speed and generation reached 72.06%. _Anyone squeezing more LLM throughput out of a macOS VM can track experimental GPU tricks like this one on daily.dev._ ### What does the Lume Metal capability shim actually change to speed up GPU performance in a macOS virtual machine? It intercepts selected Metal capability queries inside one guest process and changes the returned values: reporting Apple GPU family support up through family 9 (supportsFamily:1009) and raising maximum threadgroup memory from 32 KB to 64 KB. This tricks applications like llama.cpp into selecting newer SIMD-group matrix, SIMD-group reduction, and bfloat16 code paths on the existing paravirtualized virtual GPU, without any physical PCI or VFIO-style passthrough. _Developers weighing VM GPU tradeoffs on Apple Silicon can follow deep dives like this on daily.dev._ ### Does the macOS Virtualization.framework GPU passthrough shim work the same for every model, or does it help some workloads more than others? Gains vary by workload: TinyLlama 1.1B and Gemma 4 12B QAT Q4_0 both saw large speedups (7x-16x) with llama.cpp, but MLX-LM 0.31.3 running Llama-3.2-3B-Instruct-4bit on MLX 0.32.0 stayed essentially flat (0.99x-1.005x) because it was already fast in the stock VM, and forcing Metal 3 capability actually broke MLX's residency set requests. _Anyone benchmarking inference stacks across VM setups can keep tabs on results like this via daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 46 comments across hackernews (as of 2026-09-14). **TL;DR:** Commenters largely appreciate the technical ingenuity of the Metal capability shim and its real-world speedups inside macOS VMs, though many quickly clarified the improvement is scoped only to Virtualization.framework VMs — not a general llama.cpp or Apple Silicon speedup. A side debate erupted over whether the blog post was AI-written. **Sentiment:** 45% positive · 35% mixed · 20% skeptical **The case for** - The shim achieves near-bare-metal GPU performance inside macOS VMs, which is a meaningful unlock for local-first and isolated agent workflows. - The approach is technically elegant — intercepting Metal capability queries at the API boundary rather than modifying llama.cpp or the VM itself. - Apple Silicon hardware is so capable that developers keep finding ways to push through platform gaps, making work like this worthwhile. **The pushback** - The speedup applies only to users running llama.cpp inside Virtualization.framework VMs, not to bare-metal Apple Silicon users — the title was seen as misleading. - Apple's paravirtual GPU deliberately exposes a conservative, reduced Metal capability profile, and the shim's correctness depends on the underlying hardware actually supporting the reported capabilities — which is version-sensitive and not guaranteed. - MLX-LM showed no improvement, suggesting the shim's benefit is narrow and app-specific. - The blog post was widely suspected of being AI-generated, drawing criticism about authenticity and readability. **By community** - hackernews (mixed): Technically impressed by the VM capability-reporting fix and the benchmark numbers, but quick to flag the narrow scope, the AI-written prose controversy, and unanswered questions about why Apple exposes a reduced Metal profile in the first place. **Hottest debate:** Whether the blog post was written by AI — and whether that violates community norms — consumed nearly as much thread energy as the technical substance itself. **Open questions** - Why does Apple's Virtualization.framework deliberately report a lesser Metal GPU family and lower threadgroup memory limit instead of exposing the host's full capabilities? - Will similar improvements hold on other Apple Silicon variants like M1 Pro, M3 Pro, or M5 Max? - Could a simpler override (e.g., an environment variable or plist key) achieve the same capability-reporting fix without a custom shim? - Will future Apple Silicon chips (M6, M7) with rumored larger unified memory and Neural Accelerators change the calculus for local LLM inference in VMs? **Highlights** > Because nobody knows. Apple doesn't let you "pass" the GPU through to a VM like most other ARM/x86_64 processors (forwarding interrupts and PCIe memory regions). There are symbols defined to do this within the kernel (if you dump the binary) but they aren't used in retail macos. Instead you end up creating a paravirtual device that emulates the GPU acting like a 'normal PCI device' which you give to clients. This is usually reserved (by other hardware vendors) for when you're doing multi-tenat time sharing of higher end GPUs (like Nvidia enterprise cards can do). These paravirtualized GPUs then just have 'less features' and Apple (being Apple) states no reason why. > — [hugmynutus on hackernews · 1 comments](https://news.ycombinator.com/item?id=49261865) > > this won't speed up llama.cpp for everyone, just for users running it in this particular kind of Virtualization.framework VM. correct. these figures apply to llama.cpp inside the macOS guest configuration we tested. Lume is the VM frontend we used, while Apple's Virtualization.framework provides the virtual GPU. bare-metal llama.cpp is unaffected. > The fix here works around a problem where the VM was causing llama.cpp to select the wrong kernels. mostly, with one nuance: llama.cpp is selecting the correct kernels for the capability answers it receives. the stock guest reports an older Apple GPU family and a 32 KB threadgroup memory limit, so llama.cpp chooses slower kernels. Our process-scoped layer reports the tested Apple 9 and 64 KB values while allowing llama.cpp to select newer paths that the paravirtual GPU successfully execute the layer itself though works at the Metal API boundary, independently of llama.cpp. other Metal compute and graphics apps now may select newer paths from the same capability answers, although this is still preliminary and each app needs separate testing. for example, MLX-LM stayed flat in our tests historically related limitations have been coming up across Apple Silicon VM frontends for a while e.g. Tart tracked MPS/GPU support back in 2023: - https://github.com/openai/tart/issues/501 - https://github.com/openai/tart/issues/1032 UTM also has related cases where apps detect the Apple paravirtual Metal device but falls back to software rendering: https://github.com/utmapp/UTM/issues/7671 > — [frabonacci on hackernews · 1 comments](https://news.ycombinator.com/item?id=49260087) > > 11.08× faster and generated tokens 16.36× faster than the same workload in the same stock VM. So this was the comparison, for me the title was a bit confusing > — [thehamkercat on hackernews · 1 comments](https://news.ycombinator.com/item?id=49259643) > All this work to get the Mac to be a platform useful for AI is being done despite Apple's efforts. They're famously pissed at Nvidia since the Nvidia + Intel Macs due to heat and other issues. But then the OS is a bit closed off and they move slow and are more focused on milking iOS and the App Store and services for money BUT the PA-Semi purchase and Apple Silicon and everything following it has made the hardware just so amazing and useful that despite all that people build for it. I love the platform. I'm happy to see people building on it. AND! if we ever get an M7 chip with the rumored 1.5TB of available ram all this work will not have been in vain. You think the ai acceleration is nice in the M5 wait till M7 and M8. > — [gigatexal on hackernews · 1 comments](https://news.ycombinator.com/item?id=49261187) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49259339) · 271 points · 46 comments ## Similar posts on daily.dev - [I ran local LLMs on Intel's cheapest iGPU, and the results were surprisingly decent](https://daily.dev/posts/i-ran-local-llms-on-intel-s-cheapest-igpu-and-the-results-were-surprisingly-decent-ugoo2wwdu) · XDA Developers · 0 upvotes · 0 comments - [Your Local LLM Could Be Leaving Mac Performance on the Table](https://daily.dev/posts/your-local-llm-could-be-leaving-mac-performance-on-the-table-fxxfy93gy) · Medium · 0 upvotes · 0 comments - [Docker Model Runner Adds vLLM Support on macOS](https://daily.dev/posts/docker-model-runner-adds-vllm-support-on-macos-1lss5kmxo) · Docker · 1 upvotes · 0 comments - [Zero-Copy GPU Inference from WebAssembly on Apple Silicon](https://daily.dev/posts/zero-copy-gpu-inference-from-webassembly-on-apple-silicon-oqawturhp) · Hacker News · 13 upvotes · 0 comments - [Local LLMs Apple Silicon Mac 2026](https://daily.dev/posts/local-llms-apple-silicon-mac-2026-sx1pmkbyq) · SitePoint · 0 upvotes · 0 comments --- Tags: [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/cua-blog-gpu-passthrough-macos-vms-md-at-main-trycua-cua-3jsfxxx0k) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"cua/blog/gpu-passthrough-macos-vms.md at main · trycua/cua","url":"https://daily.dev/posts/cua-blog-gpu-passthrough-macos-vms-md-at-main-trycua-cua-3jsfxxx0k","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/cua-blog-gpu-passthrough-macos-vms-md-at-main-trycua-cua-3jsfxxx0k"},"datePublished":"2026-08-11T15:56:32.821Z","dateModified":"2026-09-14T09:12:41.438Z","description":"The Cua team released a Metal capability shim — a process-scoped compatibility layer — that intercepts Metal GPU capability queries inside macOS VMs running on...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7a0619377f0db0dde704b28b101b31ab?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7a0619377f0db0dde704b28b101b31ab?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/cua-blog-gpu-passthrough-macos-vms-md-at-main-trycua-cua-3jsfxxx0k","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llama-cpp","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"cua/blog/gpu-passthrough-macos-vms.md at main · trycua/cua"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/cua-blog-gpu-passthrough-macos-vms-md-at-main-trycua-cua-3jsfxxx0k#faq","mainEntity":[{"@type":"Question","name":"How much faster does llama.cpp run inside a macOS VM after applying the Lume Metal capability shim compared to a stock VM?","acceptedAnswer":{"@type":"Answer","text":"On an M1 Ultra running macOS Tahoe, TinyLlama 1.1B prompt processing improved 11.08x (from 431.86 to 4,786.70 tok/s) and token generation improved 16.36x (from 12.63 to 206.60 tok/s) after the shim intercepted Metal capability queries. Prompt processing reached 98.25% of bare-metal speed and generation reached 72.06%. Anyone squeezing more LLM throughput out of a macOS VM can track experimental GPU tricks like this one on daily.dev."}},{"@type":"Question","name":"What does the Lume Metal capability shim actually change to speed up GPU performance in a macOS virtual machine?","acceptedAnswer":{"@type":"Answer","text":"It intercepts selected Metal capability queries inside one guest process and changes the returned values: reporting Apple GPU family support up through family 9 (supportsFamily:1009) and raising maximum threadgroup memory from 32 KB to 64 KB. This tricks applications like llama.cpp into selecting newer SIMD-group matrix, SIMD-group reduction, and bfloat16 code paths on the existing paravirtualized virtual GPU, without any physical PCI or VFIO-style passthrough. Developers weighing VM GPU tradeoffs on Apple Silicon can follow deep dives like this on daily.dev."}},{"@type":"Question","name":"Does the macOS Virtualization.framework GPU passthrough shim work the same for every model, or does it help some workloads more than others?","acceptedAnswer":{"@type":"Answer","text":"Gains vary by workload: TinyLlama 1.1B and Gemma 4 12B QAT Q4_0 both saw large speedups (7x-16x) with llama.cpp, but MLX-LM 0.31.3 running Llama-3.2-3B-Instruct-4bit on MLX 0.32.0 stayed essentially flat (0.99x-1.005x) because it was already fast in the stock VM, and forcing Metal 3 capability actually broke MLX's residency set requests. Anyone benchmarking inference stacks across VM setups can keep tabs on results like this via daily.dev."}}]} ```