--- title: How Apple Fits an LLM on Your iPhone | daily.dev description: Apple runs a 3-billion-parameter LLM on iPhones by combining compression techniques that cut memory and processing costs. Palettization and quantization-aware... canonical: https://daily.dev/posts/how-apple-fits-an-llm-on-your-iphone-1kra85ylm twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How Apple Fits an LLM on Your iPhone | daily.dev og:description: Apple runs a 3-billion-parameter LLM on iPhones by combining compression techniques that cut memory and processing costs. Palettization and quantization-aware... og:url: https://daily.dev/posts/how-apple-fits-an-llm-on-your-iphone-1kra85ylm og:image: https://api.daily.dev/og/posts/1kRa85YlM.png og:image:alt: How Apple Fits an LLM on Your iPhone og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Apple Fits an LLM on Your iPhone **[ByteByteGo](https://daily.dev/sources/bytebytego)** · 10 min read · 15 upvotes · 0 comments ## Summary Apple runs a 3-billion-parameter LLM on iPhones by combining compression techniques that cut memory and processing costs. Palettization and quantization-aware training shrink weights to 2 bits per decoder parameter, while an earlier version averaged 3.7 bits per weight and stored the 3 billion weights in approximately 1.39GB. Cross-layer KV sharing cuts KV-cache memory by 37.5%. LoRA adapters let multiple features share one base model instead of duplicating it, and tasks too complex for the device get routed to Apple's Private Cloud Compute, which now also runs on NVIDIA GPUs in Google Cloud. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does Apple compress its on-device LLM to fit on an iPhone? Apple uses palettization, a quantization technique that groups similar weight values and stores short references to a lookup table. An earlier model mixed 2-bit and 4-bit representations averaging 3.7 bits per weight, compressing 3 billion parameters to about 1.39GB. A later model pushes decoder weights to 2 bits per weight using quantization-aware training, while the embedding table uses 4 bits. _Anyone sizing on-device models against memory limits can track these compression techniques on daily.dev._ ### What is cross-layer KV sharing and how much memory does it save? Cross-layer KV sharing is a technique that divides a model into two blocks with a 5:3 depth ratio, where layers in the second block reuse keys and values produced by the final layer of the first block. This avoids recalculating intermediate results for every layer during inference. Apple's implementation of this approach reduced KV-cache memory usage by 37.5%. _Developers optimizing inference memory for long-context models can follow techniques like this on daily.dev._ ### How does Apple use LoRA adapters to support multiple on-device AI features without duplicating the base model? Apple applies LoRA (Low-Rank Adaptation) adapters to attention matrices, attention-output projections, and feed-forward layers so multiple features, like notification summarization and text rewriting, share one base model instead of each needing a full duplicate copy. The base model's weights stay fixed while only adapter weights are trained. Apple's rank-16 adapters use 16-bit values and occupy only tens of MBs each, so they can be loaded, cached, or swapped as needed. _Engineers designing multi-feature AI products can track adapter-based approaches like this on daily.dev._ ## Similar posts on daily.dev - [How much VRAM do you need to run an LLM locally?](https://daily.dev/posts/how-much-vram-do-you-need-to-run-an-llm-locally--sjxd57zfc) · Flavio Copes · 2 upvotes · 1 comments - [Apple eyes PrismML’s on-device AI for the iPhone](https://daily.dev/posts/apple-eyes-prismml-s-on-device-ai-for-the-iphone-lsvdpozuw) · The Next Web · 0 upvotes · 0 comments - [Local LLMs Apple Silicon Mac 2026](https://daily.dev/posts/local-llms-apple-silicon-mac-2026-sx1pmkbyq) · SitePoint · 0 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#llm](https://daily.dev/tags/llm), [#apple](https://daily.dev/tags/apple), [#local-ai](https://daily.dev/tags/local-ai), [#lora](https://daily.dev/tags/lora) [View this post on daily.dev](https://daily.dev/posts/how-apple-fits-an-llm-on-your-iphone-1kra85ylm) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How Apple Fits an LLM on Your iPhone","url":"https://daily.dev/posts/how-apple-fits-an-llm-on-your-iphone-1kra85ylm","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-apple-fits-an-llm-on-your-iphone-1kra85ylm"},"datePublished":"2026-10-27T15:40:12.008Z","dateModified":"2026-10-27T15:44:13.776Z","description":"Apple runs a 3-billion-parameter LLM on iPhones by combining compression techniques that cut memory and processing costs. Palettization and quantization-aware...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1b87bca3c5d8eed1e471426aaf95cb66?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1b87bca3c5d8eed1e471426aaf95cb66?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"ByteByteGo","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ByteByteGo","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/35be29234ee14d01a9cd049c52e12753","url":"https://daily.dev/sources/bytebytego"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-apple-fits-an-llm-on-your-iphone-1kra85ylm","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":15},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,llm,apple,local-ai,lora","timeRequired":"PT10M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ByteByteGo","item":"https://daily.dev/sources/bytebytego"},{"@type":"ListItem","position":3,"name":"How Apple Fits an LLM on Your iPhone"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-apple-fits-an-llm-on-your-iphone-1kra85ylm#faq","mainEntity":[{"@type":"Question","name":"How does Apple compress its on-device LLM to fit on an iPhone?","acceptedAnswer":{"@type":"Answer","text":"Apple uses palettization, a quantization technique that groups similar weight values and stores short references to a lookup table. An earlier model mixed 2-bit and 4-bit representations averaging 3.7 bits per weight, compressing 3 billion parameters to about 1.39GB. A later model pushes decoder weights to 2 bits per weight using quantization-aware training, while the embedding table uses 4 bits. Anyone sizing on-device models against memory limits can track these compression techniques on daily.dev."}},{"@type":"Question","name":"What is cross-layer KV sharing and how much memory does it save?","acceptedAnswer":{"@type":"Answer","text":"Cross-layer KV sharing is a technique that divides a model into two blocks with a 5:3 depth ratio, where layers in the second block reuse keys and values produced by the final layer of the first block. This avoids recalculating intermediate results for every layer during inference. Apple's implementation of this approach reduced KV-cache memory usage by 37.5%. Developers optimizing inference memory for long-context models can follow techniques like this on daily.dev."}},{"@type":"Question","name":"How does Apple use LoRA adapters to support multiple on-device AI features without duplicating the base model?","acceptedAnswer":{"@type":"Answer","text":"Apple applies LoRA (Low-Rank Adaptation) adapters to attention matrices, attention-output projections, and feed-forward layers so multiple features, like notification summarization and text rewriting, share one base model instead of each needing a full duplicate copy. The base model's weights stay fixed while only adapter weights are trained. Apple's rank-16 adapters use 16-bit values and occupy only tens of MBs each, so they can be loaded, cached, or swapped as needed. Engineers designing multi-feature AI products can track adapter-based approaches like this on daily.dev."}}]} ```