--- title: Mastering Edge AI on Raspberry Pi with LiteRT and Gemma description: Google AI Edge's LiteRT runtime enables deploying Gemma language models directly on Raspberry Pi 5 for fully offline, real-time edge AI applications. On a... canonical: https://daily.dev/posts/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma-ueibnnbvo twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Mastering Edge AI on Raspberry Pi with LiteRT and Gemma | daily.dev og:description: Google AI Edge's LiteRT runtime enables deploying Gemma language models directly on Raspberry Pi 5 for fully offline, real-time edge AI applications. On a... og:url: https://daily.dev/posts/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma-ueibnnbvo og:image: https://api.daily.dev/og/posts/UEiBNNBVO.png og:image:alt: Mastering Edge AI on Raspberry Pi with LiteRT and Gemma og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Mastering Edge AI on Raspberry Pi with LiteRT and Gemma **[Google Developers](https://daily.dev/sources/googledevs)** · 7 min read · 0 upvotes · 0 comments ## Summary Google AI Edge's LiteRT runtime enables deploying Gemma language models directly on Raspberry Pi 5 for fully offline, real-time edge AI applications. On a Raspberry Pi 5, LiteRT-LM achieves 99 tokens/sec prefill and 9 tokens/sec decode for Gemma 4 E2B with a 1432 MB peak memory footprint, translating to ~27.3 characters/sec (~300 wpm) end-to-end generation. The post demonstrates a dual CPU/GPU architecture where LLM inference runs on the ARM Cortex-A76 CPU while vision and audio models offload to the VideoCore VII GPU via LiteRT's WebGPU (Vulkan) backend. A Reachy Mini robot demo illustrates real-time perception and response entirely offline. Setup is straightforward via `pip install litert-cli`, with models pulled from the LiteRT Hugging Face Community. Hailo AI accelerator support is also announced as coming soon. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What inference performance does Gemma 4 E2B achieve on a Raspberry Pi 5 with LiteRT-LM? On a Raspberry Pi 5, LiteRT-LM running Gemma 4 E2B achieves 99 tokens/sec for prefill and 9 tokens/sec for decode, with a peak memory footprint of 1432 MB. Thanks to the model's efficient tokenizer averaging ~4.2 characters per token, end-to-end generation reaches ~27.3 characters/sec, roughly 300 words per minute — twice the speed of normal human speech. _Developers targeting Raspberry Pi robotics or voice applications track LiteRT and Gemma performance updates on daily.dev._ ### How do I install and run a Gemma model on Raspberry Pi 5 using LiteRT CLI? Install LiteRT CLI with `pip install litert-cli`, then set your Hugging Face token via `export HUGGING_FACE_HUB_TOKEN=` and run `litert lm run --from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm gemma-4-E2B-it.litertlm`. Models are pulled directly from the LiteRT Hugging Face Community, and the CLI handles conversion, quantization, and inference in a single unified command set. _Engineers shipping edge AI prototypes find the latest LiteRT tooling news on daily.dev before it hits the docs._ ### How does LiteRT split workloads between CPU and GPU on Raspberry Pi 5 for real-time AI? LiteRT uses a dual-processing architecture where LLM inference runs on the quad-core ARM Cortex-A76 CPU (~153.6 GFLOPS FP32, ~2.0 TOPS INT8), while continuous vision and audio models are offloaded to the Broadcom VideoCore VII GPU via LiteRT's WebGPU (Vulkan) backend through ML Drift. This preserves CPU cycles for high-priority tasks and improves thermal efficiency. _Robotics developers balancing compute budgets on constrained hardware follow edge AI architecture discussions on daily.dev._ ## Similar posts on daily.dev - [Bring state-of-the-art agentic skills to the edge with Gemma 4](https://daily.dev/posts/bring-state-of-the-art-agentic-skills-to-the-edge-with-gemma-4-hucjhitib) · Google Developers · 1 upvotes · 0 comments --- Tags: [#raspberry-pi](https://daily.dev/tags/raspberry-pi), [#local-ai](https://daily.dev/tags/local-ai), [#gemma](https://daily.dev/tags/gemma) [View this post on daily.dev](https://daily.dev/posts/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma-ueibnnbvo) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Mastering Edge AI on Raspberry Pi with LiteRT and Gemma","url":"https://daily.dev/posts/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma-ueibnnbvo","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma-ueibnnbvo"},"datePublished":"2026-08-11T15:46:13.723Z","dateModified":"2026-08-14T07:05:05.846Z","description":"Google AI Edge's LiteRT runtime enables deploying Gemma language models directly on Raspberry Pi 5 for fully offline, real-time edge AI applications. On a...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e2624ae64f57f7a88df1a1830878fd46?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e2624ae64f57f7a88df1a1830878fd46?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Google Developers","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Google Developers","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/ea4bbded4e7f45ccb82e121a3156535f","url":"https://daily.dev/sources/googledevs"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma-ueibnnbvo","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"raspberry-pi,local-ai,gemma","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Google Developers","item":"https://daily.dev/sources/googledevs"},{"@type":"ListItem","position":3,"name":"Mastering Edge AI on Raspberry Pi with LiteRT and Gemma"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/mastering-edge-ai-on-raspberry-pi-with-litert-and-gemma-ueibnnbvo#faq","mainEntity":[{"@type":"Question","name":"What inference performance does Gemma 4 E2B achieve on a Raspberry Pi 5 with LiteRT-LM?","acceptedAnswer":{"@type":"Answer","text":"On a Raspberry Pi 5, LiteRT-LM running Gemma 4 E2B achieves 99 tokens/sec for prefill and 9 tokens/sec for decode, with a peak memory footprint of 1432 MB. Thanks to the model's efficient tokenizer averaging ~4.2 characters per token, end-to-end generation reaches ~27.3 characters/sec, roughly 300 words per minute — twice the speed of normal human speech. Developers targeting Raspberry Pi robotics or voice applications track LiteRT and Gemma performance updates on daily.dev."}},{"@type":"Question","name":"How do I install and run a Gemma model on Raspberry Pi 5 using LiteRT CLI?","acceptedAnswer":{"@type":"Answer","text":"Install LiteRT CLI with `pip install litert-cli`, then set your Hugging Face token via `export HUGGING_FACE_HUB_TOKEN=` and run `litert lm run --from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm gemma-4-E2B-it.litertlm`. Models are pulled directly from the LiteRT Hugging Face Community, and the CLI handles conversion, quantization, and inference in a single unified command set. Engineers shipping edge AI prototypes find the latest LiteRT tooling news on daily.dev before it hits the docs."}},{"@type":"Question","name":"How does LiteRT split workloads between CPU and GPU on Raspberry Pi 5 for real-time AI?","acceptedAnswer":{"@type":"Answer","text":"LiteRT uses a dual-processing architecture where LLM inference runs on the quad-core ARM Cortex-A76 CPU (~153.6 GFLOPS FP32, ~2.0 TOPS INT8), while continuous vision and audio models are offloaded to the Broadcom VideoCore VII GPU via LiteRT's WebGPU (Vulkan) backend through ML Drift. This preserves CPU cycles for high-priority tasks and improves thermal efficiency. Robotics developers balancing compute budgets on constrained hardware follow edge AI architecture discussions on daily.dev."}}]} ```