--- title: Large Language Models On-Device with MediaPipe and... description: TensorFlow Lite and MediaPipe have released the experimental MediaPipe LLM Inference API, which allows Large Language Models (LLMs) to run fully on-device... canonical: https://daily.dev/posts/large-language-models-on-device-with-mediapipe-and-tensorflow-lite-hc09lki2r twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Large Language Models On-Device with MediaPipe and TensorFlow Lite | daily.dev og:description: TensorFlow Lite and MediaPipe have released the experimental MediaPipe LLM Inference API, which allows Large Language Models (LLMs) to run fully on-device... og:url: https://daily.dev/posts/large-language-models-on-device-with-mediapipe-and-tensorflow-lite-hc09lki2r og:image: https://api.daily.dev/og/posts/hc09LkI2r.png og:image:alt: Large Language Models On-Device with MediaPipe and TensorFlow Lite og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Large Language Models On-Device with MediaPipe and TensorFlow Lite **[Google Developers](https://daily.dev/sources/googledevs)** · 5 min read · 5 upvotes · 0 comments ## Summary TensorFlow Lite and MediaPipe have released the experimental MediaPipe LLM Inference API, which allows Large Language Models (LLMs) to run fully on-device across platforms. This new capability streamlines on-device LLM integration for web developers and supports Web, Android, and iOS. The LLM Inference API can be used by converting model weights, including the SDK in your application, hosting the TensorFlow Lite Flatbuffer, and using the API to generate text responses from the model. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: --- Tags: [#python](https://daily.dev/tags/python), [#tensorflow](https://daily.dev/tags/tensorflow) [View this post on daily.dev](https://daily.dev/posts/large-language-models-on-device-with-mediapipe-and-tensorflow-lite-hc09lki2r) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Large Language Models On-Device with MediaPipe and TensorFlow Lite","url":"https://daily.dev/posts/large-language-models-on-device-with-mediapipe-and-tensorflow-lite-hc09lki2r","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/large-language-models-on-device-with-mediapipe-and-tensorflow-lite-hc09lki2r"},"datePublished":"2024-04-25T06:36:47.960Z","dateModified":"2024-05-09T09:23:16.455Z","description":"TensorFlow Lite and MediaPipe have released the experimental MediaPipe LLM Inference API, which allows Large Language Models (LLMs) to run fully on-device...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/dd196deb39753c3f005b863a7579123a?_a=AQAEufR","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/dd196deb39753c3f005b863a7579123a?_a=AQAEufR","isAccessibleForFree":true,"articleSection":"Google Developers","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Google Developers","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/ea4bbded4e7f45ccb82e121a3156535f","url":"https://daily.dev/sources/googledevs"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/large-language-models-on-device-with-mediapipe-and-tensorflow-lite-hc09lki2r","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":5},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,tensorflow","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Google Developers","item":"https://daily.dev/sources/googledevs"},{"@type":"ListItem","position":3,"name":"Large Language Models On-Device with MediaPipe and TensorFlow Lite"}]} ```