Function Gemma ships at 270 million parameters and processes nearly 2,000 tokens per second prefill on a Pixel 7. Out of the box, on a fixed set of app intents, it hits 46% accuracy. Fine-tuned on a synthetically generated dataset, it clears 90% on eight of ten functions.

Cormac Brick covers the two options developers have for on-device AI: Gemini Nano via AI core for common tasks, and LiteRT-LM for custom models that ship inside your app. The session walks through a live skill harness built on Gemma 4 with a restaurant roulette demo running fully on-device, and Eloquent, a production transcription app built by chaining two models under a few hundred million parameters.

Speaker info:
- https://www.linkedin.com/in/cbrick/

AI Engineer

A Google engineer presents techniques for deploying tiny LLMs (sub-billion parameter models) as on-device agents using the Google AI Edge stack. Key topics include the LiteRT/LiteLM runtime, a skill harness built on top of Gemma 4 via AI Core for agent-style function calling, and fine-tuning tiny models with synthetic data. A concrete example shows fine-tuning Function Gemma (270M parameters) to improve function-calling accuracy from 46% to over 90% for 8 out of 10 target functions. The talk also covers a production transcription app (Eloquent) built with chained tiny models, and the Google AI Edge Gallery open-source app for testing models on Android and iOS.

From 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents — Cormac Brick, Google