Swiggy's Crew team built an on-device predictive autocomplete system for a conversational concierge app in React Native. The system uses two small AI models totaling ~90MB: a MiniLM-L12 classifier (~30MB, ~80ms) for intent/category detection and a fine-tuned SmolLM2-135M slot extractor (~60MB, ~200ms) for structured field extraction. The classifier uses hierarchical two-phase training and attention pooling, while the slot extractor uses LoRA fine-tuning and GGUF quantization with grammar constraints to guarantee clean output. Both models run entirely on-device with no server calls, achieving ~280ms end-to-end latency and offline support. Models are shipped OTA via chunked parallel downloads with safe versioned swaps, and a feedback loop using production signals drives continuous retraining.
