On-device AI runs AI inference directly on local hardware rather than remote cloud servers. Models are trained in the cloud then compressed using techniques like quantization, pruning, and knowledge distillation before deployment to devices. Key benefits include reduced latency, improved privacy, offline functionality, and lower cloud costs. Challenges include limited compute/storage resources, model size constraints, complex OTA update distribution, and power consumption. Common use cases span mobile apps, IoT/smart home devices, retail personalization, and industrial monitoring. Hybrid edge-plus-cloud architectures offer a practical middle ground. Recommended frameworks include TensorFlow Lite, PyTorch Mobile, ONNX Runtime, Core ML, and Android NNAPI.
