A deep technical walkthrough of building an on-device, real-time face recognition pipeline in React Native using react-native-vision-camera v5. It covers the full relay of stages: capturing camera frames efficiently, face detection (comparing YuNet, BlazeFace, SCRFD, RetinaFace), landmark-based alignment, passive liveness detection against spoofing, embedding generation with model comparisons (SFace, ArcFace variants) including licensing pitfalls, multi-sample enrollment and centroid matching, frame-to-UI threading via Reanimated shared values, and detection/tracking scheduling to sustain 30fps. Includes real benchmark numbers from a Galaxy S21 Ultra, a 5,000-person gallery accuracy test, and hard-won production lessons like mirrored-frame alignment bugs, thermal throttling, and native memory pressure.
Table of contents
The pipeline at a glanceSmile for the cameraTelling who is whoGetting results back to the JS runtimeNot every frame needs the whole pipelineOne frame, end to endMaking it survive the real worldWhat it takes, and where to startQuestions this post answers
Which face detection model should I use for real-time face recognition on mobile with React Native?
YuNet is the recommended default for mobile face detection: it is about 230 KB, runs in 3 to 6 ms per frame at 128 input resolution on a Galaxy S21 Ultra class CPU, provides 5 landmarks that map directly onto the ArcFace alignment template, and carries an MIT license. Alternatives like SCRFD and RetinaFace offer higher accuracy for crowded scenes but their pretrained weights are research-only, blocking commercial use. daily.dev surfaces engineering writeups like this for teams weighing computer vision model trade-offs.
Why does mirrored front camera input break face recognition accuracy?
Front-camera buffers can arrive mirrored, and aligning a mirrored face against a non-mirrored canonical template distorts every face the same systematic way, compressing the embedding space so different people start matching each other, producing false accepts. The fix is using a flipped alignment template for mirrored frames, applied during the landmark-based warp to the 112x112 canonical crop. track subtle production bugs like this on daily.dev before they surface as failed face-recognition matches.
Which face embedding model is best for commercial on-device face recognition apps?
SFace is the recommended shipping default because it carries an Apache-2.0 license, unlike ArcFace MobileFaceNet or ArcFace ResNet-50 whose InsightFace pretrained weights are restricted to non-commercial research. SFace is 37MB (19MB quantized to int8), achieves about 99.6% on LFW, and runs in roughly 29ms per face on a Galaxy S21 Ultra class CPU using ONNX Runtime with XNNPACK. compare model licensing and performance trade-offs like these before shipping, tracked with daily.dev.