Yelp's engineering team describes building Menu Vision, a feature that lets users point their phone camera at a restaurant menu to instantly see photos, reviews, and prices for dishes. The system started as a two-day hackathon prototype using an Android ML kit for text recognition, then evolved into a production feature built in six weeks combining on-device text recognition, client-side fuzzy matching, prefetched data, and a Cassandra-backed dish database. After launch in October 2025, the team found matching accuracy issues, limited dish coverage, and menu data inconsistencies, which they addressed with an LLM-based data pipeline, a three-phase matching algorithm (exact, substring, Jaro-Winkler similarity), and a redesign from text pills to rich visual dish cards with QR code detection and inventory fallback, shipped in an April 2026 update.
Table of contents
How It Started: The Hackathon SparkMaking It Production-ReadyTaking It LiveEnhancing Menu Vision: Better Data, Smarter Matching, and a Richer Visual ExperienceConclusionAcknowledgementsQuestions this post answers
How do you match OCR-scanned menu text to database dish names when spelling and formatting vary?
A three-phase approach works well: first try exact character-by-character matching against the dish name and known synonyms, then fall back to bidirectional substring matching to catch modifiers like dietary tags, and finally apply Jaro-Winkler similarity matching prioritized toward the start of the string to handle OCR errors, spelling variants, and transliterations. daily.dev surfaces engineering writeups like this for developers designing their own fuzzy text-matching pipelines.
What is Yelp's Menu Vision feature and how does it work?
Menu Vision is a camera-based feature that lets users point their phone at a restaurant menu to identify dishes in real time and see photos, reviews, and prices from other diners. It performs on-device text recognition and dish matching using native ML frameworks on iOS and Android, prefetching dish data when a user opens a business page to avoid server round-trips during scanning. Developers researching on-device computer vision architectures can track case studies like this on daily.dev.
Why did an exact-match algorithm fail for menu dish recognition and what replaced it?
An exact character-for-character match was too strict, so scans of variants like "Garlic Noodle" or "Garlic Noodles w/ Pork" failed to match the stored "Garlic Noodles" entry, causing missed dish results even when photo data existed. This was replaced with a three-phase system adding substring matching and Jaro-Winkler similarity scoring as fallbacks. Teams debugging brittle string-matching logic can follow real-world fixes like this via daily.dev.