--- title: Building Menu Vision: Real-Time Dish Recognition | daily.dev description: Yelp's engineering team describes building Menu Vision, a feature that lets users point their phone camera at a restaurant menu to instantly see photos,... canonical: https://daily.dev/posts/building-menu-vision-real-time-dish-recognition-ifqmpn3og twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Building Menu Vision: Real-Time Dish Recognition | daily.dev og:description: Yelp's engineering team describes building Menu Vision, a feature that lets users point their phone camera at a restaurant menu to instantly see photos,... og:url: https://daily.dev/posts/building-menu-vision-real-time-dish-recognition-ifqmpn3og og:image: https://api.daily.dev/og/posts/iFQmPN3og.png og:image:alt: Building Menu Vision: Real-Time Dish Recognition og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Building Menu Vision: Real-Time Dish Recognition **[Yelp Engineering](https://daily.dev/sources/yelp)** · 13 min read · 0 upvotes · 0 comments ## Summary Yelp's engineering team describes building Menu Vision, a feature that lets users point their phone camera at a restaurant menu to instantly see photos, reviews, and prices for dishes. The system started as a two-day hackathon prototype using an Android ML kit for text recognition, then evolved into a production feature built in six weeks combining on-device text recognition, client-side fuzzy matching, prefetched data, and a Cassandra-backed dish database. After launch in October 2025, the team found matching accuracy issues, limited dish coverage, and menu data inconsistencies, which they addressed with an LLM-based data pipeline, a three-phase matching algorithm (exact, substring, Jaro-Winkler similarity), and a redesign from text pills to rich visual dish cards with QR code detection and inventory fallback, shipped in an April 2026 update. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do you match OCR-scanned menu text to database dish names when spelling and formatting vary? A three-phase approach works well: first try exact character-by-character matching against the dish name and known synonyms, then fall back to bidirectional substring matching to catch modifiers like dietary tags, and finally apply Jaro-Winkler similarity matching prioritized toward the start of the string to handle OCR errors, spelling variants, and transliterations. _daily.dev surfaces engineering writeups like this for developers designing their own fuzzy text-matching pipelines._ ### What is Yelp's Menu Vision feature and how does it work? Menu Vision is a camera-based feature that lets users point their phone at a restaurant menu to identify dishes in real time and see photos, reviews, and prices from other diners. It performs on-device text recognition and dish matching using native ML frameworks on iOS and Android, prefetching dish data when a user opens a business page to avoid server round-trips during scanning. _Developers researching on-device computer vision architectures can track case studies like this on daily.dev._ ### Why did an exact-match algorithm fail for menu dish recognition and what replaced it? An exact character-for-character match was too strict, so scans of variants like "Garlic Noodle" or "Garlic Noodles w/ Pork" failed to match the stored "Garlic Noodles" entry, causing missed dish results even when photo data existed. This was replaced with a three-phase system adding substring matching and Jaro-Winkler similarity scoring as fallbacks. _Teams debugging brittle string-matching logic can follow real-world fixes like this via daily.dev._ ## Similar posts on daily.dev - [Yelp’s AI assistant can now scan restaurant menus to show you what dishes look like](https://daily.dev/posts/yelp-s-ai-assistant-can-now-scan-restaurant-menus-to-show-you-what-dishes-look-like-ol49kkbvd) · TechCrunch · 1 upvotes · 0 comments - [Uber Improves Restaurant Recommendations Using Real-Time Signals and Listwise Ranking](https://daily.dev/posts/uber-improves-restaurant-recommendations-using-real-time-signals-and-listwise-ranking-h5fa8iuso) · InfoQ · 0 upvotes · 0 comments - [Yelp’s updated AI assistant can answer questions and book a restaurant or service in one conversation](https://daily.dev/posts/yelp-s-updated-ai-assistant-can-answer-questions-and-book-a-restaurant-or-service-in-one-conversatio-iz9vfgkay) · TechCrunch · 0 upvotes · 0 comments - [Personalized Restaurant Ranking with a Two-Tower Embedding Variant](https://daily.dev/posts/personalized-restaurant-ranking-with-a-two-tower-embedding-variant-hexnnwxgq) · Towards Data Science · 1 upvotes · 0 comments --- Tags: [#career](https://daily.dev/tags/career), [#computer-vision](https://daily.dev/tags/computer-vision), [#apache-cassandra](https://daily.dev/tags/apache-cassandra) [View this post on daily.dev](https://daily.dev/posts/building-menu-vision-real-time-dish-recognition-ifqmpn3og) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Building Menu Vision: Real-Time Dish Recognition","url":"https://daily.dev/posts/building-menu-vision-real-time-dish-recognition-ifqmpn3og","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/building-menu-vision-real-time-dish-recognition-ifqmpn3og"},"datePublished":"2026-08-20T18:47:48.966Z","dateModified":"2026-08-20T18:48:13.191Z","description":"Yelp's engineering team describes building Menu Vision, a feature that lets users point their phone camera at a restaurant menu to instantly see photos,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2e6ef5fb06f19c4f8ce4951df3d6502e?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/2e6ef5fb06f19c4f8ce4951df3d6502e?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Yelp Engineering","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Yelp Engineering","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/92002b3f0a104326937d9dbe0a0f66ee","url":"https://daily.dev/sources/yelp"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/building-menu-vision-real-time-dish-recognition-ifqmpn3og","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"career,computer-vision,apache-cassandra","timeRequired":"PT13M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Yelp Engineering","item":"https://daily.dev/sources/yelp"},{"@type":"ListItem","position":3,"name":"Building Menu Vision: Real-Time Dish Recognition"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/building-menu-vision-real-time-dish-recognition-ifqmpn3og#faq","mainEntity":[{"@type":"Question","name":"How do you match OCR-scanned menu text to database dish names when spelling and formatting vary?","acceptedAnswer":{"@type":"Answer","text":"A three-phase approach works well: first try exact character-by-character matching against the dish name and known synonyms, then fall back to bidirectional substring matching to catch modifiers like dietary tags, and finally apply Jaro-Winkler similarity matching prioritized toward the start of the string to handle OCR errors, spelling variants, and transliterations. daily.dev surfaces engineering writeups like this for developers designing their own fuzzy text-matching pipelines."}},{"@type":"Question","name":"What is Yelp's Menu Vision feature and how does it work?","acceptedAnswer":{"@type":"Answer","text":"Menu Vision is a camera-based feature that lets users point their phone at a restaurant menu to identify dishes in real time and see photos, reviews, and prices from other diners. It performs on-device text recognition and dish matching using native ML frameworks on iOS and Android, prefetching dish data when a user opens a business page to avoid server round-trips during scanning. Developers researching on-device computer vision architectures can track case studies like this on daily.dev."}},{"@type":"Question","name":"Why did an exact-match algorithm fail for menu dish recognition and what replaced it?","acceptedAnswer":{"@type":"Answer","text":"An exact character-for-character match was too strict, so scans of variants like \"Garlic Noodle\" or \"Garlic Noodles w/ Pork\" failed to match the stored \"Garlic Noodles\" entry, causing missed dish results even when photo data existed. This was replaced with a three-phase system adding substring matching and Jaro-Winkler similarity scoring as fallbacks. Teams debugging brittle string-matching logic can follow real-world fixes like this via daily.dev."}}]} ```