--- title: Real-Time Retail Intelligence: Building E-Commerce... description: A reference architecture describes how a fashion e-commerce platform in Asia, serving over 1 million monthly active users across 100,000+ SKUs, built a... canonical: https://daily.dev/posts/real-time-retail-intelligence-building-e-commerce-recommendations-with-lakebase-and-ai-search-on-da-vii3pv8zq twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks | daily.dev og:description: A reference architecture describes how a fashion e-commerce platform in Asia, serving over 1 million monthly active users across 100,000+ SKUs, built a... og:url: https://daily.dev/posts/real-time-retail-intelligence-building-e-commerce-recommendations-with-lakebase-and-ai-search-on-da-vii3pv8zq og:image: https://api.daily.dev/og/posts/vIi3pV8zq.png og:image:alt: Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks **[databricks](https://daily.dev/sources/databricks)** · 7 min read · 0 upvotes · 0 comments ## Summary A reference architecture describes how a fashion e-commerce platform in Asia, serving over 1 million monthly active users across 100,000+ SKUs, built a production recommendation and ranking engine entirely on Databricks. Clickstream events (~1,000/sec) are ingested via Zerobus into a medallion lakehouse architecture governed by Unity Catalog. Two serving paths are used: Path A pre-computes nightly batch recommendations stored in Lakebase online tables for sub-two-digit-millisecond key-value lookups, while Path B runs a real-time 3-stage funnel (AI Search retrieval, LightGBM scoring, business rules) synchronously for session-aware surfaces like 'similar items' or dynamic search re-ranking. Cold-start strategies use demographic and product-attribute embeddings, models retrain weekly with champion/challenger deployment via MLflow, and drift detection monitors feature and prediction distributions. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I build a low-latency real-time recommendation system for e-commerce using Databricks? Use a dual-path architecture: pre-compute batch recommendations nightly for predictable surfaces like homepages, storing results in Lakebase online tables for sub-50ms key-value lookups, while running a synchronous 3-stage funnel (AI Search retrieval, LightGBM scoring, business rules) for session-aware surfaces like 'similar items' or live search re-ranking, all unified under Unity Catalog governance. _daily.dev surfaces architecture deep dives like this for engineers designing real-time recommendation pipelines._ ### How does Zerobus Ingest handle clickstream data for feature engineering without a message broker? Zerobus Ingest, part of Lakeflow Connect, lands events directly into Unity Catalog Delta tables without requiring a self-managed message broker, accepting data from any standard Kafka producer client (Java, Python, Go) via a simple bootstrap server configuration change, powering offline feature computation while in-session signals bypass storage entirely during real-time inference. _Engineers evaluating streaming ingestion options can follow daily.dev for more lakehouse pipeline patterns._ ### How do recommendation systems handle cold-start problems for new users and new products? New users get a default embedding built from demographic signals like location, device type, and sign-up context, placing them in a behavioral cluster that converges as they interact. New products get an embedding from attributes like title, category, brand, price, and image features, inheriting scores from nearest neighbors in vector space, with full surfacing at the next daily batch cycle. _Teams tackling cold-start challenges in recommender systems can find similar deep dives on daily.dev._ ## Similar posts on daily.dev - [From Tribal Knowledge to Instant Answers: Building Reffy on Databricks](https://daily.dev/posts/from-tribal-knowledge-to-instant-answers-building-reffy-on-databricks-ulslgel1o) · databricks · 1 upvotes · 0 comments - [Building Real-Time Recommendations with Snowflake Online Feature Serving — Two-Tower Networks](https://daily.dev/posts/building-real-time-recommendations-with-snowflake-online-feature-serving-two-tower-networks-mqwgcqwal) · Snowflake Community · 1 upvotes · 0 comments - [Solving retail’s cold-start problem: Malachyte's recommendation reinvention](https://daily.dev/posts/solving-retail-s-cold-start-problem-malachyte-s-recommendation-reinvention-8tnuhwbus) · Google Cloud · 0 upvotes · 0 comments - [Real-Time AI Pipelines: Building a Kafka \+ Kubernetes \+ Databricks Stack for Agentic AI Systems](https://daily.dev/posts/real-time-ai-pipelines-building-a-kafka-kubernetes-databricks-stack-for-agentic-ai-systems-qoytpduhq) · Medium · 0 upvotes · 0 comments - [Databricks wants to merge the two databases every company runs](https://daily.dev/posts/databricks-wants-to-merge-the-two-databases-every-company-runs-mivqdovba) · The New Stack · 2 upvotes · 0 comments --- Tags: [#vector-search](https://daily.dev/tags/vector-search), [#databricks](https://daily.dev/tags/databricks), [#mlops](https://daily.dev/tags/mlops), [#recommendation-systems](https://daily.dev/tags/recommendation-systems) [View this post on daily.dev](https://daily.dev/posts/real-time-retail-intelligence-building-e-commerce-recommendations-with-lakebase-and-ai-search-on-da-vii3pv8zq) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks","url":"https://daily.dev/posts/real-time-retail-intelligence-building-e-commerce-recommendations-with-lakebase-and-ai-search-on-da-vii3pv8zq","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/real-time-retail-intelligence-building-e-commerce-recommendations-with-lakebase-and-ai-search-on-da-vii3pv8zq"},"datePublished":"2026-10-02T11:09:43.534Z","dateModified":"2026-10-02T11:10:07.233Z","description":"A reference architecture describes how a fashion e-commerce platform in Asia, serving over 1 million monthly active users across 100,000+ SKUs, built a...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c7a26760b404500392c6b4bcd0b9fa0c?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c7a26760b404500392c6b4bcd0b9fa0c?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"databricks","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"databricks","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/fa7aa720f2db4d1eba826814730482c8","url":"https://daily.dev/sources/databricks"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/real-time-retail-intelligence-building-e-commerce-recommendations-with-lakebase-and-ai-search-on-da-vii3pv8zq","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"vector-search,databricks,mlops,recommendation-systems","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"databricks","item":"https://daily.dev/sources/databricks"},{"@type":"ListItem","position":3,"name":"Real-Time Retail Intelligence: Building E-Commerce Recommendations with Lakebase and AI Search on Databricks"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/real-time-retail-intelligence-building-e-commerce-recommendations-with-lakebase-and-ai-search-on-da-vii3pv8zq#faq","mainEntity":[{"@type":"Question","name":"How can I build a low-latency real-time recommendation system for e-commerce using Databricks?","acceptedAnswer":{"@type":"Answer","text":"Use a dual-path architecture: pre-compute batch recommendations nightly for predictable surfaces like homepages, storing results in Lakebase online tables for sub-50ms key-value lookups, while running a synchronous 3-stage funnel (AI Search retrieval, LightGBM scoring, business rules) for session-aware surfaces like 'similar items' or live search re-ranking, all unified under Unity Catalog governance. daily.dev surfaces architecture deep dives like this for engineers designing real-time recommendation pipelines."}},{"@type":"Question","name":"How does Zerobus Ingest handle clickstream data for feature engineering without a message broker?","acceptedAnswer":{"@type":"Answer","text":"Zerobus Ingest, part of Lakeflow Connect, lands events directly into Unity Catalog Delta tables without requiring a self-managed message broker, accepting data from any standard Kafka producer client (Java, Python, Go) via a simple bootstrap server configuration change, powering offline feature computation while in-session signals bypass storage entirely during real-time inference. Engineers evaluating streaming ingestion options can follow daily.dev for more lakehouse pipeline patterns."}},{"@type":"Question","name":"How do recommendation systems handle cold-start problems for new users and new products?","acceptedAnswer":{"@type":"Answer","text":"New users get a default embedding built from demographic signals like location, device type, and sign-up context, placing them in a behavioral cluster that converges as they interact. New products get an embedding from attributes like title, category, brand, price, and image features, inheriting scores from nearest neighbors in vector space, with full surfacing at the next daily batch cycle. Teams tackling cold-start challenges in recommender systems can find similar deep dives on daily.dev."}}]} ```