--- title: The 200ms latency: A developer’s guide to real-time... description: Building real-time personalization systems that stay under the 200ms latency threshold requires deliberate architectural choices. A two-tower approach... canonical: https://daily.dev/posts/the-200ms-latency-a-developer-s-guide-to-real-time-personalization-u7s5bwkwz twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The 200ms latency: A developer’s guide to real-time personalization | daily.dev og:description: Building real-time personalization systems that stay under the 200ms latency threshold requires deliberate architectural choices. A two-tower approach... og:url: https://daily.dev/posts/the-200ms-latency-a-developer-s-guide-to-real-time-personalization-u7s5bwkwz og:image: https://api.daily.dev/og/posts/u7S5BWKwz.png og:image:alt: The 200ms latency: A developer’s guide to real-time personalization og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The 200ms latency: A developer’s guide to real-time personalization **[InfoWorld](https://daily.dev/sources/infoworld)** · 8 min read · 0 upvotes · 0 comments ## Summary Building real-time personalization systems that stay under the 200ms latency threshold requires deliberate architectural choices. A two-tower approach separates fast candidate retrieval (under 20ms using vector search and HNSW indexing) from heavier AI ranking, avoiding the impossibility of scoring entire catalogs per request. Cold-start problems are addressed with session vectors and nearest-neighbor lookups rather than full history aggregation. A head/tail decision matrix determines what gets pre-computed and cached in Redis/DynamoDB versus handled by just-in-time inference. Model quantization (FP32 to INT8/INT4) can double inference speed with under 0.5% accuracy loss. Circuit breakers with hard timeouts ensure graceful degradation to cached popular content when models are slow. Data contracts using Protobuf or Avro prevent schema drift from poisoning pipelines. Observability should focus on p99/p99.9 latency rather than averages to protect power users. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Why real-time AI at scale is so hard](https://daily.dev/posts/why-real-time-ai-at-scale-is-so-hard-5xrkdsob1) · The New Stack · 1 upvotes · 0 comments - [Teaching a child in <1000 ms: the architecture behind a real-time tutor](https://daily.dev/posts/teaching-a-child-in-1000-ms-the-architecture-behind-a-real-time-tutor-8t3zegwa2) · Hacker News · 1 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#architecture](https://daily.dev/tags/architecture), [#ecommerce](https://daily.dev/tags/ecommerce), [#vector-search](https://daily.dev/tags/vector-search) [View this post on daily.dev](https://daily.dev/posts/the-200ms-latency-a-developer-s-guide-to-real-time-personalization-u7s5bwkwz) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The 200ms latency: A developer’s guide to real-time personalization","url":"https://daily.dev/posts/the-200ms-latency-a-developer-s-guide-to-real-time-personalization-u7s5bwkwz","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-200ms-latency-a-developer-s-guide-to-real-time-personalization-u7s5bwkwz"},"datePublished":"2026-02-19T10:08:18.072Z","dateModified":"2026-02-19T10:08:48.714Z","description":"Building real-time personalization systems that stay under the 200ms latency threshold requires deliberate architectural choices. A two-tower approach...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b713c103182b77bd4c638adc5a3e51f7?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b713c103182b77bd4c638adc5a3e51f7?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"InfoWorld","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"InfoWorld","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/bf6d68a999064029b0bb09aa6268f1f3","url":"https://daily.dev/sources/infoworld"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/the-200ms-latency-a-developer-s-guide-to-real-time-personalization-u7s5bwkwz","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,architecture,ecommerce,vector-search","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"InfoWorld","item":"https://daily.dev/sources/infoworld"},{"@type":"ListItem","position":3,"name":"The 200ms latency: A developer’s guide to real-time personalization"}]} ```