--- title: Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo description: Pinterest built a vision-language model (VLM) serving stack on NVIDIA Blackwell B200 GPUs and NVIDIA Dynamo to power Pinterest Assistant and other multimodal... canonical: https://daily.dev/posts/building-pinterest-s-vlm-serving-stack-on-nvidia-dynamo-fui5s91v1 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo | daily.dev og:description: Pinterest built a vision-language model (VLM) serving stack on NVIDIA Blackwell B200 GPUs and NVIDIA Dynamo to power Pinterest Assistant and other multimodal... og:url: https://daily.dev/posts/building-pinterest-s-vlm-serving-stack-on-nvidia-dynamo-fui5s91v1 og:image: https://api.daily.dev/og/posts/FuI5S91v1.png og:image:alt: Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo **[Pinterest Engineering](https://daily.dev/sources/pint)** · 18 min read · 6 upvotes · 0 comments ## Summary Pinterest built a vision-language model (VLM) serving stack on NVIDIA Blackwell B200 GPUs and NVIDIA Dynamo to power Pinterest Assistant and other multimodal products. The stack tackles VLM-specific challenges like expensive prefill, multimodal payloads, and KV cache pressure using disaggregated prefill/decode/encode serving, LMCache-based KV offloading, and multimodal KV-aware routing built with the Dynamo team. A key innovation is projection embeddings via PinCLIP, which let requests send precomputed visual embeddings instead of raw images, yielding up to 369x faster time-to-first-token and 44x faster end-to-end latency versus pixel-based inputs, while supporting 25x more visual context at comparable latency. The platform runs on PinCompute (AWS EKS) with vLLM as the inference engine, uses AIPerf for DAG-based benchmarking of agentic multimodal workloads, and now serves as a shared foundation for reranking, OCR, safety guardrails, and other GenAI use cases at Pinterest, with LoRA hot loading and Dynamo's AI Configurator and Planner planned as next steps. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much faster is TTFT when using precomputed visual embeddings instead of raw images for VLM serving? Using precomputed PinCLIP embeddings instead of raw pixel-based images yields roughly 85x faster average time-to-first-token (TTFT) and up to 369x faster at peak, compared to pixel-based image inputs in NVIDIA Dynamo. End-to-end latency also improves by 7.3x on average (44x at peak), and TPOT improves 1.1x average (2.6x peak). Requests with 250 images as embeddings reached latencies comparable to pixel-based requests with only 10 images. _Teams weighing embedding-based versus pixel-based multimodal serving track real-world benchmarks like these on daily.dev._ ### Why is prefill more expensive than decode when serving vision-language models compared to text-only LLMs? Prefill dominates VLM serving cost because encoding visual context is computationally expensive, unlike text-only serving where decode is typically the bottleneck. Requests with many images produce much larger and more irregular KV caches, requiring KV-aware routing, cache offloading, and careful prompt design to stay within latency and memory service-level objectives, whereas text-only prompt lengths are relatively uniform and predictable. _Engineers designing multimodal inference pipelines follow prefill and KV cache strategies like these on daily.dev._ ### Why did Pinterest choose NVIDIA Dynamo over other Kubernetes-native inference serving frameworks? Pinterest selected Dynamo because it is Kubernetes-native and compatible with its Kubernetes and service discovery setup, inference-engine agnostic, offers flexible traffic management, and uses a performant Rust-based router. Earlier evaluated Kubernetes-native frameworks were easy to start with but lacked traffic-management flexibility or locked teams into a single inference ecosystem, which Dynamo avoided. _Teams evaluating inference orchestration frameworks compare tradeoffs like these on daily.dev._ ## Similar posts on daily.dev - [Deploy vLLM NVIDIA Dynamo Inference: Production Guide](https://daily.dev/posts/deploy-vllm-nvidia-dynamo-inference-production-guide-kpnabrf9j) · SitePoint · 0 upvotes · 0 comments - [Why vLLM is the best choice for AI inference today](https://daily.dev/posts/why-vllm-is-the-best-choice-for-ai-inference-today-7zxdzzpx7) · Red Hat Developer · 0 upvotes · 0 comments - [NVIDIA Dynamo Addresses Multi-Node LLM Inference Challenges](https://daily.dev/posts/nvidia-dynamo-addresses-multi-node-llm-inference-challenges-yz7e3pwz7) · InfoQ · 0 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#vllm](https://daily.dev/tags/vllm), [#vlm](https://daily.dev/tags/vlm) [View this post on daily.dev](https://daily.dev/posts/building-pinterest-s-vlm-serving-stack-on-nvidia-dynamo-fui5s91v1) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo","url":"https://daily.dev/posts/building-pinterest-s-vlm-serving-stack-on-nvidia-dynamo-fui5s91v1","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/building-pinterest-s-vlm-serving-stack-on-nvidia-dynamo-fui5s91v1"},"datePublished":"2026-09-10T23:58:13.762Z","dateModified":"2026-09-11T00:18:40.418Z","description":"Pinterest built a vision-language model (VLM) serving stack on NVIDIA Blackwell B200 GPUs and NVIDIA Dynamo to power Pinterest Assistant and other multimodal...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b887ce0e2e9d6aee084d21faa367fe35?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b887ce0e2e9d6aee084d21faa367fe35?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Pinterest Engineering","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Pinterest Engineering","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/f0dc15da4f9c44fdbc5770106dd3d0a9","url":"https://daily.dev/sources/pint"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/building-pinterest-s-vlm-serving-stack-on-nvidia-dynamo-fui5s91v1","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":6},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"kubernetes,vllm,vlm","timeRequired":"PT18M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Pinterest Engineering","item":"https://daily.dev/sources/pint"},{"@type":"ListItem","position":3,"name":"Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/building-pinterest-s-vlm-serving-stack-on-nvidia-dynamo-fui5s91v1#faq","mainEntity":[{"@type":"Question","name":"How much faster is TTFT when using precomputed visual embeddings instead of raw images for VLM serving?","acceptedAnswer":{"@type":"Answer","text":"Using precomputed PinCLIP embeddings instead of raw pixel-based images yields roughly 85x faster average time-to-first-token (TTFT) and up to 369x faster at peak, compared to pixel-based image inputs in NVIDIA Dynamo. End-to-end latency also improves by 7.3x on average (44x at peak), and TPOT improves 1.1x average (2.6x peak). Requests with 250 images as embeddings reached latencies comparable to pixel-based requests with only 10 images. Teams weighing embedding-based versus pixel-based multimodal serving track real-world benchmarks like these on daily.dev."}},{"@type":"Question","name":"Why is prefill more expensive than decode when serving vision-language models compared to text-only LLMs?","acceptedAnswer":{"@type":"Answer","text":"Prefill dominates VLM serving cost because encoding visual context is computationally expensive, unlike text-only serving where decode is typically the bottleneck. Requests with many images produce much larger and more irregular KV caches, requiring KV-aware routing, cache offloading, and careful prompt design to stay within latency and memory service-level objectives, whereas text-only prompt lengths are relatively uniform and predictable. Engineers designing multimodal inference pipelines follow prefill and KV cache strategies like these on daily.dev."}},{"@type":"Question","name":"Why did Pinterest choose NVIDIA Dynamo over other Kubernetes-native inference serving frameworks?","acceptedAnswer":{"@type":"Answer","text":"Pinterest selected Dynamo because it is Kubernetes-native and compatible with its Kubernetes and service discovery setup, inference-engine agnostic, offers flexible traffic management, and uses a performant Rust-based router. Earlier evaluated Kubernetes-native frameworks were easy to start with but lacked traffic-management flexibility or locked teams into a single inference ecosystem, which Dynamo avoided. Teams evaluating inference orchestration frameworks compare tradeoffs like these on daily.dev."}}]} ```