--- title: What real time-to-first-token looks like on DigitalOcean... description: A methodology-driven latency audit measured time-to-first-token (TTFT) across six LLMs (Gemma 4, GLM-5.3, Kimi K2.6, DeepSeek V3.2, DeepSeek V4.1 Flash, Llama... canonical: https://daily.dev/posts/what-real-time-to-first-token-looks-like-on-digitalocean-serverless-inference-rxonlv5jp twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: What real time-to-first-token looks like on DigitalOcean Serverless Inference | daily.dev og:description: A methodology-driven latency audit measured time-to-first-token (TTFT) across six LLMs (Gemma 4, GLM-5.3, Kimi K2.6, DeepSeek V3.2, DeepSeek V4.1 Flash, Llama... og:url: https://daily.dev/posts/what-real-time-to-first-token-looks-like-on-digitalocean-serverless-inference-rxonlv5jp og:image: https://api.daily.dev/og/posts/RxOnlv5jP.png og:image:alt: What real time-to-first-token looks like on DigitalOcean Serverless Inference og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # What real time-to-first-token looks like on DigitalOcean Serverless Inference **[DigitalOcean Community](https://daily.dev/sources/do_community)** · 11 min read · 0 upvotes · 0 comments ## Summary A methodology-driven latency audit measured time-to-first-token (TTFT) across six LLMs (Gemma 4, GLM-5.3, Kimi K2.6, DeepSeek V3.2, DeepSeek V4.1 Flash, Llama 4 Maverick) on DigitalOcean's Serverless Inference, running 380 cold, uncached calls across short, medium, and long prompts. Most models returned a first token in under a second for short and medium prompts. DeepSeek V3.2 was the slowest on average and twice produced extreme outlier delays (186.9s and 78.3s) with no comparable issue on other models. Gemma 4, despite being the smallest and fastest model on short prompts, degraded the most on long prompts, jumping from 0.93s to 3.35s median TTFT. The piece also details a methodology bug where reasoning-token streams were missed, causing 127 of 390 initial rows to have no TTFT reading, and shares the Python code used for prompt randomization, stream timing, and token-exact prompt trimming via binary search. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why did some of my time-to-first-token measurements come back empty when testing reasoning models like Kimi K2.6 or DeepSeek V4.1 Flash? Reasoning models like Kimi K2.6 and DeepSeek V4.1 Flash stream their internal reasoning text before the actual answer, in a separate field. If a script only watches the answer content field for the first token and the model uses its entire output budget on reasoning, no token is ever recorded. The fix is checking both the content field and the reasoning_content field and taking whichever appears first. _Developers tuning streaming latency code for reasoning models can compare real-world measurement approaches on daily.dev._ ### How can I make sure my LLM latency benchmark isn't accidentally hitting a prompt cache and showing falsely fast results? Add a random, unique marker (such as a UUID) to the start of every prompt before sending it. DigitalOcean's prompt cache matches on exact prefixes, so a repeated prompt can return a suspiciously fast cached response that has nothing to do with real serving speed. Forcing every call to contain unique text guarantees a genuinely cold, uncached measurement. _Anyone benchmarking inference latency can track techniques like this for trustworthy results via daily.dev._ ### Does DeepSeek V3.2 have consistent response latency on DigitalOcean Serverless Inference, or can it have unpredictable slow responses? DeepSeek V3.2 had a 1.43 second median time-to-first-token on short prompts, climbing to 1.9 seconds median (4.1 seconds at the 90th percentile) on a full 10,000-token prompt. Out of 80 calls, two produced extreme outlier delays of 186.9 seconds and 78.3 seconds before the first token, with no comparable slowdowns on the other five models tested, so a timeout or fallback is worth building in for interactive use cases. _Teams choosing a model for latency-sensitive apps can weigh real outlier data like this on daily.dev before committing._ ## Similar posts on daily.dev - [Metrics that Matter with Serverless Inference](https://daily.dev/posts/metrics-that-matter-with-serverless-inference-9nynkhj1z) · DigitalOcean Community · 0 upvotes · 0 comments - [Local vs Cloud AI Coding: Latency, Privacy & Performance Guide](https://daily.dev/posts/local-vs-cloud-ai-coding-latency-privacy-performance-guide-ce3vygmeo) · SitePoint · 0 upvotes · 0 comments - [How I massively improved my AI inference performance without buying new hardware](https://daily.dev/posts/how-i-massively-improved-my-ai-inference-performance-without-buying-new-hardware-jgyj1odfm) · Red Hat Developer · 0 upvotes · 0 comments - [Making an AI Agent Feel Fast on Serverless Inference](https://daily.dev/posts/making-an-ai-agent-feel-fast-on-serverless-inference-a1vzenpo1) · DigitalOcean Community · 2 upvotes · 1 comments --- Tags: [#serverless](https://daily.dev/tags/serverless), [#ai-inference](https://daily.dev/tags/ai-inference), [#digitalocean](https://daily.dev/tags/digitalocean), [#deepseek](https://daily.dev/tags/deepseek) [View this post on daily.dev](https://daily.dev/posts/what-real-time-to-first-token-looks-like-on-digitalocean-serverless-inference-rxonlv5jp) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"What real time-to-first-token looks like on DigitalOcean Serverless Inference","url":"https://daily.dev/posts/what-real-time-to-first-token-looks-like-on-digitalocean-serverless-inference-rxonlv5jp","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/what-real-time-to-first-token-looks-like-on-digitalocean-serverless-inference-rxonlv5jp"},"datePublished":"2026-10-01T22:29:05.374Z","dateModified":"2026-10-01T22:29:30.283Z","description":"A methodology-driven latency audit measured time-to-first-token (TTFT) across six LLMs (Gemma 4, GLM-5.3, Kimi K2.6, DeepSeek V3.2, DeepSeek V4.1 Flash, Llama...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/befd6880076aac1110ce3ea6def4914f?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/befd6880076aac1110ce3ea6def4914f?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"DigitalOcean Community","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"DigitalOcean Community","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/c1b9d07730e34ea388c39a498a753d6c","url":"https://daily.dev/sources/do_community"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/what-real-time-to-first-token-looks-like-on-digitalocean-serverless-inference-rxonlv5jp","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"serverless,ai-inference,digitalocean,deepseek","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"DigitalOcean Community","item":"https://daily.dev/sources/do_community"},{"@type":"ListItem","position":3,"name":"What real time-to-first-token looks like on DigitalOcean Serverless Inference"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/what-real-time-to-first-token-looks-like-on-digitalocean-serverless-inference-rxonlv5jp#faq","mainEntity":[{"@type":"Question","name":"Why did some of my time-to-first-token measurements come back empty when testing reasoning models like Kimi K2.6 or DeepSeek V4.1 Flash?","acceptedAnswer":{"@type":"Answer","text":"Reasoning models like Kimi K2.6 and DeepSeek V4.1 Flash stream their internal reasoning text before the actual answer, in a separate field. If a script only watches the answer content field for the first token and the model uses its entire output budget on reasoning, no token is ever recorded. The fix is checking both the content field and the reasoning_content field and taking whichever appears first. Developers tuning streaming latency code for reasoning models can compare real-world measurement approaches on daily.dev."}},{"@type":"Question","name":"How can I make sure my LLM latency benchmark isn't accidentally hitting a prompt cache and showing falsely fast results?","acceptedAnswer":{"@type":"Answer","text":"Add a random, unique marker (such as a UUID) to the start of every prompt before sending it. DigitalOcean's prompt cache matches on exact prefixes, so a repeated prompt can return a suspiciously fast cached response that has nothing to do with real serving speed. Forcing every call to contain unique text guarantees a genuinely cold, uncached measurement. Anyone benchmarking inference latency can track techniques like this for trustworthy results via daily.dev."}},{"@type":"Question","name":"Does DeepSeek V3.2 have consistent response latency on DigitalOcean Serverless Inference, or can it have unpredictable slow responses?","acceptedAnswer":{"@type":"Answer","text":"DeepSeek V3.2 had a 1.43 second median time-to-first-token on short prompts, climbing to 1.9 seconds median (4.1 seconds at the 90th percentile) on a full 10,000-token prompt. Out of 80 calls, two produced extreme outlier delays of 186.9 seconds and 78.3 seconds before the first token, with no comparable slowdowns on the other five models tested, so a timeout or fallback is worth building in for interactive use cases. Teams choosing a model for latency-sensitive apps can weigh real outlier data like this on daily.dev before committing."}}]} ```