--- title: Build a real-time hotel booking voice agent | daily.dev description: A hands-on newsletter walks through building a real-time hotel booking voice agent using Speechmatics Linden for speech-to-text, LiveKit for the real-time room... canonical: https://daily.dev/posts/build-a-real-time-hotel-booking-voice-agent-ntorwqf8n twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Build a real-time hotel booking voice agent | daily.dev og:description: A hands-on newsletter walks through building a real-time hotel booking voice agent using Speechmatics Linden for speech-to-text, LiveKit for the real-time room... og:url: https://daily.dev/posts/build-a-real-time-hotel-booking-voice-agent-ntorwqf8n og:image: https://api.daily.dev/og/posts/NTORWQF8n.png og:image:alt: Build a real-time hotel booking voice agent og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Build a real-time hotel booking voice agent **[Daily Dose of Data Science \| Avi Chawla \| Substack](https://daily.dev/sources/dailydoseofds)** · 7 min read · 1 upvotes · 0 comments ## Summary A hands-on newsletter walks through building a real-time hotel booking voice agent using Speechmatics Linden for speech-to-text, LiveKit for the real-time room and media routing, OpenRouter for LLM reasoning, and Fish Audio for text-to-speech, rendered in a Streamlit interface. It shows partial and final transcripts plus latency measurements between the final transcript and agent speech, and explains how the pipeline preserves mid-conversation corrections like changing a booking date. The issue also covers a vLLM feature request for multi-model serving on one GPU that was closed as not planned, contrasted with a project called SIE that cut a four-model pipeline's runtime from 18.58s to 1.47s, plus a separate explainer on how GPU VRAM is divided among model weights, KV cache, activations, and runtime overhead during LLM inference. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I build a real-time voice agent that handles hotel booking with speech-to-text and text-to-speech? Combine LiveKit for the real-time audio room, Speechmatics Linden for speech-to-text with partial and final transcripts, OpenRouter for LLM reasoning (e.g. openai/gpt-4.1-mini), and Fish Audio's s2.1-pro model for text-to-speech. LiveKit Inference handles authentication, routing, and billing for the STT and TTS providers, so only a LiveKit project and OpenRouter key are required as credentials. _daily.dev surfaces practical build guides like this for engineers wiring together voice agent pipelines._ ### How fast can Speechmatics Linden return a final transcript after a turn ends? Speechmatics Linden's ForceEndOfUtterance flow can return a final transcript in roughly 250 milliseconds after the client signals a turn has ended. It is built specifically for voice agents rather than offline transcription, targeting strong accents, non-native speech, noisy audio, and alphanumeric strings across more than 55 languages. _engineers benchmarking speech latency for voice agents can track findings like this on daily.dev._ ### Why did vLLM close the feature request to run multiple small models on one GPU through a single inference server? vLLM closed the request as not planned, leaving the documented workaround of running a separate vLLM instance per model with a routing layer between them, which gives each model its own process, memory allocation, and lifecycle. A community project called SIE instead kept four models (embedder, reranker, entity extractor, generator) behind one inference server, completing a concurrent test in 1.47 seconds versus 18.58 seconds when run directly. _developers weighing GPU multi-model serving tradeoffs follow discussions like this on daily.dev._ ## Similar posts on daily.dev - [Real-time voice AI low-latency techniques that work](https://daily.dev/posts/real-time-voice-ai-low-latency-techniques-that-work-noffnjl2b) · Netguru · 0 upvotes · 0 comments - [Real-Time Voice AI Stack for Agents: Architecture Guide](https://daily.dev/posts/real-time-voice-ai-stack-for-agents-architecture-guide-mjgl3tbs7) · Deepgram · 0 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#speech-recognition](https://daily.dev/tags/speech-recognition), [#voice-ai](https://daily.dev/tags/voice-ai), [#real-time-systems](https://daily.dev/tags/real-time-systems) [View this post on daily.dev](https://daily.dev/posts/build-a-real-time-hotel-booking-voice-agent-ntorwqf8n) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Build a real-time hotel booking voice agent","url":"https://daily.dev/posts/build-a-real-time-hotel-booking-voice-agent-ntorwqf8n","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/build-a-real-time-hotel-booking-voice-agent-ntorwqf8n"},"datePublished":"2026-09-29T21:17:59.467Z","dateModified":"2026-09-29T21:18:33.674Z","description":"A hands-on newsletter walks through building a real-time hotel booking voice agent using Speechmatics Linden for speech-to-text, LiveKit for the real-time room...","image":"https://substackcdn.com/image/fetch/$s_!nkXy!,w_1200,h_675,c_fill,f_jpg,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdc52738e-a074-4582-8dc0-6c2b8b8f1ceb_960x675.gif","thumbnailUrl":"https://substackcdn.com/image/fetch/$s_!nkXy!,w_1200,h_675,c_fill,f_jpg,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdc52738e-a074-4582-8dc0-6c2b8b8f1ceb_960x675.gif","isAccessibleForFree":true,"articleSection":"Daily Dose of Data Science | Avi Chawla | Substack","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Daily Dose of Data Science | Avi Chawla | Substack","logo":"https://media.daily.dev/image/upload/s--4IHQgTOw--/f_auto/v1710503712/logos/dailydoseofds","url":"https://daily.dev/sources/dailydoseofds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/build-a-real-time-hotel-booking-voice-agent-ntorwqf8n","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,speech-recognition,voice-ai,real-time-systems","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Daily Dose of Data Science | Avi Chawla | Substack","item":"https://daily.dev/sources/dailydoseofds"},{"@type":"ListItem","position":3,"name":"Build a real-time hotel booking voice agent"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/build-a-real-time-hotel-booking-voice-agent-ntorwqf8n#faq","mainEntity":[{"@type":"Question","name":"How can I build a real-time voice agent that handles hotel booking with speech-to-text and text-to-speech?","acceptedAnswer":{"@type":"Answer","text":"Combine LiveKit for the real-time audio room, Speechmatics Linden for speech-to-text with partial and final transcripts, OpenRouter for LLM reasoning (e.g. openai/gpt-4.1-mini), and Fish Audio's s2.1-pro model for text-to-speech. LiveKit Inference handles authentication, routing, and billing for the STT and TTS providers, so only a LiveKit project and OpenRouter key are required as credentials. daily.dev surfaces practical build guides like this for engineers wiring together voice agent pipelines."}},{"@type":"Question","name":"How fast can Speechmatics Linden return a final transcript after a turn ends?","acceptedAnswer":{"@type":"Answer","text":"Speechmatics Linden's ForceEndOfUtterance flow can return a final transcript in roughly 250 milliseconds after the client signals a turn has ended. It is built specifically for voice agents rather than offline transcription, targeting strong accents, non-native speech, noisy audio, and alphanumeric strings across more than 55 languages. engineers benchmarking speech latency for voice agents can track findings like this on daily.dev."}},{"@type":"Question","name":"Why did vLLM close the feature request to run multiple small models on one GPU through a single inference server?","acceptedAnswer":{"@type":"Answer","text":"vLLM closed the request as not planned, leaving the documented workaround of running a separate vLLM instance per model with a routing layer between them, which gives each model its own process, memory allocation, and lifecycle. A community project called SIE instead kept four models (embedder, reranker, entity extractor, generator) behind one inference server, completing a concurrent test in 1.47 seconds versus 18.58 seconds when run directly. developers weighing GPU multi-model serving tradeoffs follow discussions like this on daily.dev."}}]} ```