--- title: Voice agents with Realtime Video — Sidney Primas, LemonSlice description: Lemon Slice CTO Sidney Primas describes the company's approach to building photorealistic, real-time AI avatars using human-focused video diffusion transformer... canonical: https://daily.dev/posts/voice-agents-with-realtime-video-sidney-primas-lemonslice-z9cfnhlgp twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Voice agents with Realtime Video — Sidney Primas, LemonSlice | daily.dev og:description: Lemon Slice CTO Sidney Primas describes the company's approach to building photorealistic, real-time AI avatars using human-focused video diffusion transformer... og:url: https://daily.dev/posts/voice-agents-with-realtime-video-sidney-primas-lemonslice-z9cfnhlgp og:image: https://api.daily.dev/og/posts/Z9cfnHLgP.png og:image:alt: Voice agents with Realtime Video — Sidney Primas, LemonSlice og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Voice agents with Realtime Video — Sidney Primas, LemonSlice **[AI Engineer](https://daily.dev/sources/aidotengineer)** · 26 min read · 0 upvotes · 0 comments ## Summary Lemon Slice CTO Sidney Primas describes the company's approach to building photorealistic, real-time AI avatars using human-focused video diffusion transformer models. He details technical challenges including making bidirectional video models causal for real-time use, reducing denoising steps from 30 to 1 for speed, solving error accumulation in long-running video generation (avatars running 8-16 hours continuously), hardware cost optimization to match voice model pricing, and system orchestration across GPU/CPU. He showcases a Microsoft partnership recreating Teddy Roosevelt in a virtual Oval Office, discusses an upcoming emotion engine for more natural avatar reactions, and predicts an eventual single end-to-end model handling both video/audio input and output with separate EQ and IQ layers within two to three years. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do real-time AI video avatar models prevent error accumulation during long generation sessions? Error accumulation happens because causal video models can only look backward at previously generated frames, and each generated block carries some error that compounds over time when referenced again. Lemon Slice developed a proprietary method to suppress this, enabling avatars to generate continuously for 8 to 16 hours without noticeable degradation, a problem common to real-time video and world models generally. _Anyone evaluating real-time generative video approaches can follow architecture breakdowns like this on daily.dev._ ### How many denoising steps do real-time AI video generation models use compared to standard diffusion video models? Standard diffusion video models typically use around 30 denoising steps to produce high-quality output, but real-time avatar systems compress this down to a single step, going directly from pure noise to a finished video frame. This step reduction, combined with a causal attention mask that restricts the model to only see past frames, is what makes live, interactive video generation possible. _Developers tracking speed tradeoffs in diffusion versus real-time video pipelines can find deep dives like this on daily.dev._ ### Why is audio encoding important for generating realistic facial expressions in AI video avatars? Most audio encoders are trained on audiobook data, which is monotone and lacks emotional range, so they produce flat, unexpressive facial animation when used to drive video avatars. Building an expressive avatar model requires custom audio embeddings trained on emotionally rich audio data, which directly improves how well the video model reproduces natural emotions and micro-expressions. _Teams building expressive voice-to-video pipelines can keep up with techniques like this on daily.dev._ ## Similar posts on daily.dev - [Show HN: LemonSlice – Upgrade your voice agents to real-time video](https://daily.dev/posts/show-hn-lemonslice-upgrade-your-voice-agents-to-real-time-video-va0ykwv95) · Hacker News · 0 upvotes · 0 comments - [I created an interactive digital avatar of myself — and you can talk to it](https://daily.dev/posts/i-created-an-interactive-digital-avatar-of-myself-and-you-can-talk-to-it-trznf7yu1) · TechCrunch · 0 upvotes · 0 comments --- Tags: [#genai](https://daily.dev/tags/genai) [View this post on daily.dev](https://daily.dev/posts/voice-agents-with-realtime-video-sidney-primas-lemonslice-z9cfnhlgp) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Voice agents with Realtime Video — Sidney Primas, LemonSlice","url":"https://daily.dev/posts/voice-agents-with-realtime-video-sidney-primas-lemonslice-z9cfnhlgp","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/voice-agents-with-realtime-video-sidney-primas-lemonslice-z9cfnhlgp"},"datePublished":"2026-08-18T16:20:27.951Z","dateModified":"2026-08-18T16:51:40.440Z","description":"Lemon Slice CTO Sidney Primas describes the company's approach to building photorealistic, real-time AI avatars using human-focused video diffusion transformer...","image":"https://i.ytimg.com/vi/z1dqv74SpUs/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/z1dqv74SpUs/sddefault.jpg","isAccessibleForFree":true,"articleSection":"AI Engineer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"AI Engineer","logo":"https://media.daily.dev/image/upload/s--u5PucxNT--/f_auto/v1724338940/logos/aidotengineer","url":"https://daily.dev/sources/aidotengineer"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/voice-agents-with-realtime-video-sidney-primas-lemonslice-z9cfnhlgp","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"genai","timeRequired":"PT26M","video":{"@type":"VideoObject","name":"Voice agents with Realtime Video — Sidney Primas, LemonSlice","description":"Lemon Slice CTO Sidney Primas describes the company's approach to building photorealistic, real-time AI avatars using human-focused video diffusion transformer...","thumbnailUrl":"https://i.ytimg.com/vi/z1dqv74SpUs/sddefault.jpg","uploadDate":"2026-08-18T16:20:27.951Z","duration":"PT26M","url":"https://api.daily.dev/r/Z9cfnHLgP","embedUrl":"https://www.youtube.com/embed/z1dqv74SpUs"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"AI Engineer","item":"https://daily.dev/sources/aidotengineer"},{"@type":"ListItem","position":3,"name":"Voice agents with Realtime Video — Sidney Primas, LemonSlice"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/voice-agents-with-realtime-video-sidney-primas-lemonslice-z9cfnhlgp#faq","mainEntity":[{"@type":"Question","name":"How do real-time AI video avatar models prevent error accumulation during long generation sessions?","acceptedAnswer":{"@type":"Answer","text":"Error accumulation happens because causal video models can only look backward at previously generated frames, and each generated block carries some error that compounds over time when referenced again. Lemon Slice developed a proprietary method to suppress this, enabling avatars to generate continuously for 8 to 16 hours without noticeable degradation, a problem common to real-time video and world models generally. Anyone evaluating real-time generative video approaches can follow architecture breakdowns like this on daily.dev."}},{"@type":"Question","name":"How many denoising steps do real-time AI video generation models use compared to standard diffusion video models?","acceptedAnswer":{"@type":"Answer","text":"Standard diffusion video models typically use around 30 denoising steps to produce high-quality output, but real-time avatar systems compress this down to a single step, going directly from pure noise to a finished video frame. This step reduction, combined with a causal attention mask that restricts the model to only see past frames, is what makes live, interactive video generation possible. Developers tracking speed tradeoffs in diffusion versus real-time video pipelines can find deep dives like this on daily.dev."}},{"@type":"Question","name":"Why is audio encoding important for generating realistic facial expressions in AI video avatars?","acceptedAnswer":{"@type":"Answer","text":"Most audio encoders are trained on audiobook data, which is monotone and lacks emotional range, so they produce flat, unexpressive facial animation when used to drive video avatars. Building an expressive avatar model requires custom audio embeddings trained on emotionally rich audio data, which directly improves how well the video model reproduces natural emotions and micro-expressions. Teams building expressive voice-to-video pipelines can keep up with techniques like this on daily.dev."}}]} ```