--- title: Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples description: Do Inference Now (DIN) Deploy is an open-source collection of C++ samples that pairs ONNX Runtime with the NVIDIA TensorRT RTX execution provider, letting... canonical: https://daily.dev/posts/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples-aufd3cqlp twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples | daily.dev og:description: Do Inference Now (DIN) Deploy is an open-source collection of C++ samples that pairs ONNX Runtime with the NVIDIA TensorRT RTX execution provider, letting... og:url: https://daily.dev/posts/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples-aufd3cqlp og:image: https://api.daily.dev/og/posts/AUfd3cQLP.png og:image:alt: Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples **[NVIDIA Developer](https://daily.dev/sources/nvidiadev)** · 3 min read · 1 upvotes · 0 comments ## Summary Do Inference Now (DIN) Deploy is an open-source collection of C++ samples that pairs ONNX Runtime with the NVIDIA TensorRT RTX execution provider, letting developers go from a Hugging Face model checkpoint to a hardware-accelerated native application on Windows and Linux (including Arm64). Each sample uses a Python exporter to produce an ONNX artifact and a shared C++ CLI built on ONNX Runtime APIs, keeping vendor-specific CUDA code isolated to optional accelerated paths. Supported workloads include offline and streaming automatic speech recognition (Whisper, Parakeet TDT, Nemotron ASR Streaming), interactive video/image segmentation (Meta SAM 2.1), and prompt-driven image generation (FLUX.2-klein-4B) with Vulkan/DirectX graphics interop and post-training quantization via NVIDIA Model Optimizer. Benchmarks on DGX Spark show GPU speedups ranging from roughly 39x to over 200x real-time for ASR models and 38.3 FPS versus 0.5 FPS for SAM 2.1 segmentation compared to CPU. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is Do Inference Now (DIN) Deploy and what does it use for GPU acceleration? DIN Deploy is an open-source collection of C++ samples that combines ONNX Runtime with the NVIDIA TensorRT RTX execution provider to deploy AI models in native applications on Windows and Linux, including Arm64 builds. Each sample has a Python exporter converting a Hugging Face checkpoint to ONNX, paired with a native C++ CLI built on ONNX Runtime APIs. _Developers evaluating local AI deployment stacks can follow coverage of tools like DIN Deploy on daily.dev._ ### What AI tasks do the DIN Deploy sample pipelines support? DIN Deploy supports offline and streaming automatic speech recognition using OpenAI Whisper, NVIDIA Parakeet TDT, and NVIDIA Nemotron ASR Streaming; interactive image and video segmentation using Meta SAM 2.1; and prompt-driven image generation using FLUX.2-klein-4B with Vulkan and DirectX graphics interop plus post-training quantization support. _Those comparing on-device AI pipelines for speech, vision, and image generation can track updates like this on daily.dev._ ### How much faster is GPU inference than CPU for Whisper and SAM 2.1 on DGX Spark using TensorRT RTX? On DGX Spark, GPU acceleration runs openai/whisper-large-v3-turbo at 58.5x real-time versus 3.8x on CPU, and facebook/sam2.1-hiera-base-plus segmentation at 38.3 FPS on GPU versus 0.5 FPS on CPU. Parakeet TDT 0.6b-v3 reaches 206.41x real-time on GPU compared to 14.44x on CPU. _Engineers benchmarking GPU versus CPU inference can keep an eye on figures like these via daily.dev._ ## Similar posts on daily.dev - [Deploy an Open Model from Checkpoint to Inference in Two Commands with NVIDIA TensorRT Model Connect](https://daily.dev/posts/deploy-an-open-model-from-checkpoint-to-inference-in-two-commands-with-nvidia-tensorrt-model-connect-orlt6lexe) · NVIDIA Developer · 1 upvotes · 0 comments - [Adaptive Inference in NVIDIA TensorRT for RTX Enables Automatic Optimization](https://daily.dev/posts/adaptive-inference-in-nvidia-tensorrt-for-rtx-enables-automatic-optimization-pmsq81agb) · NVIDIA Developer · 0 upvotes · 0 comments - [LiteRT: The Universal Framework for On-Device AI](https://daily.dev/posts/litert-the-universal-framework-for-on-device-ai-dkvzmc8hv) · Google Developers · 0 upvotes · 0 comments - [NVIDIA Levels Up Local AI Agents Across RTX PCs and DGX Spark](https://daily.dev/posts/nvidia-levels-up-local-ai-agents-across-rtx-pcs-and-dgx-spark-du5cveehq) · NVIDIA · 0 upvotes · 0 comments - [Introducing DigitalOcean AI-Native Cloud for Production AI Workloads](https://daily.dev/posts/introducing-digitalocean-ai-native-cloud-for-production-ai-workloads-z3jcrd01y) · DigitalOcean · 0 upvotes · 0 comments --- Tags: [#nvidia](https://daily.dev/tags/nvidia), [#c++](https://daily.dev/tags/c++), [#tensorrt](https://daily.dev/tags/tensorrt) [View this post on daily.dev](https://daily.dev/posts/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples-aufd3cqlp) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples","url":"https://daily.dev/posts/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples-aufd3cqlp","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples-aufd3cqlp"},"datePublished":"2026-10-01T18:01:26.341Z","dateModified":"2026-10-01T18:09:15.306Z","description":"Do Inference Now (DIN) Deploy is an open-source collection of C++ samples that pairs ONNX Runtime with the NVIDIA TensorRT RTX execution provider, letting...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/743c3a138c95e8069a354f54f874d14f?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/743c3a138c95e8069a354f54f874d14f?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"NVIDIA Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"NVIDIA Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/86e45aab42ba48ce83103d01b1119910","url":"https://daily.dev/sources/nvidiadev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples-aufd3cqlp","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"nvidia,c++,tensorrt","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"NVIDIA Developer","item":"https://daily.dev/sources/nvidiadev"},{"@type":"ListItem","position":3,"name":"Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/build-local-ai-apps-with-c-and-nvidia-tensorrt-rtx-samples-aufd3cqlp#faq","mainEntity":[{"@type":"Question","name":"What is Do Inference Now (DIN) Deploy and what does it use for GPU acceleration?","acceptedAnswer":{"@type":"Answer","text":"DIN Deploy is an open-source collection of C++ samples that combines ONNX Runtime with the NVIDIA TensorRT RTX execution provider to deploy AI models in native applications on Windows and Linux, including Arm64 builds. Each sample has a Python exporter converting a Hugging Face checkpoint to ONNX, paired with a native C++ CLI built on ONNX Runtime APIs. Developers evaluating local AI deployment stacks can follow coverage of tools like DIN Deploy on daily.dev."}},{"@type":"Question","name":"What AI tasks do the DIN Deploy sample pipelines support?","acceptedAnswer":{"@type":"Answer","text":"DIN Deploy supports offline and streaming automatic speech recognition using OpenAI Whisper, NVIDIA Parakeet TDT, and NVIDIA Nemotron ASR Streaming; interactive image and video segmentation using Meta SAM 2.1; and prompt-driven image generation using FLUX.2-klein-4B with Vulkan and DirectX graphics interop plus post-training quantization support. Those comparing on-device AI pipelines for speech, vision, and image generation can track updates like this on daily.dev."}},{"@type":"Question","name":"How much faster is GPU inference than CPU for Whisper and SAM 2.1 on DGX Spark using TensorRT RTX?","acceptedAnswer":{"@type":"Answer","text":"On DGX Spark, GPU acceleration runs openai/whisper-large-v3-turbo at 58.5x real-time versus 3.8x on CPU, and facebook/sam2.1-hiera-base-plus segmentation at 38.3 FPS on GPU versus 0.5 FPS on CPU. Parakeet TDT 0.6b-v3 reaches 206.41x real-time on GPU compared to 14.44x on CPU. Engineers benchmarking GPU versus CPU inference can keep an eye on figures like these via daily.dev."}}]} ```