--- title: Run AI Models for Edge Application Development with Just... description: This tutorial shows how to run NVIDIA's Nemotron-3.5 Lightning model on a DGX Spark using the TensorRT Edge-LLM runtime and trtedge CLI. You install the... canonical: https://daily.dev/posts/run-ai-models-for-edge-application-development-with-just-a-few-commands-gjbppnyda twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Run AI Models for Edge Application Development with Just a Few Commands | daily.dev og:description: This tutorial shows how to run NVIDIA's Nemotron-3.5 Lightning model on a DGX Spark using the TensorRT Edge-LLM runtime and trtedge CLI. You install the... og:url: https://daily.dev/posts/run-ai-models-for-edge-application-development-with-just-a-few-commands-gjbppnyda og:image: https://api.daily.dev/og/posts/GjBppnyda.png og:image:alt: Run AI Models for Edge Application Development with Just a Few Commands og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Run AI Models for Edge Application Development with Just a Few Commands **[NVIDIA Developer](https://daily.dev/sources/nvidiadev)** · 6 min read · 0 upvotes · 0 comments ## Summary This tutorial shows how to run NVIDIA's Nemotron-3.5 Lightning model on a DGX Spark using the TensorRT Edge-LLM runtime and trtedge CLI. You install the tensorrt-edgellm package, run trtedge chat to test prompts, and use trtedge serve to expose an OpenAI-compatible API on port 8000. Enabling DSpark speculative decoding with three draft tokens raises mean generation speed from 73.4 to 126.9 tokens/s on HumanEval prompts in the measured tests, with peak memory use of 27.9GB on the 64GB DGX Spark. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I run NVIDIA Nemotron-3.5 Lightning on a DGX Spark? Install the tensorrt-edgellm package with pip in a virtual environment, then run 'trtedge chat nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4' to start a chat session. The CLI downloads the model (about 21GB) and builds a TensorRT engine automatically on first launch, caching files under ~/.cache/tensorrt-edgellm for reuse on later runs. _daily.dev surfaces edge-AI workflow guides like this for developers deploying LLMs on embedded hardware._ ### How much faster is speculative decoding with DSpark on DGX Spark? Enabling DSpark speculative decoding with three draft tokens increased mean generation speed from 73.4 to 126.9 tokens per second on HumanEval prompts on a DGX Spark 64GB. Time to first token stayed roughly the same across SpeedBench and HumanEval workloads, while peak system memory during DSpark inference reached 27.9GB. _benchmark-conscious developers track inference speedups like this on daily.dev before tuning their own deployments._ ### What are the requirements to use TensorRT Edge-LLM on DGX Spark? You need a DGX Spark with a supported NVIDIA software stack, CUDA 13, TensorRT 10.16.1.11 or newer within the 10.x series, and Python 3.10 to 3.12. The setup was tested with TensorRT Edge-LLM 0.11.1 and TensorRT 10.16.1.11, running on DGX OS 7.6.0 based on Ubuntu 24.04.4 LTS. _daily.dev helps edge developers keep track of exact version requirements before setting up new inference stacks._ ## Similar posts on daily.dev - [Scaling Autonomous AI Agents and Workloads with NVIDIA DGX Spark](https://daily.dev/posts/scaling-autonomous-ai-agents-and-workloads-with-nvidia-dgx-spark-lxoujr36b) · NVIDIA Developer · 1 upvotes · 0 comments - [NVIDIA Nemotron 3.5 Lightning · Ollama Blog](https://daily.dev/posts/nvidia-nemotron-3-5-lightning-ollama-blog-uy1kba1ur) · Ollama · 3 upvotes · 0 comments - [Nemotron 3.5 Lightning now available on AI Gateway for free](https://daily.dev/posts/nemotron-3-5-lightning-now-available-on-ai-gateway-for-free-2a3hhnsqr) · Vercel · 0 upvotes · 0 comments - [Databricks AI Runtime and NVIDIA make it easier to customize open models with enterprise data](https://daily.dev/posts/databricks-ai-runtime-and-nvidia-make-it-easier-to-customize-open-models-with-enterprise-data-em0w4v1yx) · databricks · 0 upvotes · 0 comments - [Run Local AI Agents with Faster Models and Multi-Node Clustering on NVIDIA DGX Spark](https://daily.dev/posts/run-local-ai-agents-with-faster-models-and-multi-node-clustering-on-nvidia-dgx-spark-moqgba8ql) · NVIDIA Developer · 1 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#edge-computing](https://daily.dev/tags/edge-computing), [#tensorrt](https://daily.dev/tags/tensorrt) [View this post on daily.dev](https://daily.dev/posts/run-ai-models-for-edge-application-development-with-just-a-few-commands-gjbppnyda) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Run AI Models for Edge Application Development with Just a Few Commands","url":"https://daily.dev/posts/run-ai-models-for-edge-application-development-with-just-a-few-commands-gjbppnyda","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/run-ai-models-for-edge-application-development-with-just-a-few-commands-gjbppnyda"},"datePublished":"2026-10-30T20:35:14.464Z","dateModified":"2026-10-30T20:35:41.941Z","description":"This tutorial shows how to run NVIDIA's Nemotron-3.5 Lightning model on a DGX Spark using the TensorRT Edge-LLM runtime and trtedge CLI. You install the...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f5707596c03bd5441179e0d36923646b?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f5707596c03bd5441179e0d36923646b?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"NVIDIA Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"NVIDIA Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/86e45aab42ba48ce83103d01b1119910","url":"https://daily.dev/sources/nvidiadev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/run-ai-models-for-edge-application-development-with-just-a-few-commands-gjbppnyda","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,edge-computing,tensorrt","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"NVIDIA Developer","item":"https://daily.dev/sources/nvidiadev"},{"@type":"ListItem","position":3,"name":"Run AI Models for Edge Application Development with Just a Few Commands"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/run-ai-models-for-edge-application-development-with-just-a-few-commands-gjbppnyda#faq","mainEntity":[{"@type":"Question","name":"How do I run NVIDIA Nemotron-3.5 Lightning on a DGX Spark?","acceptedAnswer":{"@type":"Answer","text":"Install the tensorrt-edgellm package with pip in a virtual environment, then run 'trtedge chat nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4' to start a chat session. The CLI downloads the model (about 21GB) and builds a TensorRT engine automatically on first launch, caching files under ~/.cache/tensorrt-edgellm for reuse on later runs. daily.dev surfaces edge-AI workflow guides like this for developers deploying LLMs on embedded hardware."}},{"@type":"Question","name":"How much faster is speculative decoding with DSpark on DGX Spark?","acceptedAnswer":{"@type":"Answer","text":"Enabling DSpark speculative decoding with three draft tokens increased mean generation speed from 73.4 to 126.9 tokens per second on HumanEval prompts on a DGX Spark 64GB. Time to first token stayed roughly the same across SpeedBench and HumanEval workloads, while peak system memory during DSpark inference reached 27.9GB. benchmark-conscious developers track inference speedups like this on daily.dev before tuning their own deployments."}},{"@type":"Question","name":"What are the requirements to use TensorRT Edge-LLM on DGX Spark?","acceptedAnswer":{"@type":"Answer","text":"You need a DGX Spark with a supported NVIDIA software stack, CUDA 13, TensorRT 10.16.1.11 or newer within the 10.x series, and Python 3.10 to 3.12. The setup was tested with TensorRT Edge-LLM 0.11.1 and TensorRT 10.16.1.11, running on DGX OS 7.6.0 based on Ubuntu 24.04.4 LTS. daily.dev helps edge developers keep track of exact version requirements before setting up new inference stacks."}}]} ```