NVIDIA XR AI is now in public beta as an open source library for building AI agents on AR glasses and XR headsets. It provides a modular architecture connecting live camera/microphone streams from XR devices to GPU-accelerated AI services via an XR Media Hub. The stack integrates NVIDIA Cosmos vision-language models, Nemotron language models, speech recognition (Parakeet), and enterprise connectivity via Model Context Protocol (MCP). A step-by-step guide covers cloning the repo, starting model servers, running a simple multimodal agent, connecting enterprise data through MCP servers, adding orchestration with NeMo Agent Toolkit, and optionally streaming rendered 3D content via CloudXR. Use cases include field service, healthcare, manufacturing, and industrial operations.
