NVIDIA introduces Cosmos-H-Dreams, a real-time action-conditioned generative simulator for surgical robotics. Built by distilling the Cosmos-H-Surgical-Simulator into a causal student model and served via FlashDreams (an accelerated streaming-inference library), it achieves ~160 fps on a single RTX PRO 6000 GPU. The system uses self-forcing distillation to handle autoregressive error compounding, supports few-step diffusion (as few as 2 denoising steps per latent frame), and enables closed-loop interaction via browser/WebRTC, Meta Quest/WebXR, or learned surgical policies. It was trained on dVRK tabletop suturing data including failure cases, and has been integrated with the Versius surgical platform. The project is open-sourced with model checkpoints, datasets, and fine-tuning recipes available.
