Apple introduced Foveated Streaming at WWDC26, a new framework in visionOS 26.4 that lets Apple Vision Pro connect to external devices (like PCs) to stream OpenXR content with low latency over Wi-Fi. The system uses eye tracking to compress video based on gaze direction, delivering higher detail where the user is looking. Under the hood it uses NVIDIA CloudXR for streaming. Developers can build a visionOS receiver app using the FoveatedStreaming framework alongside SwiftUI, ARKit, and RealityKit, while the PC-side OpenXR app implements Apple's lightweight TCP-based Foveated Streaming Protocol and the CloudXR SDK runtime. Real-world examples include X-Plane 12, iRacing, and Autodesk VRED. The talk walks through setting up both the visionOS receiver and the streaming endpoint, pairing via QR code, message channels for bidirectional communication, and performance profiling with a dedicated Xcode instrument.

14m watch time
104 Impressions