A WWDC26 session covering how to integrate machine learning into real-time rendering pipelines using Metal 4 on Apple platforms. Three levels of ML integration are presented: MetalFX for platform-integrated neural denoising and upscaling (with best practices around input quality, reflection/transmission handling, and motion vectors), the Metal 4 ML command encoder for deploying offline-trained models like a neural tone mapper (HDRNet) directly in the command buffer, and the TensorOps API for building tiny networks inline in shaders with support for online training. A sky illumination probe example demonstrates online training where a small MLP updates every frame to adapt to dynamic scene conditions. Code examples cover motion vector computation, MLP construction with cooperative tensors, and forward pass evaluation.

22m watch time
72 Impressions