IEEE Spectrum

The landscape of PC architecture is undergoing a significant transformation to meet the growing demands of local AI workloads. Central to this evolution is the integration of Neural Processing Units (NPUs) into modern laptops, enabling them to run complex AI models natively. By 2023, the standard NPU performance was around 10 TOPS, but advancements anticipate this to soar up to 350 TOPS in upcoming models. This improvement is a key factor in supporting sophisticated AI applications on personal devices.
A pivotal aspect of this transition is the move towards unified memory architectures. Initially spearheaded by Apple’s innovations, this approach has been embraced by AMD’s Ryzen AI Max chips. It allows CPUs, GPUs, and NPUs to share memory pools, with capacities reaching up to 128GB. Such unification eradicates the bottlenecks associated with traditional split-memory designs, significantly enhancing the speed and efficiency of processing AI workloads.
In this new ecosystem, Microsoft’s AI Foundry Local stands out by equipping developers with tools necessary to deploy and run thousands of open-source models right on the device. It ensures that tasks are automatically distributed across the available hardware, optimizing the performance and leveraging the full potential of integrated components.
This architectural transformation offers several benefits, including reduced latency due to the lack of cloud dependency, and improved data privacy, as information can be processed locally without needing to be transmitted over the internet. However, one of the trade-offs is reduced upgradeability, as the integration of components into single packages limits the ability to independently upgrade specific parts of the system.
Companies like AMD, Intel, and Qualcomm are deeply invested in this race to redefine computer capabilities, aiming to unlock powerful on-device AI performance. This change marks a departure from the decades-old architecture designed around separate processing units with individual memory allocations, setting a new standard in personal computing.

Collections

PC architecture is evolving to support local AI workloads through the integration of Neural Processing Units (NPUs) with performance expected to reach 350 TOPS from the current 10 TOPS standard. Unified memory architectures, pioneered by Apple and adopted by AMD, allow CPUs, GPUs, and NPUs to share memory pools up to 128GB, eliminating bottlenecks from traditional split-memory designs. Microsoft's AI Foundry Local enables developers to deploy thousands of open-source models directly on devices with automatic hardware optimization. This shift offers reduced latency and improved privacy through local processing, though it comes with trade-offs in upgradeability as components become more integrated.

The Evolution of PC Architecture to Support Local AI Workloads