Researchers at the University of the Witwatersrand published a study in PNAS offering a theoretical explanation for why AI models become more capable as they scale. Drawing on 'iterated learning' from linguistics — where language becomes more structured as it passes between generations — and deep neural networks, the team found that structured, compositional behaviour only emerges in networks with sufficient depth and complexity. The study used deep linear networks rather than full LLMs, making it a theoretical result rather than a direct observation in frontier models, but it suggests a fundamental mechanism linking network depth to emergent AI capabilities, analogous to how children acquire language.
Table of contents
Read: MTN to turn its African towers into an AI inference grid566 Impressions1 Comment