Did Google’s AI agents really build an operating system for $916?
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Google claimed its Gemini-powered AI agents built an entire operating system for $916 using a single prompt. A critical analysis reveals several methodological problems: the 'single prompt' was actually thousands of lines long, human intervention standards were undefined, no analysis was done to check if agents copied existing code, and Google has not released the prompt, code, or logs for independent verification. The authors argue this type of 'open-world evaluation' is becoming common among AI companies and requires new methodological norms, calling for independent academic and nonprofit evaluators to provide rigor that vendor-run experiments lack.
