Google's AI Overviews feature, a RAG-based system powered by Gemini, processes trillions of searches yearly but carries a ~9% error rate that translates to tens of millions of wrong answers per hour. A key structural flaw is source-quality blindness: the retrieval layer treats Reddit jokes and medical journals equally, partly because Google deliberately boosted forum content and signed a $60M licensing deal with Reddit. Over half of even correct answers are 'ungrounded' — citations don't actually support the claims made, and this worsened as the model improved. Behavioral data shows users click citation links only 1% of the time, making the AI answer effectively final. Common Sense Media rated Google Search 'Unacceptable Risk' for minors, noting no parental off-switch exists for AI Overviews. A feedback loop is emerging where AI Overviews reduce traffic to expert publishers, shrinking the quality source pool and forcing greater reliance on forum content. Proposed fixes include source-tier transparency in UI, abstention for low-authority queries, grounding verification before display, and a default-off switch for minors.
