MIT researchers studied how AI-based diagnostic assistance affects non-experts and clinicians differently when diagnosing skin diseases. While AI improved accuracy for both groups, non-experts showed strong deference to AI predictions — including wrong ones — especially when LLM-generated explanations were vague or confident-sounding. Clinicians, by contrast, were resilient to incorrect AI outputs and benefited least from LLM explanations. The study highlights that explainable AI can trigger automation bias, and that the timing and format of AI explanations significantly influence user behavior. Researchers suggest forcing users to form their own hypothesis before seeing AI suggestions as a way to reduce overreliance.
1 Impression