AI advice: Making people less accurate but more confident
Researchers found that access to AI advice significantly impacted human decision-making. Here are the key takeaways:
- Willingness to say "I don’t know" collapsed: From 44% to a mere 3% when AI was available.
- Accuracy decreased: Dropped from 27% to 9%.
- Confidence increased: Rose from 30% to 76%.
The study, conducted by researchers from three French and Italian universities, used specific questions where AI models typically fail, like visual details from films. They found that people trusted wrong AI answers and the mere availability of AI suppressed the tendency to recognize one’s ignorance.
Monetary incentives slightly improved willingness to admit ignorance and accuracy but had a minimal impact.
The researchers highlight the term "cognitive surrender," coined by Wharton researchers earlier, describing the trend of accepting incorrect AI answers while reporting higher confidence than without AI. They express concern for children growing up with these systems before developing critical thinking skills.
The pattern suggests that AI products are designed to answer confidently, never to say "I don’t know," influencing users to adopt a similar behavior.