AI Receptionists: A Dangerous Idea?
AI receptionists have been met with skepticism after a patient shared her experience with an accessible yet flawed system. Healthwatch discovered that the AI struggled to understand regional accents and individuals with speech impediments, raising concerns within the healthcare community.
A GP surgery in South Yorkshire removed its AI receptionist following a patient’s struggle to communicate due to a stroke. This patient, facing speech impairments, was unable to get the system to comprehend her, leading her to switch practices.
The NHS Accessible Information Standard, implemented in 2016, requires practices to cater to patients’ communication needs. However, the AI receptionist, named Emma by its developer QuantumLoopAI, appears to have fallen short. While the company claims Emma is trained on diverse accents and supports multiple languages, users tell a different story.
Other patients echoed similar experiences, stating they couldn’t get the system to understand them, leading to frustration and abandoned attempts to book appointments.
Despite the setbacks, the national trend points towards increased AI integration. The NHS App is set to utilize AI for patient triage as part of a £10 billion technology overhaul. South Yorkshire’s integrated care board is working with Healthwatch to provide guidance on implementing such systems, leaving individual GP practices to make their own decisions.
As AI technology advances, standards and regulations must keep pace. The EU AI Act classifies emergency healthcare triage as high-risk, implying stringent testing, logging, and human oversight—requirements that current AI booking lines may not meet.