Artificial intelligence tools for note-taking in Ontario’s healthcare system have come under scrutiny. A recent report by Ontario’s Auditor General, Shelley Spence, revealed serious flaws in these AI systems. These issues could potentially jeopardize patient health outcomes by leading to inaccurate medical records.
Key Findings from the Auditor General’s Report
The auditor conducted an investigation into the use of AI across provincial services, particularly focusing on AI medical systems that transcribe doctor-patient conversations. The evaluation included testing 20 AI programs, which revealed significant transcription errors.
- Nine systems exhibited “hallucinations,” inaccurately fabricating information.
- Twelve programs reported incorrect drug prescriptions.
- Seventeen systems failed to capture essential details related to patients’ mental health.
- Some vendors did not submit necessary third-party audits during the procurement process.
Potential Risks
The inaccuracies pinpointed by the audit could result in harmful treatment plans. For instance, some AI scribes mistakenly recommended patient referrals and ordered blood tests that were not part of the doctors’ original instructions. This ultimately underscores the need for rigorous testing of AI technologies employed in the healthcare sector.
Government Response and Recommendations
In light of these findings, the auditor made ten recommendations aimed at enhancing the security and efficacy of AI in public service. Notably, the government has agreed to nine of these proposals. One crucial suggestion is to develop IT controls that require doctors to confirm they have reviewed AI-generated notes.
Stephen Crawford, the Minister of Public and Business Service Delivery and Procurement, acknowledged that errors were discovered during the initial testing phase. He emphasized that physicians oversee the entire process when utilizing these AI tools. Crawford stated that these AI scribes aim to save doctors time, allowing them to concentrate more on patient care rather than administrative tasks.
Current Usage of AI Scribes
Currently, around 5,000 physicians in Ontario are using AI scribe technology. According to Ema Popovic, a spokesperson for Health Minister Sylvia Jones, physicians must review all AI-produced documentation before it is entered into medical records. The use of such technology is voluntary and requires patient consent.
Concerns from Political Leaders
Green Party Leader Mike Schreiner expressed alarm regarding the audit’s conclusions. He raised concerns about patient safety, urging that proper testing must precede the implementation of AI tools in healthcare.
The findings from the Ontario Auditor General highlight critical areas for improvement in the application of AI in the medical field. Ensuring these systems are accurate and reliable is essential for patient care and safety.
