Although we believe that AI only comes to help us, the reality is that it can be very dangerous in certain areas. Especially when it perpetuates racist biases in critical sectors such as health.
Google Cloud, Google’s cloud services and products division, is collaborating with Highmark Health, a non-profit healthcare company based in Pittsburgh, on generative AI tools designed to personalize the patient admission experience.
The AWS division of Amazon claims to be working with anonymous clients on a way to use generative AI to analyze medical databases in search of “social determinants of health”.
And Microsoft Azure is helping to create a generative AI system for Providence, the non-profit healthcare network, to automatically classify messages sent by patients to healthcare professionals.
Among the most prominent generative AI startups in the healthcare sector are Ambience Healthcare, which is developing a generative AI application for doctors; Nabla, an ambient AI assistant for doctors; and Abridge, which creates analysis tools for medical documentation.
The widespread enthusiasm for generative AI is reflected in investments in generative AI initiatives aimed at healthcare. Collectively, generative AI startups in healthcare have raised tens of millions of dollars in venture capital to date, and the vast majority of healthcare investors claim that generative AI has significantly influenced their investment strategies.
But both professionals and patients have doubts about whether generative AI focused on healthcare is ready for prime time.
Generative AI may not be what people want
In a recent survey by Deloitte, only about half (53%) of US consumers said they believed that generative AI could improve healthcare, for example, by making it more accessible or reducing waiting times for appointments. Less than half said they expected generative AI to make healthcare more affordable.
Andrew Borkowski, Director of AI at the VA Sunshine Healthcare Network, the largest healthcare system in the US Department of Veterans Affairs, does not believe that cynicism is unwarranted. Borkowski warns that the deployment of generative AI could be premature due to its “significant” limitations and the doubts it raises about its effectiveness.
Several studies suggest that these points are well-founded
In an article published in the JAMA Pediatrics magazine, it was discovered that OpenAI’s generative AI chatbot, ChatGPT, which some healthcare organizations have tested in limited use cases, made errors in diagnosing pediatric diseases 83% of the time.
And when testing OpenAI’s GPT-4 as a diagnostic assistant, doctors at Beth Israel Deaconess Medical Center in Boston observed that the model gave an incorrect diagnosis as the main response almost two out of every three times.
