Read Advisory Board's take: Why we need to keep the limitations of AI in mind

A new artificial intelligence (AI) system diagnosed patients roughly as accurately as human doctors, according to a paper published in Nature Medicine—but it could be years before the technology is ready to treat patients in the United States.

The Artificial Intelligence (AI) ecosystem

About the study

For the new study, Kang Zhang—chief of ophthalmic genetics at the University of California, San Diego—and colleagues trained a new AI system using data from almost 600,000 patients at the Guangzhou Women and Children's Medical Center. The data was gathered in China in part because the country's large population and lax privacy norms make it easier for researchers to collect huge volumes of patient information.

Then, the researchers used "deep learning" systems and neural networks—a learning algorithm through which a machine can largely teach itself to recognize patterns in data—to train an AI system identify how patient symptoms relate to specific disease diagnoses within the data.

Finally, the researchers fed their AI system information on new patients and asked it to diagnose their medical conditions. They compared the AI-generated diagnoses to those generated by human doctors.

The result? By and large, the computers beat the humans.

When diagnosing asthma, the AI system was more than 90% accurate, while physicians were between 80% and 94% accurate. Similarly, the AI was 87% accurate at diagnosing gastrointestinal disease, while physicians were between 82% and 90% accurate.

Artificial Emotional Intelligence (AEI) 101

Why AI doctors may not be coming to America just yet

While the study illuminated the promise of using AI as a diagnostic tool, it could be years before such systems are deployed in the United States, according to the Times.

One challenge is that while neural networks can be very powerful in identifying patterns in data, it's not always clear to human operators why the algorithms make the decisions they do, the Times reports.

As a result, the technology would require extensive testing to make sure it generates reliable results. As Ben Shickel, a researcher at the University of Florida, said, "No one is just going to deploy one of these techniques without rigorous testing that shows exactly what is going on."

Further, the technology is likely to be deployed first in countries that face acute shortages of qualified providers, such as India and China.

Even so, it's clear that the United States is investing heavily in developing AI. On Monday, President Trump signed an executive order to establish the "American AI Initiative," which will promote technical education and apprenticeships and boost STEM and computer science programs in U.S. schools. The White House in a fact sheet said, "Investment in AI is critical to creating the industries of the future like autonomous cars, industrial robots, algorithms for disease diagnosis, and more."

And once AI diagnosis technology is implemented in the United States, it could be very useful as an assistant to physicians, Zhang said. "In some situations, physicians cannot consider all the possibilities," he said. "This system can spot-check and make sure the physician didn't miss anything" (Metz, New York Times, 2/11; Miliard, Healthcare IT News, 2/11; White House fact sheet, 2/11).

Advisory Board's take

Greg Kuhnen, Senior Director, Health Care IT Advisor, and Andrew Rebhan, Consultant, Health Care IT Advisor

While this study shows the tremendous potential of AI, it's worth keeping its limitations in mind as well. As other recent studies have shown, clinical decision making is often not as simple as this study concludes. Rather, it is often quite messy and highly dependent on doctor intuition—and understanding this fact is essential to understanding the strengths and limitations of AI.

While medicine aspires to be a quantified science with precisely documented observations, in reality our data often has blind spots and much of medical practice is still an art. Even the most meticulous documentation must leave out details that play a role in the clinical thought process, and doctors make decisions on more than just the data available in a patient's chart.

Our minds are powerful observation and pattern matching machines, but they have limited attention and suffer from a host of well-known decision-making biases. Recent advances in artificial intelligence technologies have made dramatic progress on solving narrowly-focused tasks— predicting sepsis, classifying radiology images, and calculating all kinds of future risks and probabilities. However, progress on broader, more interconnected decision making—often referred to as general intelligence—has been slow.

As a result, mature practitioners of artificial intelligence see the technology as an assistant to human intelligence, not a replacement. Health systems should focus on areas where the strengths of AI can augment human decision making by leveraging AI's infinite attention span, its ability to absorb massive amounts of data, its low cost at opportunity to scale, and its calculated focus on getting answers that are accurate—whether or not they're desirable.

It's also important to point out that artificial intelligence technologies are going through a period of rapid advancement. Future systems will not necessarily be limited to the information that fits neatly into a chart, but may be able to rely on reading body language, recognizing speech patterns, and analyzing vast streams of activity and biosensor data to build a more nuanced understanding—far beyond the simplified picture that makes it to the chart.

To learn more about the increasing role AI is playing in US health care, and what health care organizations should consider when beginning their AI journey, be sure to download our report on The Artificial Intelligence Ecosystem.

Download the Report

Not a Health Care IT Advisor member? Download our AI cheat sheet to get a brief overview of what health care leaders need to know about AI. 

Get the Cheat Sheet