Cardiologists at Mayo Clinic have brought data scientists into exam rooms and ORs with the hopes of building algorithms that can succeed where others often fail: becoming part of clinical practice to meaningfully improve patient outcomes, Casey Ross reports for STAT News.

Infographic: Follow the path to AI at your organization

The 'long, last mile'

Ross reports that a key challenge for AI developers is producing algorithms that can bridge "the long, last mile" and maintain performance "amid the innumerable complexities of clinical settings."

Eric Topol, a cardiologist and AI expert at the Scripps Research Translational Institute in San Diego, said, "You don't want to go forward with something validated [on a computer] and start using it on patients." First, you need to be sure it's up for the real-world patient challenge.

For instance, Ross reports that algorithms must demonstrate that they are effective not just for one patient population but on diverse groups of people and are free of bias. Further, the technology must be clear of unintended outcomes, namely "false positives," Ross reports. On top of that, the technology has to fit into clinicians' routines and be "clinically meaningful," according to Ross. "The final—and perhaps the most daunting hurdle is for AI developers to win physicians' trust in an algorithm," Ross writes.

What Mayo is doing

To overcome these challenges, Mayo has embedded five data scientists and software engineers into doctors' daily rounds and ORs to find ways to use AI to fill patient care gaps, Ross reports. The idea is that by shadowing doctors on these daily clinical tasks, the AI team will be better able to develop technology that meets clinicians' needs and gains their trust.

Zachi Attia, a machine learning engineer, is part of the team of researchers who've joined Mayo cardiologists in their practice. Attia has watched doctors insert stents into arteries, shadowed doctors as they performed workups, and has even dissected cadavers to better understand heart anatomy, Ross reports.

"If I wasn't here [at Mayo] I'd look at my results and say, 'Well, I have an accuracy of X and a sensitivity of Y,'" Attia said. "But because I'm here I can go to physicians with examples from the data and ask, 'What do you think is happening here, or what is going on there?' And then I can improve the algorithm."

Detecting a-fib sooner

Over the last three years, the Mayo team has published more than two dozen studies about AI in cardiology, Ross reports. For the most part, the teams' algorithms focus not on surgery or emergency care but rather on "the physiological events" that go undetected, such as problems that cause patients; hearts to skip and sometimes go into cardiac arrest. According to Ross, events like these are treatable if detected, but when they are not they can turn deadly, causing hundreds of thousands of patient deaths per year.

One area where Attia and the team have focused their attention is detecting atrial fibrillation (a-fib), a condition in which the atria beat irregularly, resulting in inefficient blood flow and potential complications, such as clots and eventually stroke, Ross reports. One of the challenges in detecting a-fib is that it occurs intermittently; it may not be occurring at the time an electrocardiogram (EKG) is taken.

To better detect a-fib, the Mayo team is using a type of machine learning system known as a convolutional neural network that is often used in facial recognition software. To train the algorithm to detect a-fib, the team fed 10-second clips of electrocardiograms (EKGs) to the algorithm. The EKGs fell into one of two groups: patients with a-fib and a similar condition labelled "a-flutter" and those without.

However, instead of giving the algorithm EKGs from when the condition surfaced on the EKG, the researchers gave it data from when the a-fib patients' hearts were in their normal rhythm and asked it to identify which of the normal EKGs were the a-fib patients and which were not, Ross reports.

Mayo researchers in August published a study that showed the AI identified a-fib patients with 80% accuracy.

In one case, the AI was shown to detect a-fib retrospectively in a patient whose EKG stumped a seasoned electrophysiologist, Ross reports. The EKG of the patient was taken before he had a stroke of an unknown cause, which might have resulted from a-fib, Ross reports.

Enduring challenges

But the case for intervention based on AI findings is not always cut and dried, according to Ross.

For instance, the blood-thinning drugs used to treat a-fib can lead to serious complications, so doctors generally only prescribe them once an episode of a-fib has been verified.

Topol noted that the algorithm could help treating patients who suffer strokes from unknown causes that may be explained by a-fib if it was clear the patient had an extremely high risk of the condition.

"But you'd still want to nail it," Topol said. "Putting someone on blood thinners means that for the rest of their life they are going to have [an increased risk] of a serious bleed."

The researchers faced another challenge when they conducted a randomized trial to evaluate how a different algorithm influenced primary care physicians' decision making. The researchers previously had found the algorithm successfully identified patients with a weak heart pump—what it was designed to do—in 86% of cases.

But when the trial launched earlier this year it raised repeated questions about whether recommendations for follow-up testing could be followed, given the cost of testing for patients.

Another challenge with the algorithm was it would flag patients who'd already been identified as sick, such as those in hospice care.

Peter Noseworthy, an electrophysiologist who is part of the research team running the trial, said, "[W]e're definitely seeing the reality is much messier than we anticipated." He added, "It's easy to develop the algorithm and prove something in a single database, but to be able to apply it in [clinics] and try to get real results is a whole different challenge. That's what this study is about" (Ross, STAT News, 12/18).