Researchers at Geisinger have developed an algorithm uses electrocardiogram (ECG) data to determine whether a person will die within a year, Geisinger researchers recently announced. 

Study details

Scientists from Geisinger looked at how well artificial intelligence (AI) could predict whether patients would die within one year based on their ECG reading. To build the AI, researchers used the data from 1.77 million ECGs from nearly 400,000 patients over three decades who had not yet developed any irregular heart rhythms.

The researchers trained two types of the AI model. One model used only raw ECG data—which measures voltage over time—while the other model used "human-derived measures," such as the ECG features recorded by a cardiologist, as well as other disease patterns.

To analyze performance of the models, the researchers used a metric known as AUC, or Area Under Curve, which measures how well a model distinguishes between two groups. Here, the groups were patients who lived and those who died after one year.

Key findings

The researchers presented their findings on Nov. 16 at the American Heart Association's Scientific Sessions in Dallas.

They found that the model that used only raw ECG data was far superior at predicting the odds that a patient would die within one year—consistently scoring above .85, where 1 is a perfect score and 0.5 shows no distinction between the models. In contrast, the AUCs for models currently used by doctors range from .65 to 0.8.

Surprisingly, the model was able to predict risk of death accurately even among patients whom a physician said had a normal ECG. According to the researchers, three cardiologists separately reviewed the ECGs which had been deemed as normal but the AI had correctly deemed as risky, and generally could not recognize the risk patterns that the AI did.

Brandon Fornwalt, co-senior author on the study and chair of the Department of Imaging Science and Innovation at Geisinger, said, "This is the most important finding of this study." He added, "This could completely alter the way we interpret ECGs in the future."

Further, Fornwalt noted when reviewing the results that, "No matter what, the voltage-based model [trained on the raw ECG data] was always better than any model you could build out of things that we already measure from an ECG."

Therefore, "The model is seeing things that humans probably can't see, or at least that we just ignore and think are normal." He added, "AI can potentially teach us things that we've been maybe misinterpreting for decades."

Still, the fact that physicians don't know what patterns the AI is picking up makes some physicians reluctant to rely on them for clinical practice. This reflects a common problem with new AI applications—that they often operate as a "black box" without providing insight into their inner workings. Since these inner workings are shielded from human eyes within layers of computations and lines of code, it can often be hard to diagnose any errors or biases. This presents a major hurdle to FDA approval and widespread usage, at least until further advancements in "explainability," or the ability for models to provide clues about what parts of an patient's record or test were the most important factors in drawing their conclusions (Tangerman, Futurism, 11/11; Lu, NewScientist, 11/11; Geisinger release, 11/15; American Heart Association release, 11/11).