In 2011, IBM's supercomputer, known as Watson, looked like it could be the future of artificial intelligence. But after a number of stumbles—especially in the health care industry—IBM is redefining, and potentially reinvigorating, what Watson can do, Steve Lohr reports for the New York Times.

From Radio Advisory: When disruptors fail to disrupt: Lessons from Amazon's venture with Haven Healthcare

'The future of knowing'

The idea that eventually became Watson was first pitched in 2006 by David Ferrucci, a scientist at IBM. Ferrucci wanted to develop a supercomputer that could win question-and-answer games as a way to further the AI field called natural language processing, in which computers are programmed to recognize and respond to words.

Cutting through the hype of big data and AI

By 2011, Watson was featured on the game show "Jeopardy!" where it beat Ken Jennings, who is considered the best human to play the game. The day after Watson's appearance on "Jeopardy!" IBM released an ad that said, "Already, we are exploring ways to apply Watson skills to the rich, varied language of health care, finance, law, and academia."

At the time, IBM said Watson's potential uses were endless, Lohr reports, with one company report calling it "the future of knowing."

But even at that time, Ferrucci cautioned others at IBM that Watson was made to identify word patterns and predict correct answers to questions; it wasn't an "all-purpose answer box ready to take on the commercial world," Lohr reports. According to Ferrucci, however, the warning fell on deaf ears.

"It wasn't the marketing message," he said. He left IBM the following year, Lohr reports.

Stumbles in health care

As IBM executives started looking for ways to utilize Watson, the company decided to dive into the health care field, Lohr reports.

"Our moon shot will be the impact we have on health care," Ginni Rometty, IBM's CEO at the time, said. "I'm absolutely positive about it."

IBM's first Watson-related foray into health care involved cancer, Lohr reports. Specifically, the company looked for medical centers that had access to large amounts of data that Watson could process in hopes of improving treatment.

However, when IBM partnered with the University of North Carolina School of Medicine (UNC) to develop what eventually became called Watson for Genomics, "the difficulties soon became apparent," Lohr writes. UNC's oncologists—having seen Watson perform on "Jeopardy!"—"assumed it was an answer machine," while IBM's technologists, in turn, "were frustrated by the complexity, messiness and gaps in the genetic data at the cancer center."

How to combat AI bias

"We thought it would be easy, but it turned out to be really, really hard," Norman Sharpless, former head of UNC's cancer center and now director of the National Cancer Institute, said. "We talked past each other for about a year."

Ultimately, the oncologists and technologists found a use for Watson: having it process thousands of medical research papers in hopes of becoming a new diagnostic tool, Lohr reports. But that application was not "useful or flexible enough to be a winning product," Lohr writes, and IBM last year discontinued Watson for Genomics.

Similarly, MD Anderson Cancer Center ended its collaboration with IBM on another cancer initiative that relied on Watson, called the Oncology Expert Advisor. The project had aimed to make treatment recommendations by analyzing patients' EHRs and related scientific literature, but Watson struggled to decipher MD Anderson's recently updated EHR system, among other issues.

"They chose the highest bar possible, real-time cancer diagnosis, with an immature technology," Shane Greenstein, a professor at MD Anderson, said. "It was such a high-risk path."

IBM also discontinued Watson for Oncology, a separate project that had come out of a partnership with Memorial Sloan Kettering Cancer Center, and it is currently scaling back Watson Health, a separate business the company launched in 2015.

Where Watson stumbled

According to a number of former and current IBM managers who spoke to the Times, IBM's problems with Watson stemmed in part from the company's emphasis on large, difficult initiatives, rather than narrower objectives.

Manoj Saxena, a former general manager of the Watson business at IBM, said the company's original, big-scale goals were laudable but unrealistic. "The challenges turned out to be far more difficult and time-consuming than anticipated," Saxena said.

Separately, Martin Kohn, a former chief medical scientist at IBM Research, said he remembered advising that Watson be used for very specific "credibility demonstrations," such as predicting whether a patient would have an adverse reaction to a specific drug, instead of recommending cancer treatments.

"I was told I didn't understand," Kohn said.

Where Watson is now

For its part, IBM has said that stumbles are part of developing any new technology.

"Innovation is always a process," Rob Thomas, the executive in charge of Watson over the past few years and current SVP for global sales, said.

And after its past challenges, Watson now appears to be making "major strides," Lohr reports.

For example, in an analysis performed for the Times, the Allen Institute for Artificial Intelligence compared how well AIs from Amazon, Google, Microsoft, and IBM performed on standard natural language tasks, such as identifying people or places. The analysis found that Watson performed just as well as, and sometimes better than, its competitors.

"I was quite surprised," Oren Etzioni, CEO of the Allen Institute, said. "IBM has gotten its act together, certainly in these capacities."

In addition, IBM has successfully bolstered Watson's business prospects, posting it primarily as a collection of software tools that streamline and automate tasks such as accounting, payments, technology operations, marketing, and customer service, Lohr reports. For example, the company's Watson Assistant is used by companies, including Anthem, to automatic customer service inquiries, Lohr reports.

Overall, IBM said it currently has 40,000 Watson customers across 20 industries worldwide—twice the number it had four years ago. "Adoption is accelerating," Thomas said. (Lohr, New York Times, 7/16)