Why big data has (so far) failed medicine

YORKTOWN HEIGHTS, NY—Can big data improve human decision-making? That was the question that MIT's Irving Wladawsky-Berger put to a panel of IBM clients at the company's research colloquium on cognitive computing Wednesday. The meat of his discussion focused on Douglas Johnston, a surgeon with the Cleveland Clinic. We'll share their interchange here.

“I think that what's happening now with data science,” said Wladawsky-Berger, “is we can now turn these microscopes on ourselves, systems where the critical components are people, communities, and organizations, and get a level of understanding we didn't have before.”

“In healthcare in general we’ve been applying data science poorly,” admitted Johnston. “We have a medical literature that is contradictory, and we are relying on 100 year old transcription technology for our records. We still have to dig through those records to get the data. I see the results are failing because it's garbage in and garbage out.”

“And that to me is the challenge for all of you,” he said, addressing the audience of computer scientists. “How do we create a system that makes that body of data robust enough that we can work from it?”

“We keep talking about the ability to monitor people with chronic illnesses,” Wladawsky-Berger said, “I’m assuming that in five to ten years we'll be able to generate huge amounts of real time data. Will that change things?"

“I think that will be a huge advance,” Johnston replied. “We need to know how sick someone is when they come in the door, how well do we do, and how well do they do when they leave. I think we've done an okay job with the second two but not the first. We need a way to look at those outcomes that is not dictated by a doctor into a dictation device.”

“In hearing you talk,” said Wladawsky-Berger, “it sounds also as if as we get all this data and start analyzing, whole new models of healthcare will have to be developed. Something different is coming along."

“I hope you're right,” Johnston replied. “The inefficiencies are so rampant in what we do. Cleveland Clinic is the biggest heart care center in the country, but the systems aren't there to support it. It requires a total realignment in how we acquire the data and a change in what the data looks like. We have to evolve as physicians and you have to help us with a system that makes that better."

“So Doug,” said Wladawsky-Berger, “you’ve got a compelling use case in the medical field of why these systems will have a big impact. But you are also in an industry that's extremely risk averse. How do you balance that? How do you think about risk in new technologies in terms of adopting these types of systems?”

“I think right now,” Johnston said, “there's a perfect storm where the powers at be are starting to realize that business as usual is not working, the tools we have available are really poor. We have I think 1,300 coders at the Cleveland Clinic and all they do is look through records. There's a whole separate body for regularity compliance and legal issues. I think the best way to answer you question is it's a notoriously risk-averse profession, but finally the ball has rolled to the point that people realize that the system is broke. What was supposed to be a great savior, electronic medical records, hasn't been the savior we were hoping for. The risk is essentially there but it's sort of like a dying patient, you need to take risks to save them. The healthcare system is terminal, we need a dramatic intervention.”

“One of the changes that seems to be happening in medicine is the realization that a lot of procedures may not be necessary,” Wladawsky-Berger proposed. “One would suspect that data science could help make those decisions better. Is that a possibility?”

“I think you hit the nail on the head,” Johnston said. “There are certain conditions, where there are very well valued predictive models that determine risk of intervention versus non-intervention. The percent of patients whose conditions fit into those models is probably 10%. I think that the challenge to the group is, a large need is to develop predicative models that handle patients with more than one condition. That’s one of the most difficult things for us to measure.”

“So far the cognitive economy rests on a central figure that processes all the data,” Wladawsky-Berger said. “What are your thoughts on cognitive systems that help you nurture your organization at large. How do you think cognitive systems might be able to help you in terms of space and time?”

“It's always been the physician leading the shop and the people doing what we say,” Johnston explained. “I think the cognitive economy will liberate the data in a way that's helpful to the consumer. This has the potential to provide a means of exchanging information in a way that's totally different from what we have right now. Even in the small scale scenarios of a single patient and a team, there is no good way to do that now. I think the potential is huge to distribute that information throughout the team.”

“Do you spend a lot of time training the team so that they are ready?” Wladawsky-Berger asked.

“Training is part of the culture,” Johnston replied, “but the bigger problem is the larger team, the physicians that may not be in contact with. The systems don't support that kind of follow-up. I think that distributed communication will change the way we do business for the better.”

“I think back when Deep Blue beat Gary Kasparov at chess,” Wladawsky-Berger said, “and his reaction was almost violent. And when you think back to when Watson won at Jeopardy, if you go on YouTube and see Ken Jenning's speeches after that, he was disorientated. He was questioning who he was himself, because his identity was all about being the best at Jeopardy. In your field of expertise, what will your identity be in a world where those systems are logically superior and have access to more data?”

“I think you're talking about people with a very specific skills,” Johnston said of Kasparov and Jennings. “I think for us this opens up whole other areas of pursuit. I guess I’m thankful my specialty is a very physical one! I hope that it brings us in my practice to another level, hopefully it's the less fun stuff it makes irrelevant.”