How an Indian firm predicted a Trump victory when every American pollster got it wrong
It is the end of polling as we know it. (And I feel fine.)
Okay, maybe that's too harsh or premature. But all of these pundits and pollsters, who with their fancy mathematical models and surveys and assumptions pontificated endlessly in this, their finest hour, only to get it absolutely, disastrously wrong, seem to be facing not just widespread opprobrium, but maybe even their demise.
Someone apparently got it right though.
In an era where self-driving cars promise to soon swarm down roads and Black Mirror is the hottest show on Netflix, it is only appropriate that an algorithm created by a Mumbai-based company MogIA called the election correctly on October 28.
Oh, and MogIA didn't just call this election correctly. It did so with both the Democratic and Republican primaries this year and with three of the last US Presidential elections (although there's been no public verification of those results).
As MogIA's creator Sanjiv Rai explained it to CNBC when he sent them his results in late October, MogIA (named after Rudyard Kipling's Mowgli) plumbs the vast and endless terrain offered by social media to make sense of national sentiments.
Specifically, it scans 20 million data points from public platforms such as Twitter, Google, and Facebook to come up with its predictions. Rai told CNBC that his AI system clearly showed that in any of the elections that MogIA had analysed, the winning candidate was the one that had leading engagement data.
Featured
"If Trump loses, it will defy the data trend for the first time in the last 12 years since internet engagement began in full earnest," Rai wrote in a report sent to CNBC on October 28. Not that anyone was listening. Rai also pointed out that Trump had managed to overtake even Barrack Obama's engagement number in 2008 by 25 percent.
Rai is clearly no slouch. He has been intimately involved in the design of the network convergence-enabling D5 chip; was the founder and chief architect of the CHANDRA project for NASA, which aims to set a communication infrastructure on the moon for permanent human inhabitancy; was named in the Top 10 Asian Technovisionaries by ZDNet; is a former Young Global Leader at the World Economic Forum in 2012; is a Harvard alumnus; and is someone consumed by initiatives in the world of AI convergence.
While the baby he has birthed seems preternatural in its abilities, Rai does admit that there is some learning and improving to be done -- it isn't always easy to figure out whether an engagement is positive or negative for instance. "If you look at the primaries, in the primaries, there were immense amounts of negative conversations that happen with regards to Trump," Rai told CNBC. "However, when these conversations started picking up pace, in the final days, it meant a huge game opening for Trump and he won the primaries with a good margin."
Meanwhile, the rest of the vaunted national pollsters, 60-plus-strong -- including Bloomberg Politics, CBS News, Fox News, Reuters/Ipsos, USA TODAY/Suffolk, Quinnipiac, Monmouth, Economist/YouGov, and NBC News/SM, as well as the much-celebrated polling maestro Nate Silver's FiveThirtyEight website -- consistently put Hillary comfortably ahead, while overestimating Clinton's support among minorities and low-balling Trump's support among white voters.
Only six polls gave Trump the lead -- all six belonging to the Los Angeles Times/University of Southern California tracking polls, not historically famous in this department. Not only did these polls have Trump consistently ahead throughout the dying months of the campaign, they did so by utilizing a methodology that most other polls shunned -- using internet surveys of recruited voters.
THE FUTURE
Not only are new methodologies of gauging political fortunes going to dominate, they are going to get better and better. MogIA was born in 2004 and has relentlessly self-improved over time.
"While most algorithms suffer from programmers/developer's biases, MoglA aims at learning from her environment, developing her own rules at the policy layer and develop expert systems without discarding any data," said Rai.
He thinks that the more granular data gets, the more accurate MogIA becomes. For instance, this AI aficionado thinks it entirely possible to decipher what is going on inside the craniums of potential voters if he were to simply receive anonymized, unique internet addresses of devices from Google where, say, someone searching for a primer on how to vote then surfed a site on how to vote for Trump.
Clearly, the era of a new way of monitoring an election has arrived and a little-known company from Mumbai is pioneering it in relative anonymity -- but that now is not destined to last much longer much like the pollsters it may soon replace.
Related Stories: