The Upshot data was crunched from many different polls and fed into a special algorithm based on historical and other relevant data. Other organizations also used reams of Big Data to feed their analytical models and were coming to similar predictions: Trump would lose.
So how was it that all these sophisticated analytical models with access to high quality data got the election forecast so wrong?
Jim Rutenberg in the New York Times writes that there was a cultural bias.
Journalists didn't question the polling data when it confirmed their gut feeling that Mr. Trump could never in a million years pull it off. They portrayed Trump supporters who still believed he had a shot as being out of touch with reality. In the end, it was the other way around.
There's an important lesson for enterprises here is that simply getting access to all your Big Data is not enough. It won't result in valuable business predictions unless the analysis is the right one.
Domain knowledge counts for a tremendous amount of success with Big Data because analysis matters and knowing the right questions to ask comes from experience. The right analytical model is vital but being aware of cultural bias in those models is key.