SGI searches for Twitter's 'heartbeat' using big data

Needless to say, there's a lot of emotions and expressed on Twitter everyday, but it feels virtually impossible to navigate through all of them to extract a general consensus on most topics.

However, technical computing giant SGI thinks it might have found a way to pinpoint just that.

SGI has teamed up with researchers from the University of Illinois to analyze the entire Twitter feed for sentiments and volume in real-time using SGI's UV 2000 Big Brain data-mining computer.

By combining geo-tagged tweets with a Twitter-focused sentiment engine, SGI said researchers have been able create a sophisticated streaming map of the "global heartbeat" within Twitter.

Essentially, that heartbeat is described to represent a collective view of the global population's (public) dreams and fears.

Here's how it works. The Global Twitter Heartbeat project processes 10 percent of Twitter's tweets daily as they are posted. (For perspective, SGI asserted there are roughly 500 million per day, on average.)

From there, the project analyzes every tweet to assign location (not just geo-tagged tweets but processing the text of the tweet itself), and tone values. The conversations are then reformated into a visualization (in this case, a heat map infographic) that integrates tweet location, intensity and tone into a unified geospatial perspective.

So far, the SGI-backed project has already taken on two recent events that have caused a flurry on Twitter: Hurricane Sandy and the U.S. Presidential Election.

In the case of last week's election, for example, SGI said that the heat maps showed the dynamics of intensity and location of tweets favorable to either President Barack Obama or Governor Mitt Romney over the course of November 6 -- from the first polls opening to after President Obama’s victory speech.

For a closer look at the Global Twitter Heartbeat's analysis of Hurricane Sandy and Election Day in the U.S., check out the videos below: