Why it is crucial to make sense of 'Twitter speak'

Guest editorial by Jeff Catlin

We often get asked what the next step in analyzing social media will be, so it seems appropriate that with Twitter being the “it” micro-blogging and messaging service these days, Twitter sentiment analysis is the natural selection.  Twitter has seen its user base grow dramatically over the last 24 months, far outstripping the growth numbers for the established social media players, like MySpace and Facebook.  Text and sentiment analysis naturally play into anything that contains words and text – and Twitter is no exception.

The interesting thing in the growth numbers of Twitter is that it is growing across a wide age range, which means that it’s starting to garner some seasoned business users (our “over 35” VP of Marketing is certainly an avid business user).  One undeniable effect of this change is that Twitter is becoming the incubator of opinion for a whole host of consumer products.  Companies certainly realize this and are trying to monitor Twitter, but technologically-speaking Twitter is a tough thing to monitor.  Sentiment measurement is at the forefront of much business analysis these days, but in some ways Twitter seems as if it was designed from the ground up to defeat any automated sentiment engine.

For instance, there isn’t much sentence structure in tweets, and what’s there is often wrong. And many of the tweets are just tinyurl or bit.ly links with absolutely no content contained in the URL itself.

Given these challenges, is monitoring and measuring sentiment in Twitter a hopeless chore?  Fortunately the answer is No. Even though there are some challenges to automated scoring of Twitter content,  there are also some advantages to processing  tweets and in particular the tone within Twitter.

The beauty of Twitter is that there is very little grey area in tweets.  You’re either posting some source of information, posting an opinion you have, or replying to another informative or opinion-oriented tweet.  Clearly, opinion can easily bleed into information (“Great sale on laptops at @BestBuy”), but even in these cases the tone is clear.  When people put out an opinion on Twitter it’s not a muted, polite, and carefully written opinion, it’s pretty much stream of consciousness, and typically includes strongly toned words.  This was made more evident than ever in the Motrin mom’s ad campaign controversy last year as well as Skittle’s Twitter campaign.  These controversial advertising and marketing campaigns, among others, are solid proof of the need for companies to be actively monitoring and engaging with consumers in the myriad of social networks and platforms, like Twitter, where conversations are living and changing right before their very eyes.

<="" span="">

With Twitter for example, the concise use of words in tweets lends itself to automated sentiment scoring very nicely, so if you can learn the outlying terminology (essentially trendy ways of keeping tweets to 140 characters) and filter out the factual posts, then sentiment analysis will, in fact, work well with Twitter.  The key to this is putting in the time to understand how tone is expressed in these short, “conversational” posts.

Lexalytics has put two of our summer interns on this very task with the goal of enhancing our ability to measure the tone of tweets.  A number of other vendors appear to be doing similar work, so in the very near future the accurate measurement of sentiment in Twitter will be a reality.

Jeff Catlin is the CEO of Lexalytics and has over 15 years of experience in the fields of search, classification and text analytics products and services.