When I posted the original article "Critical Fault: Twitter's Sentiment Paradox" I sent it
to a colleague I respect, who in turn forwarded it to one of his colleagues he respects, who wrote: "My quick opinion is that I don't buy it, I'm afraid. Sorry, Sid." he followed, speaking of the Twitter sentiment analysis companies as "people I don't think deserve to be called Charlatans." And He's right. I called the entire field charlatans. He sent me a couple links to companies that it would be mean spirited to throw in with the bad guys. So I have some apologizing and retracting to do.


As my research practices ensure I never write anything completely off-base, before the retraction commences, let me tell you what made me so concerned I put the field in inquiry and not just the company that concerned me: There is a company harvesting Twitter and selling it in bulk. Your tweets. My Tweets. Tweets that we deleted months ago. Tweets that are in the API to support apps. Content that even Twitter has promised not to sell. If Facebook or Twitter sold our content we would be outraged. We heard that loud and clear in the blogosphere a few months back. For some reason, there's no one up in arms that we have all been made a commodity at the basic unit of modern influence: the Tweet. Infochimps.com is commercially selling our content without permission. (don't show me a TOS Mr Lawyer somewhere, I mean OUR permission, yours, mine, and all the lawyers that tweet). Also without Evan's permission.

TheReadWriteWeb posted some interesting information on the subject last week. It's my opinion that it feels icky and the news had me exclaim WTF? and put on my graph theory hat to take a look. I  found some rather serious anomalies in the field, but I was quick to judge in my post. There are some compelling computer science frontiers being explored that I will definitely write about in the coming days. So I apologize to the emerging sentiment tracking industry, you are not the new SEO. You are asking interesting questions and you have a lot of ground to cover before you'll boom, but with luck and a kind wind, one day our TV might hear us screaming at it, and make sure the sentiment reaches Hollywood, Washington or your local NewsMedia. I'm very grateful people are working on that. My Edits appear strike-through retracted

The emerging field of Twitter sentiment analysis sits upon a fragile assumption: they assume that meaning can be derived from an analysis of a volume of tweets. In this article I will examine the nature of realtime data, prove that the only entity that can make sentiment analysis work is Twitter, debunk the claims of every sentiment analysis hype product, and highlight the dangerous road ahead for the field. This seems like a tough task, but I assure you it is my pleasure. soberingly clear.

It's difficult to get access to, and even more difficult to capture Twitter’s realtime stream. Twitter peaks sometimes at 5k tweets per second, and even if you captured them, how would you make sense of them?

Think about it. You only really have time as an axis. It would be easier if you had network connections to form a graph like the ones seen in the Trends Through Influence and Flow graphic. With the hashtags connecting the tweets and forming a multidimensional graph. But this is not the data sentiment analysis companies examine. They are measuring frequency of positive/neutral/negative. Which looks more like tv static. But how can they discern +/ /- if the words have been mutilated to fit in 140 characters and sub-dialects have emerged like "o em ged" click to examine the sample tweet and see if you can tell if this is positive, neutral or negative.

Some approaches, like this one to "money" , are just a shot in the dark and only barely work in English.

Visualizing Twitter Trends through influence and flow giladlotan'sTwitter Visualizations Set via Flickr

Twitter API harvesting methods like infochimps require a search term to pull the data that they are converting into an asset, but No matter what method phrases you use to pull your sample from Twitter you can not ensure you have an accurate sample for that moment. In the case of Attensity’s semantic analysis of Obama’s Nobel Peace Prize (http://www.attensity.com/en/News-and-Events/Press-Releases/2009/Obama_No...) the variations of Obama’s name are signifigant to examine. He is sometimes referred to by inference and sometimes merely as “O” or “BO”. It's possible that missing even one variant of the target keyword, in this case “obama” and “nobel”, you may miss the majority of messages you are claiming to analyze. For example: “He didn’t deserve it”. Would make total sense in realtime, but is not in Attensity’s graph, how do I know this? Because to get the data from the API they had to provide the keywords, and if they did search for "he did deserve it" and "he didn't deserve it" they couldn't know that they were talking about Obama. This is the beautiful efficiency of realtime data. The relationship with the memespace of the moment. So how can they say that the majority was in favor of Obama's Nobel Award? I’m not attacking Attensity, I’m hi-lighting the road ahead for the industry. I'm sure Attensity's graph is magnificent. I would love to speak with them and examine the technology, but it would have to be some kind of epic graph to be able to see anything at the 6.48 billion characters per day (5k tweets per second) that Twitter is reported to peak at. The entity that has that graph is Twitter.

Natural language processing is not effective where there is no sentence structure. Keywords don't pull all the variations and you can't get a statistically significant sample if you don't get all the keyword variations. In addition to that, the number of tweets is immaterial. More volumes positive an less negative means nothing if you don't know who you are measuring. People have multiple accounts and often retweet themselves, not to gain influence, but to be kind to their followers. For example: I will tweet a ton of Android News from my own account @sidgabriel, because all my followers know and love that I went crazy about android. I will retweet the hardware focused tweets from my @Androidmakers account to alert the people who follow my AndroidMakers group, who don’t want to follow me personally. This does not mean that I am two people buzzing. Yet most of the field does not report twitter accounts per human. Or group accounts like @SplendoraHQ, which I know for a fact is a sea of amazing women around the planet, counted as one user When you are simply comparing a sample from the twitter feed to nothing, you gain no comprehensive insight. You need some kind ofa universal authority metric to qualify each source and turn your tabular data into a graph. I took a stab at it and I think it's a decent sketch. Though I am not a mathemagician.

I believe, to derive authority, you need to track the growth/decline followers over time as a curve, and measure the proximity of a user's curve to the curves of second and third degree followers. Then the delta of those curves to eachother is the position of the user's authority. That makes our first index, which we then compare that to the average curve of all of Twitter users for our second index. Grow your followers like a bot and you position yourself next to bots. Like attracts like and viola. You have a graph. Express each calculation as an interger and magically authority is born. @sidgabriel#538 5 is my own position, 3 is my average follower position factoring in the average position of my followers followers position and 8 is the network-wide average position. It even works if everyone alive is on Twitter, it just makes the final number steady, time is still flowing over the network, so there is still activity to provide relative position of users. It works well as a hash tag too. I'd be interested in who else shares my coordinates in the Twitterverse, and to watch my position change throughout my life. I can assure you that @sidgabriel would be far from @mikesbotnet and close, I hope, to some accounts I love.

Twitter could sell those three numbers for a lot of fun. But unless you are Twitter, and can examine an individual Twitter account in the context of the whole network, in realtime, you can not prove the authenticity of the tweet, so each analytic company will have it's own approach to these problems.

The charlatans marketing sentiment measurement in realtime using semantic technologies have have not examined the long term impact of their products, and aren't proofing their work. Think of Twitter’s stream as a river. The current approach in the field claims to measure the temperature of the river with a cup of water they carry a mile back to a thermometer in a lab. have to remove from the river to test. They don't have the data. It's completely misleading. It's not quite there yet.

As far as I can tell only Twitter can accurately measure the authenticity of a Twitter account, and that authenticity is required to measure sentiment. Something that can only be measured in realtime because it is only relative in realtime. So your questions for an analytic firm should be "How do you derive authority?", "how do you contextualize a tweet in it's moment when you index it?, and finally "how do you process dialect?".

Each active realtime analytics company must answer or be researching those three factors. The danger in looking at it any other way is in creating market incentive for sentiment analysis, without an authority baseline (without it working). The hype that you can create value if you just query Twitter and analyse what it gives you has already begun to seed an ecosystem of spammers that will make an enterprise of faking realtime sentiment in exchange for cash. Very similar to the SEO industry of 03 to 08. They will use our meticulously researched semantic maps to game Twitter. Pouring warm water in the river at intervals. Spamming and toppling Twitter's already straining infrastructure. The response from Twitter will be to filter the specific approach and we will loose our semantic map's integrity as an analytical tool. We didn't know how far SEO would go.

Spam is a powerful piece of looping branching math that must be factored into any system designed for profit influence. This is actually the best argument against Twitter AdSense. Look what happened to Google, they did not anticipate the SEO industry. Now the web is littered with thousands of different versions of the same v1agr4 page, all trying to game the index. Google had to evolve PageRank specifically for the SEO strategies that were gaming the algorithm. You could argue that Google would be closer to realtime discovery and ranking of web pages if they had not created market incentive for SEO. They were mired in that dynamic and the search deal with Twitter makes them literally yesterday's internet. The freshest data will always be on Twitter first.

"This is how page rank works" via Wikipedia

What we can learn from Google is: to make money without destroying the ecosystem you seek to monetize, you must anticipate seo-like industry and design a model in a defensible graph that hasn't already, economically been made efficient. To establish an industry on such shaky ground as sentiment analysis without authority or representing volumes of data pulled by keyword searches of the api  as “10% of Twitter 2009” is plain near fraudulent.
Nothing accurate can be derived from the data.

If you feel moved to protect twitter from the potential threat of API abuse: Twitter wants you: Communicators Apply Here: http://sfbay.craigslist.org/sfc/csr/1450040693.html
Engineers Apply Here: http://sfbay.craigslist.org/sfc/eng/1450040698.html

Good luck Evan, The Twitter Team and The Ecosystem of New Analysys!

Get ready, I talk for 45 minutes straight. This is riveting if you are following Twitter, not so much if you're not. I rarely present so I think it's a hoot. Questions in the comments are very welcome.