Twitter Sentiment Analysis Technics

I'm doing a project on twitter sentiment analysis but there're some things I ponder over. Since tweets are extremely short (less than 140 chars) what text analysis technics apply b

Solution 1:

I did some classwork analyzing celebrities tweets and comparing their similarities.

The biggest thing, which you figured, is the length of a tweet. At 140 chars a lot of words are shortened, or unusual "txt-speech". So even a well know stemmer such as Porter is going to give some odd results. It was best to keep almost everything and only normalize after words counts, vectors, etc.

For extrapolating from the words, n-grams and following links are a big factor for quality inference. I could only tolerate the space and time requirements of 4-grams, but even creating simple 2-grams gave a large improvement.

If you noticed I said earlier "almost everything". In my case of following only popular celeb tweets, I ran into the problem that alot of their tweets were links or shout outs to their events, or sponsors, etc. So a big part was removing the large duplicates of spam.

For the methods to extract accurate sentiment or whatever measures your looking for, I would first try naive bayes based methods. It is simple and relatively accurate for a baseline. K-means will do fairly well but remember that it does not take into account variances and co-variances, but nonetheless is another baseline to try.

Hope that provides some insight.

Solution 2:

I did an analysis recently for a movie on the basis of twitter to find out what are people tweeting about the movie, weather they are liking it or not. This link http://www.laurentluce.com/posts/twitter-sentiment-analysis-using-python-and-nltk/ helped me a lot. In addition I had to gather a list of shortcuts used generally while tweeting which covers the sentiments.

Plus, tweets of a person are only saved until 3000 (or 3.5k not sure ?) and your own Timeline stream also has similar limitations. So you can fetch tweets of your choice or topic using http://topsy.com and fetch old tweets of a particular topic from there for analysis. You might also want to save tweets regularly of your need for future reference because twitter is not going to save for you .

:)