text-analysis

Training data for sentiment analysis [closed]

by

http://www.cs.cornell.edu/home/llee/data/ http://mpqa.cs.pitt.edu/corpora/mpqa_corpus You can use twitter, with its smileys, like this: http://web.archive.org/web/20111119181304/http://deepthoughtinc.com/wp-content/uploads/2011/01/Twitter-as-a-Corpus-for-Sentiment-Analysis-and-Opinion-Mining.pdf Hope that gets you started. There’s more in the literature, if you’re interested in specific subtasks like negation, sentiment scope, etc. To get a focus on companies, you might pair a method with topic detection, or cheaply just a lot of mentions of … Read more

How to extract common / significant phrases from a series of text entries

by

I suspect you don’t just want the most common phrases, but rather you want the most interesting collocations. Otherwise, you could end up with an overrepresentation of phrases made up of common words and fewer interesting and informative phrases. To do this, you’ll essentially want to extract n-grams from your data and then find the … Read more

Stemmers vs Lemmatizers

by

Q1: “[..] are English stemmers any useful at all today? Since we have a plethora of lemmatization tools for English” Yes. Stemmers are much simpler, smaller, and usually faster than lemmatizers, and for many applications, their results are good enough. Using a lemmatizer for that is a waste of resources. Consider, for example, dimensionality reduction … Read more

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)