stemming
What is the best stemming method in Python?
The results you are getting are (generally) expected for a stemmer in English. You say you tried “all the nltk methods” but when I try your examples, that doesn’t seem to be the case. Here are some examples using the PorterStemmer import nltk ps = nltk.stemmer.PorterStemmer() ps.stem(‘grows’) ‘grow’ ps.stem(‘leaves’) ‘leav’ ps.stem(‘fairly’) ‘fairli’ The results are … Read more
Stemmers vs Lemmatizers
Q1: “[..] are English stemmers any useful at all today? Since we have a plethora of lemmatization tools for English” Yes. Stemmers are much simpler, smaller, and usually faster than lemmatizers, and for many applications, their results are good enough. Using a lemmatizer for that is a waste of resources. Consider, for example, dimensionality reduction … Read more
How do I do word Stemming or Lemmatization?
If you know Python, The Natural Language Toolkit (NLTK) has a very powerful lemmatizer that makes use of WordNet. Note that if you are using this lemmatizer for the first time, you must download the corpus prior to using it. This can be done by: >>> import nltk >>> nltk.download(‘wordnet’) You only have to do … Read more