Definitely read this section of the docs as there’s some subtleties involved. See also Scikit-learn predict_proba gives wrong answers Basically, if you have a multi-class problem with plenty of data predict_proba as suggested earlier works well. Otherwise, you may have to make do with an ordering that doesn’t yield probability scores from decision_function. Here’s a … Read more
Finally I found the answer to my question with the help of some ideas from @larsmans and @eickenberg. The problem was that X_train did not have the same number of elements in all the arrays. So, it was not able to form a 2D array. Now that I have added an additional value to that … Read more
There are only a few ML libraries that i have used enough so that i am comfortable recommending them; dlib ml is certainly one of them. Sourceforge download here; and bleeding-edge check-out: hg clone http://hg.code.sf.net/p/dclib/code dclib-code The original library creator and current maintainer is Davis King. Your wishlist versus the relevant dlib features: good documentation: … Read more
Are SVMs better than ANN with many classes? You are probably referring to the fact that SVMs are in essence, either either one-class or two-class classifiers. Indeed they are and there’s no way to modify a SVM algorithm to classify more than two classes. The fundamental feature of a SVM is the separating maximum-margin hyperplane … Read more
The most scalable kernel SVM implementation I know of is LaSVM. It’s written in C hence wrap-able in Python if you know Cython, ctypes or cffi. Alternatively you can use it from the command line. You can use the utilities in sklearn.datasets to load convert data from a NumPy or CSR format into svmlight formatted … Read more
There are two parts to this question. The first part is “what is the form of function learned by these methods?” For NN and SVM this is typically the same. For example, a single hidden layer neural network uses exactly the same form of model as an SVM. That is: Given an input vector x, … Read more
In classification tasks for which every test case is guaranteed to be assigned to exactly one class, micro-F is equivalent to accuracy. It won’t be the case in multi-label classification.
Feature scaling is a general trick applied to optimization problems (not just SVM). The underline algorithm to solve the optimization problem of SVM is gradient descend. Andrew Ng has a great explanation in his coursera videos here. I will illustrate the core ideas here (I borrow Andrew’s slides). Suppose you have only two parameters and … Read more
Mathematically, optimizing an SVM is a convex optimization problem, usually with a unique minimizer. This means that there is only one solution to this mathematical optimization problem. The differences in results come from several aspects: SVC and LinearSVC are supposed to optimize the same problem, but in fact all liblinear estimators penalize the intercept, whereas … Read more