That isn’t how you set parameters in xgboost. You would either want to pass your param grid into your training function, such as xgboost’s train or sklearn’s GridSearchCV, or you would want to use your XGBClassifier’s set_params method. Another thing to note is that if you’re using xgboost’s wrapper to sklearn (ie: the XGBClassifier() or … Read more
Keras can predict multiple values at the same time, like if you input a vector of 100 elements, Keras can compute one prediction for each element, giving 100 outputs. This computation can also be done in batches, defined by the batch_size. This is just in case you cannot fit all the data in the CPU/GPU … Read more
You’re confusing the cross-entropy for binary and multi-class problems. Multi-class cross-entropy The formula that you use is correct and it directly corresponds to tf.nn.softmax_cross_entropy_with_logits: -tf.reduce_sum(p * tf.log(q), axis=1) p and q are expected to be probability distributions over N classes. In particular, N can be 2, as in the following example: p = tf.placeholder(tf.float32, shape=[None, … Read more
You can add class weights to the loss function, by multiplying logits. Regular cross entropy loss is this: loss(x, class) = -log(exp(x[class]) / (\sum_j exp(x[j]))) = -x[class] + log(\sum_j exp(x[j])) in weighted case: loss(x, class) = weights[class] * -x[class] + log(\sum_j exp(weights[class] * x[j])) So by multiplying logits, you are re-scaling predictions of each class … Read more
Below is some sample output for a naive Bayes classifier, using 10-fold cross-validation. There’s a lot of information there, and what you should focus on depends on your application. I’ll explain some of the results below, to get you started. === Stratified cross-validation === === Summary === Correctly Classified Instances 71 71 % Incorrectly Classified … Read more
I will take an attempt to explain: Suppose our training data set is represented by T and suppose data set has M features (or attributes or variables). T = {(X1,y1), (X2,y2), … (Xn, yn)} and Xi is input vector {xi1, xi2, … xiM} yi is the label (or output or class). summary of RF: Random … Read more
The classifiers themselves do not record feature names, they just see numeric arrays. However, if you extracted your features using a Vectorizer/CountVectorizer/TfidfVectorizer/DictVectorizer, and you are using a linear model (e.g. LinearSVC or Naive Bayes) then you can apply the same trick that the document classification example uses. Example (untested, may contain a bug or two): … Read more
The FailedPreconditionError arises because the program is attempting to read a variable (named “Variable_1″) before it has been initialized. In TensorFlow, all variables must be explicitly initialized, by running their “initializer” operations. For convenience, you can run all of the variable initializers in the current session by executing the following statement before your training loop: … Read more
You have at least two options: Transform all your data into a categorical representation by computing percentiles for each continuous variables and then binning the continuous variables using the percentiles as bin boundaries. For instance for the height of a person create the following bins: “very small”, “small”, “regular”, “big”, “very big” ensuring that each … Read more
What you want is called multi-label classification. Scikits-learn can do that. See here: http://scikit-learn.org/dev/modules/multiclass.html. I’m not sure what’s going wrong in your example, my version of sklearn apparently doesn’t have WordNGramAnalyzer. Perhaps it’s a question of using more training examples or trying a different classifier? Though note that the multi-label classifier expects the target to … Read more