Are SVMs better than ANN with many classes? You are probably referring to the fact that SVMs are in essence, either either one-class or two-class classifiers. Indeed they are and there’s no way to modify a SVM algorithm to classify more than two classes. The fundamental feature of a SVM is the separating maximum-margin hyperplane … Read more
To understand how backpropagation is even possible with functions like ReLU you need to understand what is the most important property of derivative that makes backpropagation algorithm works so well. This property is that : f(x) ~ f(x0) + f'(x0)(x – x0) If you treat x0 as actual value of your parameter at the moment … Read more
If your first 9 layers are consistently named between your original trained model and the new model, then you can use model.load_weights() with by_name=True. This will update weights only in the layers of your new model that have an identically named layer found in the original trained model. The name of the layer can be … Read more
From the recent Deep Learning book by Goodfellow et al., chapter 8: Minibatch sizes are generally driven by the following factors: Larger batches provide a more accurate estimate of the gradient, but with less than linear returns. Multicore architectures are usually underutilized by extremely small batches. This motivates using some absolute minimum batch size, below … Read more
ROC Curves: x-axis: False Positive Rate FPR = FP /(FP + TN) = FP / N y-axis: True Positive Rate TPR = Recall = TP /(TP + FN) = TP / P Precision-Recall Curves: x-axis: Recall = TP / (TP + FN) = TP / P = TPR y-axis: Precision = TP / (TP + … Read more
Check out the three-sigma rule: mu = mean of the data std = standard deviation of the data IF abs(x-mu) > 3*std THEN x is outlier An alternative method is the IQR outlier test: Q25 = 25th_percentile Q75 = 75th_percentile IQR = Q75 – Q25 // inter-quartile range IF (x < Q25 – 1.5*IQR) OR … Read more
No, you cannot use Batch Normalization on a recurrent neural network, as the statistics are computed per batch, this does not consider the recurrent part of the network. Weights are shared in an RNN, and the activation response for each “recurrent loop” might have completely different statistical properties. Other techniques similar to Batch Normalization that … Read more
Fitting finds the internal parameters of a model that will be used to transform data. Transforming applies the parameters to data. You may fit a model to one set of data, and then transform it on a completely different set. For example, you fit a linear model to data to get a slope and intercept. … Read more