In the neural network terminology: one epoch = one forward pass and one backward pass of all the training examples batch size = the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you’ll need. number of iterations = number of passes, each pass using [batch size] … Read more
The accepted answer has many elements of k-NN (k-nearest neighbors), a different algorithm. Both k-NN and NaiveBayes are classification algorithms. Conceptually, k-NN uses the idea of “nearness” to classify new entities. In k-NN ‘nearness’ is modeled with ideas such as Euclidean Distance or Cosine Distance. By contrast, in NaiveBayes, the concept of ‘probability’ is used … Read more
Let’s say you have input data x and you want to classify the data into labels y. A generative model learns the joint probability distribution p(x,y) and a discriminative model learns the conditional probability distribution p(y|x) – which you should read as “the probability of y given x“. Here’s a really simple example. Suppose you … Read more
I think that biases are almost always helpful. In effect, a bias value allows you to shift the activation function to the left or right, which may be critical for successful learning. It might help to look at a simple example. Consider this 1-input, 1-output network that has no bias: The output of the network … Read more