The reason for this apparent performance discrepancy between categorical & binary cross entropy is what user xtof54 has already reported in his answer below, i.e.: the accuracy computed with the Keras method evaluate is just plain wrong when using binary_crossentropy with more than 2 labels I would like to elaborate more on this, demonstrate the … Read more
You can easily get the outputs of any layer by using: model.layers[index].output For all layers use this: from keras import backend as K inp = model.input # input placeholder outputs = [layer.output for layer in model.layers] # all layer outputs functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs] # evaluation functions # Testing test … Read more
The lower the loss, the better a model (unless the model has over-fitted to the training data). The loss is calculated on training and validation and its interperation is how well the model is doing for these two sets. Unlike accuracy, loss is not a percentage. It is a summation of the errors made for … Read more
In PyTorch, for every mini-batch during the training phase, we typically want to explicitly set the gradients to zero before starting to do backpropragation (i.e., updating the Weights and biases) because PyTorch accumulates the gradients on subsequent backward passes. This accumulating behaviour is convenient while training RNNs or when we want to compute the gradient … Read more
Found this page on their github repo: Recommended approach for saving a model There are two main approaches for serializing and restoring a model. The first (recommended) saves and loads only the model parameters: torch.save(the_model.state_dict(), PATH) Then later: the_model = TheModelClass(*args, **kwargs) the_model.load_state_dict(torch.load(PATH)) The second saves and loads the entire model: torch.save(the_model, PATH) Then later: … Read more
Units: The amount of “neurons”, or “cells”, or whatever the layer has inside it. It’s a property of each layer, and yes, it’s related to the output shape (as we will see later). In your picture, except for the input layer, which is conceptually different from other layers, you have: Hidden layer 1: 4 units … Read more
Logits is an overloaded term which can mean many different things: In Math, Logit is a function that maps probabilities ([0, 1]) to R ((-inf, inf)) Probability of 0.5 corresponds to a logit of 0. Negative logit correspond to probabilities less than 0.5, positive to > 0.5. In ML, it can be the vector of … Read more
As a complement to the accepted answer, this answer shows keras behaviors and how to achieve each picture. General Keras behavior The standard keras internal processing is always a many to many as in the following picture (where I used features=2, pressure and temperature, just as an example): In this image, I increased the number … Read more
If you like ascii art: “VALID” = without padding: inputs: 1 2 3 4 5 6 7 8 9 10 11 (12 13) |________________| dropped |_________________| “SAME” = with zero padding: pad| |pad inputs: 0 |1 2 3 4 5 6 7 8 9 10 11 12 13|0 0 |________________| |_________________| |________________| In this example: … Read more
In the neural network terminology: one epoch = one forward pass and one backward pass of all the training examples batch size = the number of training examples in one forward/backward pass. The higher the batch size, the more memory space you’ll need. number of iterations = number of passes, each pass using [batch size] … Read more