Keras does this by default. In Keras dropout is disabled in test mode. You can look at the code here and see that they use the dropped input in training and the actual input while testing. As far as I know you have to build your own training function from the layers and specify the … Read more
This paper describes an approach for building a stacked convolutional autoencoder. Based on that paper and some Google searches I was able to implement the described network. Basically, everything you need is described in the Theano convolutional network and denoising autoencoder tutorials with one crucial exception: how to reverse the max-pooling step in the convolutional … Read more
Caffe net juggles two “streams” of numbers. The first is the data “stream”: images and labels pushed through the net. As these inputs progress through the net they are converted into high-level representation and eventually into class probabilities vectors (in classification tasks). The second “stream” holds the parameters of the different layers, the weights of … Read more
Yes that can be done. Its explained in the paper ‘Striving for simplicity: The all convolutional net’ https://arxiv.org/pdf/1412.6806.pdf. Quote from the paper: ‘We find that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks’
Yes. This is a known problem of Adam. The equations for Adam are t <- t + 1 lr_t <- learning_rate * sqrt(1 – beta2^t) / (1 – beta1^t) m_t <- beta1 * m_{t-1} + (1 – beta1) * g v_t <- beta2 * v_{t-1} + (1 – beta2) * g * g variable <- … Read more
A decrease in binary cross-entropy loss does not imply an increase in accuracy. Consider label 1, predictions 0.2, 0.4 and 0.6 at timesteps 1, 2, 3 and classification threshold 0.5. timesteps 1 and 2 will produce a decrease in loss but no increase in accuracy. Ensure that your model has enough capacity by overfitting the … Read more
Unfortunately what you want to do is impossible with Keras … I’ve also struggle a lot of time on this problems and the only way is to dive into the rabbit hole and work with Tensorflow directly to do LSTM rolling prediction. First, to be clear on terminology, batch_size usually means number of sequences that … Read more
The tf.reduce_max() operator provides exactly this functionality. By default it computes the global maximum of the given tensor, but you can specify a list of reduction_indices, which has the same meaning as axis in NumPy. To complete your example: x = tf.constant([[1, 220, 55], [4, 3, -1]]) x_max = tf.reduce_max(x, reduction_indices=[1]) print sess.run(x_max) # ==> … Read more
It is a common practice to decrease the learning rate (lr) as the optimization/learning process progresses. However, it is not clear how exactly the learning rate should be decreased as a function of the iteration number. If you use DIGITS as an interface to Caffe, you will be able to visually see how the different … Read more
it helped me when I changed: validation_data=[X_val, y_val] into validation_data=(X_val, y_val) Actually still wonder why?