for linear regression type of problem, you can simply create the Output layer without any activation function as we are interested in numerical values without any transformation. more info : https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/ for classification : You can use sigmoid, tanh, Softmax etc.
How is the convolution operation carried out when multiple channels are present at the input layer? (e.g. RGB) In such a case you have one 2D kernel per input channel (a.k.a plane). So you perform each convolution (2D Input, 2D kernel) separately and you sum the contributions which gives the final output feature map. Please … Read more
since ReLU doesn’t have a derivative. No, ReLU has derivative. I assumed you are using ReLU function f(x)=max(0,x). It means if x<=0 then f(x)=0, else f(x)=x. In the first case, when x<0 so the derivative of f(x) with respect to x gives result f'(x)=0. In the second case, it’s clear to compute f'(x)=1.
Actually – there is no a good answer to your question. Most of the architectures are usually carefully designed and finetuned during many experiments. I could share with you some of the rules of thumbs one should apply when designing its own architecture: Avoid a dimension collapse in the first layer. Let’s assume that your … Read more
The learning rate looks a bit high. The curve decreases too fast for my taste and flattens out very soon. I would try 0.0005 or 0.0001 as a base learning rate if I wanted to get additional performance. You can quit after several epochs anyways if you see that this does not work. The question … Read more
What is exactly attention? To be able to understand this question, we need to dive a little into certain problems which attention seeks to solve. I think one of the seminal papers on hard attention is Recurrent Models of Visual Attention and I would encourage the reader to go through that paper, even if it … Read more
Each layer has its own default value for initializing the weights. For most of the layers, such as Dense, convolution and RNN layers, the default kernel initializer is ‘glorot_uniform’ and the default bias intializer is ‘zeros’ (you can find this by going to the related section for each layer in the documentation; for example here … Read more
For binary outputs you can use 1 output unit, so then: self.outputs = nn.Linear(NETWORK_WIDTH, 1) Then you use sigmoid activation to map the values of your output unit to a range between 0 and 1 (of course you need to arrange your training data this way too): def forward(self, x): # other layers omitted x … Read more
Pytorch doc for register_buffer() method reads This is typically used to register a buffer that should not to be considered a model parameter. For example, BatchNorm’s running_mean is not a parameter, but is part of the persistent state. As you already observed, model parameters are learned and updated using SGD during the training process. However, … Read more
This worked for me from keras import backend as K and after predicting my data i inserted this part of code then i had again loaded the model. K.clear_session() i faced this problem in production server, but in my pc it was running fine ……….. from keras import backend as K #Before prediction K.clear_session() #After … Read more