When you are not interested in some values returned by a function we use underscore in place of variable name . Basically it means you are not interested in how many times the loop is run till now just that it should run some specific number of times overall.
You may not appreciate the irony, but basically what you have there is legacy code: a chunk of software without any unit tests. Naturally you don’t know where to begin. So you may find it helpful to read up on handling legacy code. The definitive thought on this is Michael Feather’s book, Working Effectively with … Read more
The learning rate looks a bit high. The curve decreases too fast for my taste and flattens out very soon. I would try 0.0005 or 0.0001 as a base learning rate if I wanted to get additional performance. You can quit after several epochs anyways if you see that this does not work. The question … Read more
What is exactly attention? To be able to understand this question, we need to dive a little into certain problems which attention seeks to solve. I think one of the seminal papers on hard attention is Recurrent Models of Visual Attention and I would encourage the reader to go through that paper, even if it … Read more
Short answer, if you’re only interested in solving a prediction task: use Naive Bayes. A Bayesian network (has a good wikipedia page) models relationships between features in a very general way. If you know what these relationships are, or have enough data to derive them, then it may be appropriate to use a Bayesian network. … Read more
Each layer has its own default value for initializing the weights. For most of the layers, such as Dense, convolution and RNN layers, the default kernel initializer is ‘glorot_uniform’ and the default bias intializer is ‘zeros’ (you can find this by going to the related section for each layer in the documentation; for example here … Read more
A RandomForestClassifier is a collection of DecisionTreeClassifier‘s. No matter how big your training set, a decision tree simply returns: a decision. One class has probability 1, the other classes have probability 0. The RandomForest simply votes among the results. predict_proba() returns the number of votes for each class (each tree in the forest makes its … Read more
You can do proj = pca.inverse_transform(X_train_pca) That way you do not have to worry about how to do the multiplications. What you obtain after pca.fit_transform or pca.transform are what is usually called the “loadings” for each sample, meaning how much of each component you need to describe it best using a linear combination of the … Read more
For only one parameter group like in the example you’ve given, you can use this function and call it during training to get the current learning rate: def get_lr(optimizer): for param_group in optimizer.param_groups: return param_group[‘lr’]
With only 3 line of code… scaler = StandardScaler() X_train = scaler.fit_transform(X_train.reshape(-1, X_train.shape[-1])).reshape(X_train.shape) X_test = scaler.transform(X_test.reshape(-1, X_test.shape[-1])).reshape(X_test.shape)