Although import torch torch.cuda.empty_cache() provides a good alternative for clearing the occupied cuda memory and we can also manually clear the not in use variables by using, import gc del variables gc.collect() But still after using these commands, the error might appear again because pytorch doesn’t actually clears the memory instead clears the reference to … Read more
In keras – while building a sequential model – usually the second dimension (one after sample dimension) – is related to a time dimension. This means that if for example, your data is 5-dim with (sample, time, width, length, channel) you could apply a convolutional layer using TimeDistributed (which is applicable to 4-dim with (sample, … Read more
What I use is the following: with open(‘/trainHistoryDict’, ‘wb’) as file_pi: pickle.dump(history.history, file_pi) In this way I save the history as a dictionary in case I want to plot the loss or accuracy later on. Later, when you want to load the history again, you can use: with open(‘/trainHistoryDict’, “rb”) as file_pi: history = pickle.load(file_pi) … Read more
Early stopping is basically stopping the training once your loss starts to increase (or in other words validation accuracy starts to decrease). According to documents it is used as follows; keras.callbacks.EarlyStopping(monitor=”val_loss”, min_delta=0, patience=0, verbose=0, mode=”auto”) Values depends on your implementation (problem, batch size etc…) but generally to prevent overfitting I would use; Monitor the validation … Read more
how many hidden layers? a model with zero hidden layers will resolve linearly separable data. So unless you already know your data isn’t linearly separable, it doesn’t hurt to verify this–why use a more complex model than the task requires? If it is linearly separable then a simpler technique will work, but a Perceptron will … Read more
I came across this phenomenon several times. Here are my observations: Gradient blow up Reason: large gradients throw the learning process off-track. What you should expect: Looking at the runtime log, you should look at the loss values per-iteration. You’ll notice that the loss starts to grow significantly from iteration to iteration, eventually the loss … Read more
Gradient clipping needs to happen after computing the gradients, but before applying them to update the model’s parameters. In your example, both of those things are handled by the AdamOptimizer.minimize() method. In order to clip your gradients you’ll need to explicitly compute, clip, and apply them as described in this section in TensorFlow’s API documentation. … Read more
Reference is from this github issue. When the error is RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #4 ‘mat1’, you would need to use the .float() function since it says Expected object of scalar type Float. Therefore, the solution is changing y_pred = model(X_trainTensor) to y_pred = model(X_trainTensor.float()). … Read more
Generally Your pkl file is, in fact, a serialized pickle file, which means it has been dumped using Python’s pickle module. To un-pickle the data you can: import pickle with open(‘serialized.pkl’, ‘rb’) as f: data = pickle.load(f) For the MNIST data set Note gzip is only needed if the file is compressed: import gzip import … Read more
Keras has now added Train / validation split from a single directory using ImageDataGenerator: train_datagen = ImageDataGenerator(rescale=1./255, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, validation_split=0.2) # set validation split train_generator = train_datagen.flow_from_directory( train_data_dir, target_size=(img_height, img_width), batch_size=batch_size, class_mode=”binary”, subset=”training”) # set as training data validation_generator = train_datagen.flow_from_directory( train_data_dir, # same directory as training data target_size=(img_height, img_width), batch_size=batch_size, class_mode=”binary”, subset=”validation”) # … Read more