I suspect the place where you copied the code from had eager execution enabled, i.e. had invoked tf.enable_eager_execution() at the start of the program. You could do the same. UPDATE: Note that eager execution is enabled by default in TensorFlow 2.0. So the answer above applies only to TensorFlow 1.x
You can use x.item() to get a Python number from a Tensor that has one element.
I made a diagram. The names follow the PyTorch docs, although I renamed num_layers to w. output comprises all the hidden states in the last layer (“last” depth-wise, not time-wise). (h_n, c_n) comprises the hidden states after the last timestep, t = n, so you could potentially feed them into another LSTM. The batch dimension … Read more
TL;DR Use .clone().detach() (or preferrably .detach().clone()) If you first detach the tensor and then clone it, the computation path is not copied, the other way around it is copied and then abandoned. Thus, .detach().clone() is very slightly more efficient.– pytorch forums as it’s slightly fast and explicit in what it does. Using perflot, I plotted … Read more
I have stumbled upon this problem too and below is what I figured out. When training RNN (LSTM or GRU or vanilla-RNN), it is difficult to batch the variable length sequences. For example: if the length of sequences in a size 8 batch is [4,6,8,5,4,3,7,8], you will pad all the sequences and that will result … Read more
view() reshapes the tensor without copying memory, similar to numpy’s reshape(). Given a tensor a with 16 elements: import torch a = torch.range(1, 16) To reshape this tensor to make it a 4 x 4 tensor, use: a = a.view(4, 4) Now a will be a 4 x 4 tensor. Note that after the reshape … Read more
Found this page on their github repo: Recommended approach for saving a model There are two main approaches for serializing and restoring a model. The first (recommended) saves and loads only the model parameters: torch.save(the_model.state_dict(), PATH) Then later: the_model = TheModelClass(*args, **kwargs) the_model.load_state_dict(torch.load(PATH)) The second saves and loads the entire model: torch.save(the_model, PATH) Then later: … Read more
Units: The amount of “neurons”, or “cells”, or whatever the layer has inside it. It’s a property of each layer, and yes, it’s related to the output shape (as we will see later). In your picture, except for the input layer, which is conceptually different from other layers, you have: Hidden layer 1: 4 units … Read more