In general, when you shuffle the training data (a set of sequences), you shuffle the order in which sequences are fed to the RNN, you don’t shuffle the ordering within individual sequences. This is fine to do when your network is stateless: Stateless Case: The network’s memory only persists for the duration of a sequence. … Read more
Below is an example that sets up time series data to train an LSTM. The model output is nonsense as I only set it up to demonstrate how to build the model. import pandas as pd import numpy as np # Get some time series data df = pd.read_csv(“https://raw.githubusercontent.com/plotly/datasets/master/timeseries.csv”) df.head() Time series dataframe: Date A … Read more
TL;DR Several possible errors, most fixed with x = np.asarray(x).astype(‘float32′). Others may be faulty data preprocessing; ensure everything is properly formatted (categoricals, nans, strings, etc). Below shows what the model expects: [print(i.shape, i.dtype) for i in model.inputs] [print(o.shape, o.dtype) for o in model.outputs] [print(l.name, l.input_shape, l.dtype) for l in model.layers] The problem’s rooted in using … Read more
Your understanding of most of the concepts is accurate, but, there are some missing points here and there. Interfacing embedding to LSTM (Or any other recurrent unit) You have embedding output in the shape of (batch_size, seq_len, embedding_size). Now, there are various ways through which you can pass this to the LSTM. * You can … Read more
I recommend you to firstly learn the concepts of BPTT (Back Propagation Through Time) and mini-batch SGD(Stochastic Gradient Descent), then you’ll have further understandings of LSTM’s training procedure. For your questions, Q1. In stateless cases, LSTM updates parameters on batch1 and then, initiate hidden states and cell states (usually all zeros) for batch2, while in … Read more
Why don’t you try it out for yourself and see? In my case, training a model with LSTM took 10mins 30seconds. Simply switching the call from LSTM() to CuDNNLSTM() took less than a minute. I also noticed that switching to CuDNNLSTM() speeds up model.evaluate() and model.predict() substantially as well.
Cheking with some tests, I got to the following conclusion, which is according to the documentation and to Nassim’s answer: First, there isn’t a single state in a layer, but one state per sample in the batch. There are batch_size parallel states in such a layer. Stateful=False In a stateful=False case, all the states are … Read more
From this brilliant article num_units can be interpreted as the analogy of hidden layer from the feed forward neural network. The number of nodes in hidden layer of a feed forward neural network is equivalent to num_units number of LSTM units in a LSTM cell at every time step of the network. See the image … Read more
Have you checked whether your training/testing data and training/testing labels are all numpy arrays? It might be that you’re mixing numpy arrays with lists.
I am not clear about the embedding procedure. But still here is a way to implement a variable-length input LSTM. Just do not specify the timespan dimension when building LSTM. import keras.backend as K from keras.layers import LSTM, Input I = Input(shape=(None, 200)) # unknown timespan, fixed feature size lstm = LSTM(20) f = K.function(inputs=[I], … Read more