In general, when you shuffle the training data (a set of sequences), you shuffle the order in which sequences are fed to the RNN, you don’t shuffle the ordering within individual sequences. This is fine to do when your network is stateless:
Stateless Case:
The network’s memory only persists for the duration of a sequence. Training on sequence B before sequence A doesn’t matter because the network’s memory state does not persist across sequences.
On the other hand:
Stateful Case:
The network’s memory persists across sequences. Here, you cannot blindly shuffle your data and expect optimal results. Sequence A should be fed to the network before sequence B because A comes before B, and we want the network to evaluate sequence B with memory of what was in sequence A.