From RNNs in Tensorflow, a Practical Guide and Undocumented Features by Denny Britz, published in August 21, 2016.
tf.nn.rnncreates an unrolled graph for a fixed RNN length. That
means, if you calltf.nn.rnnwith inputs having 200 time steps you are
creating a static graph with 200 RNN steps. First, graph creation is
slow. Second, you’re unable to pass in longer sequences (> 200) than
you’ve originally specified.
tf.nn.dynamic_rnnsolves this. It uses atf.Whileloop to dynamically
construct the graph when it is executed. That means graph creation is
faster and you can feed batches of variable size.