If you want to find how much time was spent on each operation at TF, you can do this in tensorboard using runtime statistics. You will need to do something like this (check the full example in the above-mentioned link):
run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
run_metadata = tf.RunMetadata()
sess.run(<values_you_want_to_execute>, options=run_options, run_metadata=run_metadata)
your_writer.add_run_metadata(run_metadata, 'step%d' % i)
Better than just printing it you can see it in tensorboard:
Additionally, clicking on a node will display the exact total memory,
compute time, and tensor output sizes.
Also now tensorflow has a debugger. Here is a tutorial of how to use it.
