From this reddit thread: This is useful in making the networks ready for fixed-point inference. If you unbound the upper limit, you lose too many bits to the Q part of a Q.f number. Keeping the ReLUs bounded by 6 will let them take a max of 3 bits (upto 8) leaving 4/5 bits for … Read more
Here’s a recipe I copied from one of Alex Mordvintsev deep dream notebook at some point from IPython.display import clear_output, Image, display, HTML import numpy as np def strip_consts(graph_def, max_const_size=32): “””Strip large constant values from graph_def.””” strip_def = tf.GraphDef() for n0 in graph_def.node: n = strip_def.node.add() n.MergeFrom(n0) if n.op == ‘Const’: tensor = n.attr[‘value’].tensor size … Read more
I came across this same issue in jupyter notebooks. This could be an easy fix. $ pip uninstall tensorflow $ pip install tensorflow-gpu You can check if it worked with: tf.test.gpu_device_name() Update 2020 It seems like tensorflow 2.0+ comes with gpu capabilities therefore pip install tensorflow should be enough
Assuming you have all_dataset variable of tf.data.Dataset type: test_dataset = all_dataset.take(1000) train_dataset = all_dataset.skip(1000) Test dataset now has first 1000 elements and the rest goes for training.
Note: The solution you’ve posted (erase TensorBoard’s log files and kill the process) will work, but it isn’t preferred, because it destroys historical information about your training. Instead, you can have each new training job write to a new subdirectory (of your top-level log directory). Then, TensorBoard will consider each job a new “run” and … Read more
Mainly taken from official documentation: The only difference with a regular Session is that an InteractiveSession installs itself as the default session on construction. The methods Tensor.eval() and Operation.run() will use that session to run ops. This allows to use interactive context, like shell, as it avoids having to pass an explicit Session object to … Read more
Old question, but may help others. Try to close interactive sessions active in other processes (if IPython Notebook – just restart kernels). This helped me! Additionally, I use this code to close local sessions in this kernel during experiments: if ‘session’ in locals() and session is not None: print(‘Close interactive session’) session.close()
Here is a brief explanation based on my understanding: momentum helps SGD to navigate along the relevant directions and softens the oscillations in the irrelevant. It simply adds a fraction of the direction of the previous step to a current step. This achieves amplification of speed in the correct direction and softens oscillation in wrong … Read more
Great question! It is quite a trick to find out! As you can see, it is not documented in tf.layers.conv2d If you look at the definition of the function you see that the function calls variable_scope.get_variable: In code: self.kernel = vs.get_variable(‘kernel’, shape=kernel_shape, initializer=self.kernel_initializer, regularizer=self.kernel_regularizer, trainable=True, dtype=self.dtype) Next step: what does the variable scope do when … Read more
I have used the Timeline object to get the time of execution for each node in the graph: you use a classic sess.run() but also specify the optional arguments options and run_metadata you then create a Timeline object with the run_metadata.step_stats data Here is an example program that measures the performance of a matrix multiplication: … Read more