All the optimizers have a private variable that holds the value of a learning rate. In adagrad and gradient descent it is called self._learning_rate. In adam it is self._lr. So you will just need to print sess.run(optimizer._lr) to get this value. Sess.run is needed because they are tensors.
The get_variable() function creates a new variable or returns one created earlier by get_variable(). It won’t return a variable created using tf.Variable(). Here’s a quick example: >>> with tf.variable_scope(“foo”): … bar1 = tf.get_variable(“bar”, (2,3)) # create … >>> with tf.variable_scope(“foo”, reuse=True): … bar2 = tf.get_variable(“bar”) # reuse … >>> with tf.variable_scope(“”, reuse=True): # root variable … Read more
Example usage: from tensorflow.python.tools.inspect_checkpoint import print_tensors_in_checkpoint_file import os checkpoint_path = os.path.join(model_dir, “model.ckpt”) # List ALL tensors example output: v0/Adam (DT_FLOAT) [3,3,1,80] print_tensors_in_checkpoint_file(file_name=checkpoint_path, tensor_name=””) # List contents of v0 tensor. # Example output: tensor_name: v0 [[[[ 9.27958265e-02 7.40226209e-02 4.52989563e-02 3.15700471e-02 print_tensors_in_checkpoint_file(file_name=checkpoint_path, tensor_name=”v0″) # List contents of v1 tensor. print_tensors_in_checkpoint_file(file_name=checkpoint_path, tensor_name=”v1″) Update: all_tensors argument was added to … Read more
There is just one cross (Shannon) entropy defined as: H(P||Q) = – SUM_i P(X=i) log Q(X=i) In machine learning usage, P is the actual (ground truth) distribution, and Q is the predicted distribution. All the functions you listed are just helper functions which accepts different ways to represent P and Q. There are basically 3 … Read more
Do the averaging of your measure in Python and create a new Summary object for each mean. Here is what I do: accuracies = [] # Calculate your measure over as many batches as you need for batch in validation_set: accuracies.append(sess.run([training_op])) # Take the mean of you measure accuracy = np.mean(accuracies) # Create a new … Read more
Update (2018/03/19): I wrote a blog post detailing how to implement triplet loss in TensorFlow. You need to implement yourself the contrastive loss or the triplet loss, but once you know the pairs or triplets this is quite easy. Contrastive Loss Suppose you have as input the pairs of data and their label (positive or … Read more
tf.data builds a performance model of the input pipeline and runs an optimization algorithm to find a good allocation of its CPU budget across all parameters specified as AUTOTUNE. While the input pipeline is running, tf.data tracks the time spent in each operation, so that these times can be fed into the optimization algorithm. The … Read more
Turns out I can use Dataset.map if I make the generator super lightweight (only generating meta data) and then move the actual heavy lighting into a stateless function. This way I can parallelise just the heavy lifting part with .map using a py_func. Works; but feels a tad clumsy… Would be great to be able … Read more
I think you want tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope=”my_scope”). This will get all variables in a scope. To pass to an optimizer you do not want all variables you would just want the trainable variables. Those are also kept in a default collection, which is tf.GraphKeys.TRAINABLE_VARIABLES.