Yes, this function is hard to understand, until you get the point. In its simplest form, it is similar to tf.gather. It returns the elements of params according to the indexes specified by ids. For example (assuming you are inside tf.InteractiveSession()) params = tf.constant([10,20,30,40]) ids = tf.constant([0,1,2,3]) print tf.nn.embedding_lookup(params,ids).eval() would return [10 20 30 40], … Read more
The most important thing to realize about TensorFlow is that, for the most part, the core is not written in Python: It’s written in a combination of highly-optimized C++ and CUDA (Nvidia’s language for programming GPUs). Much of that happens, in turn, by using Eigen (a high-performance C++ and CUDA numerical library) and NVidia’s cuDNN … Read more
A training step is one gradient update. In one step batch_size examples are processed. An epoch consists of one full cycle through the training data. This is usually many steps. As an example, if you have 2,000 images and use a batch size of 10 an epoch consists of: 2,000 images / (10 images / … Read more
If you read the Keras documentation entry for Dense, you will see that this call: Dense(16, input_shape=(5,3)) would result in a Dense network with 3 inputs and 16 outputs which would be applied independently for each of 5 steps. So, if D(x) transforms 3 dimensional vector to 16-d vector, what you’ll get as output from … Read more
You can also set the environment variable to CUDA_VISIBLE_DEVICES=”” without having to modify the source code.
Here is what I do to avoid the issues of making the remote server accept your local external IP: when I ssh into the machine, I use the option -L to transfer the port 6006 of the remote server into the port 16006 of my machine (for instance): ssh -L 16006:127.0.0.1:6006 olivier@my_server_ip What it does … Read more
UPDATE 8/1730/2020: TF 2.3 has finally done it: all cases run as fast, or notably faster, than any previous version. Further, my previous update was unfair to TF; my GPU was to blame, has been overheating lately. If you see a rising stem plot of iteration times, it’s a reliable symptom. Lastly, see a dev’s … Read more
According to TF 1:1 Symbols Map, in TF 2.0 you should use tf.compat.v1.Session() instead of tf.Session() https://docs.google.com/spreadsheets/d/1FLFJLzg7WNP6JHODX5q8BDgptKafq_slHpnHVbJIteQ/edit#gid=0 To get TF 1.x like behaviour in TF 2.0 one can run import tensorflow.compat.v1 as tf tf.disable_v2_behavior() but then one cannot benefit of many improvements made in TF 2.0. For more details please refer to the migration guide … Read more
To get started, you should download the source code from Github, by following the instructions here (you’ll need Bazel and a recent version of GCC). The C++ API (and the backend of the system) is in tensorflow/core. Right now, only the C++ Session interface, and the C API are being supported. You can use either … Read more
You can easily get the outputs of any layer by using: model.layers[index].output For all layers use this: from keras import backend as K inp = model.input # input placeholder outputs = [layer.output for layer in model.layers] # all layer outputs functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs] # evaluation functions # Testing test … Read more