We need to write down the loss function. For example, we can use basic mean square error as our loss function for predicted y and target y_:
loss_mse = 1/n(Sum((y-y_)^2))
There are basic functions for tensors like tf.add(x,y), tf.sub(x,y), tf.square(x), tf.reduce_sum(x), etc.
Then we can define our loss function in Tensorflow like:
cost = tf.reduce_mean(tf.square(tf.sub(y,y_)))
Note: y and y_ are tensors.
Moreover, we can define any other loss functions if we can write down the equations. For some training operators (minimizers), the loss function should satisfy some conditions (smooth, differentiable …).
In one word, Tensorflow define arrays, constants, variables into tensors, define calculations using tf functions, and use session to run though graph. We can define whatever we like and run it in the end.