Found it!
import tensorflow as tf
for example in tf.python_io.tf_record_iterator("data/foobar.tfrecord"):
print(tf.train.Example.FromString(example))
You can also add:
from google.protobuf.json_format import MessageToJson
...
jsonMessage = MessageToJson(tf.train.Example.FromString(example))