As it says in the documentation,
In order to make a
for
loop the most efficient way of looping over the lines of a file (a very common operation), thenext()
method uses a hidden read-ahead buffer.
And you can see by looking at the implementation of the csv
module (line 784) that csv.reader
calls the next()
method of the underlyling iterator (via PyIter_Next
).
So if you really want unbuffered reading of CSV files, you need to convert the file object (here sys.stdin
) into an iterator whose next()
method actually calls readline()
instead. This can easily be done using the two-argument form of the iter
function. So change the code in test_reader.py
to something like this:
for row in csv.reader(iter(sys.stdin.readline, '')):
print("Read: ({}) {!r}".format(time.time(), row))
For example,
$ python test_writer.py | python test_reader.py
Read: (1388776652.964925) ['R0', '$']
Read: (1388776653.466134) ['R1', '$$']
Read: (1388776653.967327) ['R2', '$$$']
Read: (1388776654.468532) ['R3', '$$$$']
[etc]
Can you explain why you need unbuffered reading of CSV files? There might be a better solution to whatever it is you are trying to do.