Read a large zipped text file line by line in python

by

Python file objects provide iterators, which will read line by line. file.readlines() reads them all and returns a list – which means it needs to read everything into memory. The better approach (which should always be preferred over readlines()) is to just loop over the object itself, E.g:

import zipfile
with zipfile.ZipFile(...) as z:
    with z.open(...) as f:
        for line in f:
            print line

Note my use of the with statement – file objects are context managers, and the with statement lets us easily write readable code that ensures files are closed when the block is exited (even upon exceptions). This, again, should always be used when dealing with files.

Leave a Comment