Trying to use open(filename, ‘w’ ) gives IOError: [Errno 2] No such file or directory if directory doesn’t exist

You are correct in surmising that the parent directory for the file must exist in order for open to succeed. The simple way to deal with this is to make a call to os.makedirs.

From the documentation:

os.makedirs(path[, mode])

Recursive directory creation function. Like mkdir(), but makes all intermediate-level directories needed to contain the leaf directory.

So your code might run something like this:

filename = ...
dirname = os.path.dirname(filename)
if not os.path.exists(dirname):
    os.makedirs(dirname)
with open(filename, 'w'):
    ...