scikit-image has implemented a working version of downsampling here, although they shy away from calling it downsampling for it not being a downsampling in terms of DSP, if I understand correctly:
http://scikit-image.org/docs/dev/api/skimage.measure.html#skimage.measure.block_reduce
but it works very well, and it is the only downsampler that I found in Python that can deal with np.nan in the image. I have downsampled gigantic images with this very quickly.