Each row sharey individually?

The pyplot.subplots documentation describes the 'col' and 'row' options for the sharex and sharey kwargs. In particular, I think you want:

fig, axes = plt.subplots(nrows=2, ncols=2, sharex='col', sharey='row')