You can create a double-headed arrow using the annotate method with blank text annotation and setting the arrowprops dict to include arrowstyle="<->" as shown below:
import matplotlib.pyplot as plt
plt.annotate(s="", xy=(1,1), xytext=(0,0), arrowprops=dict(arrowstyle="<->"))
plt.show()
