You are right in that the color argument changes all the plot elements. However, if you read the last bit of the relevant sentence in the documentation:

color : matplotlib color

Color to apply to all plot elements; will be superseded by colors
passed in scatter_kws or line_kws.

Therefore, using scatter_kws or line_kws we can change the color of them individually. Taking the first example given in the documentation:

import seaborn as sns

tips = sns.load_dataset("tips")
ax = sns.regplot(x="total_bill", y="tip", data=tips,
                 scatter_kws={"color": "black"}, line_kws={"color": "red"})

plt.show()

Gives: