Several answers suggest converting to Lab color space – which is probably a good approach for more complex color manipulation.
But if you simply need a quick way to take the average of two colors, this can be done in the RGB space. You just have to mind a caveat: You must square the RGB values before averaging them, and then take the root of the result. (If you simply take the average, the result will tend to be too dark.)
Like this:
NewColor = sqrt((R1^2+R2^2)/2),sqrt((G1^2+G2^2)/2),sqrt((B1^2+B2^2)/2)
Here’s a great vid which explains why this method is efficient: https://www.youtube.com/watch?v=LKnqECcg6Gw