In vim, how do I go back to where I was before a search?

Ctrl+O takes me to the previous location. Don’t know about location before the search.

Edit: Also, `. will take you to the last change you made.