OK so after more testing, turns out you just have to set:
searchController.active = false
// or swift 4+
searchController.isActive = false
This is the first thing I tried but I called it in one of the UISearchControllerDelegate methods which didn’t work (probably should have called it with dispatch_async (halbano’s answer seems to confirm that)).
Anyway, since I couldn’t find that answer online, I’m answering my own question, I hope that it’ll help someone.