I would probably add a retry mechanism within the client. That way not only does it help in the “starting up from Visual Studio” case – it also helps if the server happens to be restarting while the real client connects. The fact that the server is on a faster machine doesn’t mean the server will never need to restart, does it?
Indeed, you may well want to add this retry mechanism in such a way that the client can recover even if the server is restarted while it’s connected. It depends on what the project is doing, of course.