I was able to kill my persistent vshost process by following these steps (VS2010):
- open the properties of my executable project
- in the “Debug” tab, uncheck “Enable the Visual Studio hosting process”
- save the project file
That was it, the process stopped, and there was no need to restart Visual Studio.