Another one but very similar cause of the automatic rebuild and restart is Auto build and refresh browser after saving changes option under the Tools -> Options -> Projects and Solutions -> ASP .NET Core settings.
Just set it to None or any other option than Auto build and refresh browser after saving changes.
