Check the properties of your main Window project.. You should be able to set an icon there.
Update:
Is it such a problem that your icon does not show in debug mode? As long as it works when you deliver the program to your customer, its all alright, not?
On a side note: You could check some things though. Perhaps your icon is not included in your project, or it isnt copied when building to your Debug folder?
Update 2:
You also need to set the Main Form’s Icon for it to show in Debug. If you also set the icon for the main form, it will display in the Taskbar during Debug / Runtime.