ToggleButton in C# WinForms

You can just use a CheckBox and set its appearance to Button:

CheckBox checkBox = new System.Windows.Forms.CheckBox(); 
checkBox.Appearance = System.Windows.Forms.Appearance.Button;