You can “remove” the app badge icon by setting it to 0:
Swift < 3.0
UIApplication.sharedApplication().applicationIconBadgeNumber = 0
Swift 3.0+
UIApplication.shared.applicationIconBadgeNumber = 0
This question shows when you can use it: How to clear push notification badge count in iOS?