You can use the UIView method isDescendantOfView:
if mySubview.isDescendant(of: someParentView) {
mySubview.removeFromSuperview()
} else {
someParentView.addSubview(mySubview)
}
You may also need to surround everything with if mySubview != nil depending on your implementation.