Since there hasn’t been much traffic with this post I guess I will post a solution:
I added this CSS rule:
svg use {
pointer-events: none;
}
This is not ideal but it keeps IE 11 from locking up and that is all I am required to support with this project. However I would like for this post to help others in the future who might encounter this bug and do need to support older versions of IE. If anyone is willing to take the time to come up with a more robust solution I will accept that as the answer to this post.
Also should I file a bug report to microsoft regarding this issue?