remove object from array with just the object’s reference

Simply use Array.prototype.indexOf:

let index = MyArray.indexOf(TheObject);
if(index !== -1) {
  MyArray.splice(index, 1);
}

Keep in mind that if targeting IE < 9 you will need to introduce a polyfill for indexOf; you can find one in the MDN page.