Can I force abstract methods to be protected when someone overrides them?

No. A subclass can always make a method more public.

Even if they couldn’t do this with the method you have in your class, they could always write:

public void callsMethodIDontWantExposed() {
    methodIDontWantExposed();
}

… so you’d be in the same situation.