How to delete my follower at github?

Removing a follower on Github in fact is possible with this workaround:

Block the user who watches you, as described in the github help.

Then unblock him again.

After that he will not be blocked anymore and he will also no longer follow you.