What exactly is “exit” in PowerShell?

It’s a reserved keyword (like return, filter, function, break).

Reference

Also, as per Section 7.6.4 of Bruce Payette’s Powershell in Action:

But what happens when you want a script to exit from within a function defined in that script? … To make this easier, Powershell has the exit keyword.

Of course, as other have pointed out, it’s not hard to do what you want by wrapping exit in a function:

PS C:\> function ex{exit}
PS C:\> new-alias ^D ex