How to uninstall Rust that was installed via rustup?

To uninstall rustc, rustup and cargo from my Ubuntu 16.04 installation, I did:

rustup self uninstall

and it worked.