If you’re using GNOME, you can go to: Places → Connect to Server in Nautilus and choose SSH. If you have an SSH agent running and configured, no password will be asked!
(This is the same as sftp://root@servername/directory in Nautilus)
In Konqueror, you can simply type: fish://servername.
Per Mike R: In Ubuntu 14.04 (with Unity) it’s under Files → Connect to Server in the menu or Network → Connect to Server in the sidebar.