Right! This seems to stop it for good – simply rename the following package to something other than IDESubversion.ideplugin (I’ve renamed mine IDESubversion-DISABLED.ideplugin)…
Xcode 5.1+
This is finally a built-in Xcode preference – see @MANIAK_dobrii’s answer for details.
Xcode 5/Xcode 4.3+:
/Applications/Xcode.app/Contents/PlugIns/IDESubversion.ideplugin
Xcode 4.0 – 4.2:
/Developer/Library/Xcode/PrivatePlugIns/IDESubversion.ideplugin
Note that in Xcode 4.0 & 4.1 (and possibly 4.2) renaming the extension seemed to work fine. From Xcode 4.3 however, you might need to leave the extension intact, and rename the name portion of the bundle instead (to avoid the error on launch that people have described in the comments below).
Tip o’ the hat to: Remove git from project in Xcode 4