No, you’re good to go removing the prior version’s directory which in this case is the .AndroidStudio1.3 directory.
On a side note, make sure you’re importing settings from 1.3 when upgrading to 1.4. That will spare the Android Studio from re-downloading all the existing project dependencies you worked with before.