I’ve gone through the GHC docs a bit, and there doesn’t seem to be a built-in way to remove the temporary files automatically — after all, GHC needs those intermediate files to build the final executable, and their presence speeds up overall compilation when GHC knows it doesn’t have to recompile a module.
However, you might find that setting the -outputdir option will help you out; that will place all of your object files (.o), interface files (.hi), and FFI stub files in the specified directory. It’s still “clutter,” but at least it’s not in your working directory anymore.