Go to File -> Settings -> Version control -> Confirmation -> When files are created You’re probably looking for “Add silently“.
Note:
In IntelliJ 6 or earlier versions this is done with File -> Settings -> Version control -> General Settings -> Add silently