You can safely delete the ~/.gradle directory. It is created by the Gradle wrapper to store and cache downloaded files, so it will just repopulate the folder with the files necessary for future builds.
Note that if you made any changes yourself in that directory, you may need to keep those files, such as a ~/.gradle/gradle.properties file.