The problem was that another compile object was also dependent on the first level dependency that had the guava-jdk5 as a transitive dependency, so this other object was bringing in the unwanted module.
I was able to finally see this using
./gradlew -q :app:dependencyInsight --dependency guava --configuration compile
and could exclude it using
configurations {
compile.exclude module: 'guava-jdk5'
}