I think JAVA_HOME is the best you can do. The command-line tools like java and javac will respect that environment variable, you can use /usr/libexec/java_home -v '1.7*' to give you a suitable value to put into JAVA_HOME in order to make command line tools use Java 7.
export JAVA_HOME="`/usr/libexec/java_home -v '1.7*'`"
But standard double-clickable application bundles don’t use JDKs installed under /Library/Java at all. Old-style .app bundles using Apple’s JavaApplicationStub will use Apple Java 6 from /System/Library/Frameworks, and new-style ones built with AppBundler without a bundled JRE will use the “public” JRE in /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home – that’s hard-coded in the stub code and can’t be changed, and you can’t have two different public JREs installed at the same time.
Edit: I’ve had a look at VisualVM specifically, assuming you’re using the “application bundle” version from the download page, and this particular app is not an AppBundler application, instead its main executable is a shell script that calls a number of other shell scripts and reads various configuration files. It defaults to picking the newest JDK from /Library/Java as long as that is 7u10 or later, or uses Java 6 if your Java 7 installation is update 9 or earlier. But unravelling the logic in the shell scripts it looks to me like you can specify a particular JDK using a configuration file.
Create a text file ~/Library/Application Support/VisualVM/1.3.6/etc/visualvm.conf (replace 1.3.6 with whatever version of VisualVM you’re using) containing the line
visualvm_jdkhome="`/usr/libexec/java_home -v '1.7*'`"
and this will force it to choose Java 7 instead of 8.