You may find OpenJDK 6 and 7 binaries for Windows in openjdk-unofficial-builds github project.
Update: OpenJDK 8 and 11 LTS binaries for Windows x86_64 can be found in ojdkbuild github project.
Disclaimer: I’ve built them myself.
Update (2019): OpenJDK Updates Project Builds for 8 and 11 are available now.