The comment by MK pointed me in the right direction.
In the case of Apache 2.4 and up, there are different defaults and a new directive.
I am running Apache 2.4.6, and I had to add the following directives to get it working:
SSLProxyEngine on
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off