I was able to solve this today and wanted to post the solution in case others run into the same issue.
It turns out that the problem was related to SNI after all. I found this ticket on nginx.org:
https://trac.nginx.org/nginx/ticket/229
Which led me to the proxy_ssl_server_name directive:
http://nginx.org/r/proxy_ssl_server_name
By setting to “on” in your config, you’ll be able to proxy to upstream hosts using SNI.
Thanks to all who commented with suggestions!