The solution for me was to use puttygen to export a .ppk version of my private key to OpenSSH as follows:
- Open PuttyGen
- Menu: Conversion -> Export OpenSSH key
- Save your new key (don’t forget to make it clear it is openssh)
This was weird seeing as my .ppk was converted from a private key generated by OpenSSH itself.
I think it has to do with the fact that my original key starts with
---- BEGIN SSH2 ENCRYPTED PRIVATE KEY ----
whereas the exported key starts with
-----BEGIN RSA PRIVATE KEY-----