You can achieve that using a ssh config file.
First create a file inside your ~/.ssh
folder named config
, you can use some command like the following
$ nano ~/.ssh/config
Then, the content of the file should have the location of your key based on each host name. for example:
Host github.com
IdentityFile ~/myPublicKeyFolder/myGitHubFile
Host heroku.com
IdentityFile ~/myPublicKeyFolder/myHerokuFile
So, when git tries to access each host it will follow the rules inside this config file based on the git host your trying to reach