First do:
git remote add heroku git@heroku.com:{my-project-name}.git
Where you replace {my-project-name} with the name of the Heroku application as it appears in your Heroku account. For example, if your Heroku account says you have an application named flowing-water-397, then the line would be:
git remote add heroku git@heroku.com:flowing-water-397.git
Then Git and the Heroku gem will know that this Git repo is connected to a Heroku.com application. Then you can do things such as:
git push heroku master
heroku open
Finally, learn a little bit more about Git Remotes.