Unfortunately Google does not offer an easy way to directly download fonts. You can browse the git repository to search for the file you want, though there are only TTF files on GitHub available.
Also, do not directly link to the GitHub hosted fonts in your CSS! GitHub serves the files with the wrong mime type, which causes issues in some browsers.
While there is not a mainstream CDN for all the formats, you can use http://google-webfonts-helper.herokuapp.com to download the font files and host them yourselves.