The trick lies with this optimization:
unicode-range: U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;
With this, the browser knows whether it needs to download the font, depending on the characters it just loaded in the html. Chrome is currently the only one with full support for this.
MDN lists Firefox 36+ as completely ignoring this spec, therefore Google Fonts has to serve it a minimal font-face specification. The same happens with Safari.
EDIT: In Nov 2022 browser support is over 98%.
Anyway, that was an interesting 30 minute journey on the internet which I hope helps any further internet travellers. This should lend weight towards why you could use Google Fonts as a CDN for hosting fonts for optimal performance as opposed to serving it yourself.