You can at least try and mitigate the issue, with a shallow clone (meaning not cloning the all history):
git clone --depth 1 <repository>
Make sure to have a git 1.9+, as I explained in “Is git clone --depth 1 (shallow clone) more useful than it makes out?”.
Note: Git 2.5 (Q2 2015) even supports a single fetch commit!See “Pull a specific commit from a remote git repository”.