I finally ended using the following :
bower install --save http://twitter.github.com/bootstrap/assets/bootstrap.zip
Seems cleaner to me since it doesn’t clone the whole repo, it only unzip the required assests.
The downside of that is that it breaks the bower philosophy since a bower update will not update bootstrap.
But I think it’s still cleaner than using bower install bootstrap and then building bootstrap in your workflow.
It’s a matter of choice I guess.
Update : seems they now version a dist folder (see: https://github.com/twbs/bootstrap/pull/6342), so just use bower install bootstrap and point to the assets in the dist folder