I just posted an answer to a similar StackOverflow question.
Google calls this Hybrid Apps and explains how an “Android app obtains offline access for Web back-end”.
The gist of it is that you’ll have to pass a massaged scope string into GoogleAuthUtil.getToken in order to get it to return an Authorization Code (not an OAuth2 Token). That Authorization Code can be passed from your mobile app to your server and be exchanged for an OAuth2 Token and Refresh Token, according to this schematic.
The scope parameter needs to look something like this:
oauth2:server:client_id:<your_server_client_it>:api_scope:<scope_url_1> <scope_url_2> ...