Try android:includeFontPadding="false" to see if it helps. In my experience that will help a little bit, but there’s no way of reducing the TextView dimensions to the exact pixel-perfect text size.
The only alternative, which may or may not give better results, is to cheat a bit and hard-wire the dimensions to match the text size, e.g. "24sp" instead of "wrap_content" for the height.