I found this page here explaining how scaling works with XSL-FO.
Essentially, this is the snippet that I used to make it work:
<fo:external-graphic
src="https://stackoverflow.com/questions/6220316/url("https://stackoverflow.com/questions/6220316/...")"
width="100%"
content-height="100%"
content-width="scale-to-fit"
scaling="uniform"
xsl:use-attribute-sets="img"/>
I found that width="100%"
was the missing piece in your own attempts. Hope this helps future visitors.