The purpose of raw.github.com
is to allow users to view the contents of a file, so for text based files this means (for certain content types) you can get the wrong headers and things break in the browser.
When this question was asked (in 2012) SVGs didn’t work. Since then Github has implemented various improvements. Now (at least for SVG), the correct Content-Type headers are sent.
Examples
All of the ways stated below will work.
I copied the SVG image from the question to a repo on github in order to create the examples below
Linking to files using relative paths (Works, but obviously only on github.com / github.io)
Code

<img src="https://stackoverflow.com/questions/13808020/./controllers_brief.svg">
Result
See the working example on github.com.
Linking to RAW files
Code

<img src="https://raw.github.com/potherca-blog/StackOverflow/master/question.13808020.include-an-svg-hosted-on-github-in-markdown/controllers_brief.svg">
Result
Linking to RAW files using ?sanitize=true
Code

<img src="https://raw.github.com/potherca-blog/StackOverflow/master/question.13808020.include-an-svg-hosted-on-github-in-markdown/controllers_brief.svg?sanitize=true">
Result
Linking to files hosted on github.io
Code

<img src="https://potherca-blog.github.io/StackOverflow/question.13808020.include-an-svg-hosted-on-github-in-markdown/controllers_brief.svg">
Result
Some comments regarding changes that happened along the way:
-
Github has implemented a feature which makes it possible for SVG’s to be used with the Markdown image syntax. The SVG image will be sanitized and displayed with the correct HTTP header. Certain tags (like
<script>
) are removed.To view the sanitized SVG or to achieve this effect from other places (i.e. from markdown files not hosted in repos on http://github.com/) simply append
?sanitize=true
to the SVG’s raw URL. -
As stated by AdamKatz in the comments, using a source other than github.io can introduce potentially privacy and security risks. See the answer by CiroSantilli and the answer by DavidChambers for more details.
-
The issue to resolve this was opened on Github on October 13th 2015 and was resolved on August 31th 2017