(updated 2019, removed unnecessarily costly method)
Just do:
from IPython.display import Video
Video("test.mp4")
If you get an error No video with supported format or MIME type found, just pass embed=True to the function: Video("test.mp4", embed=True).
Or if you want to use the HTML element:
from IPython.display import HTML
HTML("""
<video alt="test" controls>
<source src="test.mp4" type="video/mp4">
</video>
""")