This is actually a comment about the answer suggesting using apt-get but I don’t have enough reputation points to leave one.
If you use virtualenv a lot, then installing the python-packages through apt-get can become a pain, as you can get mysterious errors when the python packages installed system-wide and the python packages installed in your virtualenv try to interact with each other. One thing that I have found that does help is to use the build-dep feature. To build the matplotlib dependencies, for example:
sudo apt-get build-dep python-matplotlib
And then activate your virtual environment and do pip install matplotlib. It will still go through the build process but many of the dependencies will be taken care of for you.
This is sort what the cran repositories suggest when installing R packages in ubuntu.