out of memory issue in installing packages on Ubuntu server

Extend your RAM by adding a swap file:
http://www.cyberciti.biz/faq/linux-add-a-swap-file-howto/

a swap file is a file stored on the computer hard drive that is used
as a temporary location to store information that is not currently
being used by the computer RAM. By using a swap file a computer has
the ability to use more memory than what is physically installed in
the computer
http://www.computerhope.com/jargon/s/swapfile.htm

In Short:

1. Login as root: su - or execute the commands with sudo in front
2. dd if=/dev/zero of=/swapfile1 bs=1024 count=524288
3. mkswap /swapfile1
4. chown root:root /swapfile1
5. chmod 0600 /swapfile1
6. swapon /swapfile1

Now the swap file will be activated temporarily, but will be gone after reboot.
You should have enough RAM for your installing process

To Remove the File:

1. swapoff -v /swapfile1
2. rm /swapfile1