The most common reason for this in the case of Apache is the usage of DNS Reversal Lookup. What this means is that the server tries to figure out what the name of your machine is, each time you make a request. This can take several seconds, and that explains why you have a long WAIT time and then a very quick load, because the matter is not about bandwidth.
The obvious solution for this is to disable hostnamelookup in /etc/httpd/conf/httpd.conf
HostnameLookups Off
However…this is usually NOT enough. The fact is that in many cases, apache still does a reversal lookup even when you have disabled host name lookup, so you need to take a careful look at each line of your apache config. In particular, one of the most common reasons for this are LOGS. By default on many red hat – centos installations, the log format includes %h which stands for “hostname”, and requires apache to do a reverse lookup. You can see this here:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
You should change those %h for %a to solve this problem.