Hi,
I've recently been helping out with a server performance problem, for a
club I'm a member of. The site runs a vBulletin forum (PHP+MySQL). The
server also hosts a few other domains but these are personal blogging
sites and don't receive too many hits.
In a nutshell, I'm looking for hints, tips and experiences from people
who've dealt with web servers that receive a high volume of hits.
Last month the site transfered 250Gb of data and averages 2 million page
views per month. The bandwidth consumption has been increasing month on
month and the club is constantly growing. The forum has 11,000 users, of
which 4,331 are active. On average there are 200 users online but can
sometimes peak at over 500.
Over the last few weeks the site has become very unstable. Apache often
falls over.
The first issue is funds. The club is non-profit so the members who
choose to sign up as "full members" fund things such as hosting costs,
merchandise, shows etc. Some revenue is generated by advertising. A nice
big beefy server with 4Gb of RAM or even two servers (web server and
database server running something like memcached) would be ideal but
isn't currently economically feasible.
The current server is an unmanaged dedicated server:
AMD Athlon(tm) 64 X2 Dual Core Processor 4000+
960264 kB RAM
3919840 kB swap (appears to be spread over two partitions)
Appears to be running RAID5 (Linux MD)
Apache 2.2.3
PHP 5.1.6
MySQL 5.0.45
As mentioned, I've only just stepped in to lend a hand. I wasn't
involved in the purchasing of the hosting package. The hosting company
pre-installed and configured it, including the drive configuration, swap
configuration etc. I know that this isn't ideal but there's not much I
can do unfortunately.
I'm now looking at optimising the server configuration. Someone else is
looking at optimising PHP code. My first port of call was to observe
what happens when everything starts to go belly up. I used top, vmstat
and of course ps to monitor what was going on. The number of Apache
processes would slowly start to increase, physical memory decreases and
then Apache starts to heavily swap in/out. The disks start to thrash and
finally Apache will stop responding for anything up to 5 minutes at a time.
So, my current proposal for Apache is:
1) Use Apache's prefork MPM:
The site uses PHP which isn't considered thread safe, so I'm loathed to
use the worker MPM. I did consider using the worker MPM with
fastcgi/fcgid but again, I read stories of lots of defunct processes
appearing.
2) Remove any un-needed Apache modules to reduce the memory footprint.
3) Use mod_deflate to reduce bandwidth (trade off is server load?).
And then in terms of tuning (based on using the prefork MPM):
1) Calculate MaxClients based on:
(Physical RAM - RAM in use by other processes) / Average Apache process size
I've actually written a (hacky) shell script to do this for me. You can
see it here:
http://0wned.it/temp/calcmaxclients.sh.txt
2) KeepAlive On but set the KeepAliveTimeout to a low value, so that a
client doesn't use a process for massive amount of time.
3) Lower the Timeout value.
4) Calculate MaxRequestsPerChild based on:
(Requests per 24 hours / Apache processes) which will hopefully mean
processes don't linger for more than 24 hours.
I'm also looking in to implementing a PHP caching system like APC.
So if anyone agrees or disagrees with the above, has any tips, hints or
experiences with Apache performance tuning for sites that receive lots
of hits, I'd be grateful to hear from you :)
Regards,
David.
--
.''`. David Ramsden
: :' : http://0wned.it/
`. `'` PGP key ID: 3454B217 on wwwkeys.eu.pgp.net
`- Debian - Because it works (tm).
