Horible performance

ffeingol

ffeingol

Well-Known Member
#1
#1
Some background,

We're running LSWS 3.2.1 On a Dual Xeon 3.0 ghz, 2 gb of memory. It's a cPanel box and it was running Apache. PHP is a fresh build of 4.4.7 with eAccelerator.

The PHP LSAPI is set:

Max Connections: 30
PHP_LSAPI_CHILDREN=30
Initial Request Timeout: 60
Process Soft Limt: 220
Process Hard Limit: 220

We also have Throttling set to 20 / sec for static and 1 / sec for dynamic

On the real-time reort for External Applications we see:

Max Conn: 60
Eff Max: 60

And a lot of time there are processes in the wait queue. This is not a particularly busy server. I'm watching the VHOST Requests: _Server graph and it's maxing at 30 requests per second.

Any clues, thoughs, suggestions on why PHP is running so slowly? We have a couple of clients that actively monitor their sites and it's noticeably slower since we put in LSWS. I want to give it a chance, but at this rate, we're going to have to back it out and go back to Apache.

TIA,

Frank
 
M

mistwang

LiteSpeed Staff
#2
#2
First, you need to verify that eAccelerator works properly.
Then, you can increase the "Max Connections" and "PHP_LSAPI_CHILDREN" to 40, to keep the Wait Q at zero for the most of the time.

We recommend PHP suEXEC, it assigns dedicate PHP processes for each account, so you can easily identify a site which is causing problem. This way, one site was hitting hard will not affect other sites.

The slow down might because one site got higher than normal traffic like you said in other thread. You can switch back and forth and compare between Apache and LSWS, and we are interested to know the result as well.
 
ffeingol

ffeingol

Well-Known Member
#3
#3
mistwang said:
First, you need to verify that eAccelerator works properly.
Then, you can increase the "Max Connections" and "PHP_LSAPI_CHILDREN" to 40, to keep the Wait Q at zero for the most of the time.
Click to expand...
Done. I'd also love to get the cacti plugin working but we don't have php V5 on our cacti box, so I can't. That (I think) would help at least know what we're dealing with.

We recommend PHP suEXEC, it assigns dedicate PHP processes for each account, so you can easily identify a site which is causing problem. This way, one site was hitting hard will not affect other sites.
Click to expand...
Unfortunately this is not something that can be easily done (unless I'm totally missing things). We have dozens of site (for example) that run gallery. All the uploaded images are owned by "nobody" as well as various .dat files. We'd have to chown all those to the site owner id and keep a list of all of them if we needed to backup to Apache. That is a huge task.

The slow down might because one site got higher than normal traffic like you said in other thread. You can switch back and forth and compare between Apache and LSWS, and we are interested to know the result as well.
Click to expand...
Yes, it may be but the anti-dos features of LSWS is what we were hoping would "smooth" all this out. Believe me, I totally want this solution to work as it opens so many possibilities.

TIA,

Frank
 
ffeingol

ffeingol

Well-Known Member
#4
#4
Sorry, I forgot to mention, eAccelerator is working fine. From the eAcceletaor control page:

Code: 
Caching enabled:  yes
Optimizer enabled:  yes
Memory usage:  99.99% (32.00MB/ 32.00MB)
Free memory:  0.00MB
Cached scripts:  410
Removed scripts:  0
Cached keys:  0
 
M

mistwang

LiteSpeed Staff
#5
#5
ffeingol said:
Unfortunately this is not something that can be easily done (unless I'm totally missing things). We have dozens of site (for example) that run gallery. All the uploaded images are owned by "nobody" as well as various .dat files. We'd have to chown all those to the site owner id and keep a list of all of them if we needed to backup to Apache. That is a huge task.
Click to expand...
If the uploaded images are world readable, then you don't have to change the ownership.


Yes, it may be but the anti-dos features of LSWS is what we were hoping would "smooth" all this out. Believe me, I totally want this solution to work as it opens so many possibilities.
Click to expand...
Anti-DoS feature will help defending attacks, traffic from valid users still need to to be served as usual. So, you will definitely see different load when serving 1000 users than serving 500 users. Switching between Apache and LiteSpeed while the server receiving same amount of traffic and compare.
 
M

medialayer

Active Member
#6
#6
Try using Apache Benchmark with and without LSWS and see how many "requests per second" you are able to obtain. On a standard cPanel install, ab is located in /usr/local/apache/bin/ab. Please share the results too :)
 
ffeingol

ffeingol

Well-Known Member
#7
#7
I can certainly do that. It seems to mostly be a php/mysql issue and our backups may be playing a wee part of the issue also.

Any suggestions for ab parms? I'd like a good test, but I don't want to totally knock the server over either (it's a prod box).

TIA,

Frank
 
M

medialayer

Active Member
#8
#8
ffeingol said:
I can certainly do that. It seems to mostly be a php/mysql issue and our backups may be playing a wee part of the issue also.

Any suggestions for ab parms? I'd like a good test, but I don't want to totally knock the server over either (it's a prod box).

TIA,

Frank
Click to expand...
Try tuning-primer to help optimize your MySQL configuration if you haven't already.

You can start with very basic parameters, I usually do something like "ab -c 5 -n 10" for a small test and expand from there. I don't think you'll have a problem with "ab -c 20 -n 100". Do a few tests and get an average.

edit: Also, what are you doing as far as gzip compression? If you are doing dynamic compression I would try disabling it, since many PHP applications (i.e. vBulletin) already have this built in. A better way would of course be to disable the compression in those applications themselves, but I guess you can't really force your customers to change that. If you are using the default of 6, I tend to use 1 which seems most beneficial to me (I'm more CPU-conscious than I am about 2-3% increase in file size).
 
Last edited:
ffeingol

ffeingol

Well-Known Member
#9
#9
Try tuning-primer to help optimize your MySQL configuration if you haven't already.
Click to expand...
Its more bad client code/sql calls that mysql tuning. It's hard to get clients to change their code, but they are usually happy if we get them better sql that speeds up their sites.

You can start with very basic parameters, I usually do something like "ab -c 5 -n 10" for a small test and expand from there. I don't think you'll have a problem with "ab -c 20 -n 100". Do a few tests and get an average.
Click to expand...
If I understand this coorect -c 20 -n 100 is 20 clients doing 100 calls (so 5 calls each)? I don't think that either Apache or LSWS would even blink at that.

edit: Also, what are you doing as far as gzip compression? If you are doing dynamic compression I would try disabling it, since many PHP applications (i.e. vBulletin) already have this built in. A better way would of course be to disable the compression in those applications themselves, but I guess you can't really force your customers to change that. If you are using the default of 6, I tend to use 1 which seems most beneficial to me (I'm more CPU-conscious than I am about 2-3% increase in file size).
Click to expand...
We can't turn off gzip (don't want the bandwidth bill going through the roof) but we plan on adjusting the compression level back to 1. We have also found that the cpu utilization goes way up past 1 but the additional compression is nominal.

Frank
 
M

medialayer

Active Member
#10
#10
If I understand this coorect -c 20 -n 100 is 20 clients doing 100 calls (so 5 calls each)? I don't think that either Apache or LSWS would even blink at that.
Click to expand...
Nope, they won't, but it should be a start for measuring performance. Just keep increasing as you feel comfortable. It depends on the script you are benchmarking as well.
 
M

mistwang

LiteSpeed Staff
#11
#11
medialayer said:
what are you doing as far as gzip compression? If you are doing dynamic compression I would try disabling it, since many PHP applications (i.e. vBulletin) already have this built in. A better way would of course be to disable the compression in those applications themselves, but I guess you can't really force your customers to change that. If you are using the default of 6, I tend to use 1 which seems most beneficial to me (I'm more CPU-conscious than I am about 2-3% increase in file size).
Click to expand...
If PHP send back compressed content, LiteSpeed will send it as is without trying to compress it again. So, it does not hurt to leave it on.
Yes, use compression level 1 if you want to save some CPU cycles.
 
M

medialayer

Active Member
#14
#14
Frank,

I wanted to share my test results:

I did ab -c 5 -n 100 against a phpinfo() on both Apache & LSWS. Same server, same php.ini, same PHP version, same PHP configure command (besides --with-litespeed, etc.):

Apache mod_php5 average:
Requests per second: 714.29 [#/sec]

LSWS average:
Requests per second: 1449.28 [#/sec]

Both have eAccelerator.
 
ffeingol

ffeingol

Well-Known Member
#15
#15
Our results bounce around way too much to be meaningful with those parms:

LSWS:

Requests per second: 316.58 [#/sec] (mean)
Requests per second: 251.63 [#/sec] (mean)
Requests per second: 215.95 [#/sec] (mean)

Apache:

Requests per second: 674.75 [#/sec] (mean)
Requests per second: 127.43 [#/sec] (mean)
Requests per second: 239.89 [#/sec] (mean)

Exact same php/php.ini yadda, yadda, yadda.

Frank
 
You must log in or register to reply here.
Top