Server Load!
- Thread starter masood_y
- Start date
You CPU load is not high, it's only 4%. Load average (5min) is a bit high, but not abnormal.
try shell commands:
top
it shows realtime values. quit with q. Press s +2 to set 2 second sampling interval.
ps auwx
It'll show all processes and what is running them.
free -t -m
shows you memory usage in MB:s. Normally nearly all memory should be used in Linux, but swap only by a little amount (2-5% of swap size). If you're swapping a lot it will kill performance.
It's hard to say anything specific as we don't know what scripts, databases, traffic do you run, and how.
try shell commands:
top
it shows realtime values. quit with q. Press s +2 to set 2 second sampling interval.
ps auwx
It'll show all processes and what is running them.
free -t -m
shows you memory usage in MB:s. Normally nearly all memory should be used in Linux, but swap only by a little amount (2-5% of swap size). If you're swapping a lot it will kill performance.
It's hard to say anything specific as we don't know what scripts, databases, traffic do you run, and how.
Last edited:
I see most of cpu load is for mysql:
Pid Owner Priority Cpu % Mem % Command
13639 mysql 0 81.8 4.1 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/lib/mysql/giga.hostjewelry.info.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
How can resolve this issue?
Pid Owner Priority Cpu % Mem % Command
13639 mysql 0 81.8 4.1 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/lib/mysql/giga.hostjewelry.info.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
How can resolve this issue?
I dont know what is IOWAIT, here is my curren mysql configuration.
Can you please check it?
Can you please check it?
Code:
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
skip-locking
skip-networking
safe-show-database
query_cache_limit=1M
query_cache_size=256M
query_cache_type=1
max_user_connections=200
max_connections=500
interactive_timeout=10
wait_timeout=20
connect_timeout=20
thread_cache_size=128
key_buffer=256M
join_buffer=1M
max_connect_errors=20
max_allowed_packet=16M
table_cache=1024
record_buffer=1M
sort_buffer_size=4M
read_buffer_size=4M
read_rnd_buffer_size=4M
thread_concurrency=4
myisam_sort_buffer_size=64M
server-id=1
log_slow_queries=/var/log/mysql-slow-queries.log
long_query_time=2
collation-server=latin1_general_ci
[safe_mysqld]
err-log=/var/log/mysqld.log
pid-file=/var/lib/mysql/mysql.pid
open_files_limit=8192
[mysqldump]
quick
max_allowed_packet=16M
[mysql]
no-auto-rehash
#safe-updates
[isamchk]
key_buffer=32M
sort_buffer=32M
read_buffer=16M
write_buffer=16M
[myisamchk]
key_buffer=32M
sort_buffer=32M
read_buffer=16M
write_buffer=16M
[mysqlhotcopy]
interactive-timeout
I will be the IOWAIT guy.... ha ha..
What is your servers IOWAIT?
If high, mySQL will fall on it's face - with high load... and this will snowball, with excessive threads being created, and get worse and worse.
Check and fix your IOWAIT (in addition to optimizing mySQL my.cnf file).
Warm regards.
What is your servers IOWAIT?
If high, mySQL will fall on it's face - with high load... and this will snowball, with excessive threads being created, and get worse and worse.
Check and fix your IOWAIT (in addition to optimizing mySQL my.cnf file).
Warm regards.
As for noatime... it will help.. but it is actually minimal. I have done testing... only expect a very very slight performance gain.
You can google "noatime" and find many pages about how to enable this attribute.
As for optimizing your mySQL, download and install MONyog... 30 days is free... it is helpful.
http://www.webyog.com/en/
Let it run for a few days, and use the Monitors / Advisors page to see what you should optimize in your my.cnf.
Turn on your slow-log also, and get the users on your server who have slow queries to optimize them.
As for IOWAIT, SSH into your server and run top
It will give you and idea of your IOWAIT times.
Also, monitor your server with Wormly - and setup health monitoring - it will keep graphs about your server performance giving you data to reference when doing tweaks.
http://www.wormly.com/
Or hire an experienced SysAdmin... we offer these services if you need it.
http://www.mysitegroup.com
Warm regards.
You can google "noatime" and find many pages about how to enable this attribute.
As for optimizing your mySQL, download and install MONyog... 30 days is free... it is helpful.
http://www.webyog.com/en/
Let it run for a few days, and use the Monitors / Advisors page to see what you should optimize in your my.cnf.
Turn on your slow-log also, and get the users on your server who have slow queries to optimize them.
As for IOWAIT, SSH into your server and run top
It will give you and idea of your IOWAIT times.
Also, monitor your server with Wormly - and setup health monitoring - it will keep graphs about your server performance giving you data to reference when doing tweaks.
http://www.wormly.com/
Or hire an experienced SysAdmin... we offer these services if you need it.
http://www.mysitegroup.com
Warm regards.
When I go to LSWS main menu to view load summary, I have;
System Load 36 - over 50 sometimes
But RAM & CPU load showing low, such as;
RAM Usage = 24MB
CPU Usage = 3%
Why is this?
I have high traffic database site, how do I find out more details on what's causing the system load? Yest does not take much RAM... Does this look like I/O waits, any ideas?
System Load 36 - over 50 sometimes
But RAM & CPU load showing low, such as;
RAM Usage = 24MB
CPU Usage = 3%
Why is this?
I have high traffic database site, how do I find out more details on what's causing the system load? Yest does not take much RAM... Does this look like I/O waits, any ideas?
Last edited:
Here's my top, when "system load" showing as 70
top - 08:12:33 up 1 day, 4:11, 1 user, load average: 68.10, 70.86, 72.51
Tasks: 230 total, 68 running, 159 sleeping, 0 stopped, 3 zombie
Cpu0 : 7.5%us, 5.5%sy, 87.1%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 5.5%us, 6.0%sy, 88.5%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 4.0%us, 2.5%sy, 93.5%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 11.4%us, 7.5%sy, 80.1%ni, 0.0%id, 0.0%wa, 0.0%hi, 1.0%si, 0.0%st
Mem: 8103672k total, 7169728k used, 933944k free, 632792k buffers
Swap: 8393952k total, 0k used, 8393952k free, 4961600k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30845 nobody 17 1 115m 11m 5680 R 47.8 0.1 0:59.91 lsphp5
2694 mysql 15 0 904m 324m 4928 S 42.8 4.1 1182:21 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pi
1339 nobody 17 1 114m 9940 4728 R 38.3 0.1 0:11.29 lsphp5
1792 nobody 17 1 115m 10m 4668 R 21.4 0.1 0:05.21 lsphp5
31483 nobody 17 1 115m 10m 5108 R 20.9 0.1 0:48.48 lsphp5
825 nobody 17 1 113m 10m 4608 R 19.9 0.1 0:18.87 lsphp5
1904 nobody 17 1 112m 7844 2988 R 18.4 0.1 0:01.46 lsphp5
998 nobody 17 1 115m 10m 5008 R 16.9 0.1 0:23.12 lsphp5
1408 nobody 17 1 112m 8472 3284 R 16.4 0.1 0:06.03 lsphp5
1788 nobody 17 1 113m 9.9m 4300 R 16.4 0.1 0:03.36 lsphp5
30012 nobody 17 1 115m 11m 5628 R 15.9 0.1 1:23.07 lsphp5
30057 nobody 17 1 115m 10m 5340 R 15.4 0.1 1:29.99 lsphp5
1802 nobody 17 1 113m 9852 4028 R 14.9 0.1 0:01.81 lsphp5
31382 nobody 17 1 115m 10m 5360 R 14.4 0.1 0:46.58 lsphp5
823 nobody 18 1 112m 9292 4424 R 11.9 0.1 0:19.28 lsphp5
1126 nobody 17 1 112m 8908 4044 R 10.5 0.1 0:10.94 lsphp5
29161 nobody 17 1 115m 11m 5712 R 9.5 0.1 1:39.35 lsphp5
31012 nobody 18 1 115m 11m 5680 R 9.5 0.1 0:45.01 lsphp5
1638 nobody 18 1 114m 11m 4460 R 9.0 0.1 0:09.96 lsphp5
29728 nobody 17 1 115m 11m 6000 R 8.0 0.1 1:44.28 lsphp5
1771 nobody 18 1 115m 10m 4512 R 4.5 0.1 0:05.26 lsphp5
31038 nobody 18 1 115m 11m 5568 R 4.5 0.1 0:57.89 lsphp5
1933 nobody 18 1 112m 8048 2896 R 3.0 0.1 0:01.35 lsphp5
2010 nobody 17 1 169m 15m 8424 R 2.0 0.2 0:00.10 lsphp5
1897 nobody 17 1 168m 15m 8760 S 1.5 0.2 0:00.39 lsphp5
14861 nobody 15 0 21428 6656 1692 S 1.5 0.1 0:20.62 lshttpd
14863 nobody 15 0 31780 15m 2688 R 1.5 0.2 5:38.95 lshttpd
1787 nobody 16 1 169m 16m 8808 S 1.0 0.2 0:00.51 lsphp5
1564 nobody 18 1 115m 10m 4892 R 0.5 0.1 0:11.69 lsphp5
1 root 15 0 10348 688 576 S 0.0 0.0 0:01.69 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.09 [migration/0]
3 root 34 19 0 0 0 R 0.0 0.0 0:00.02 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
5 root RT -5 0 0 0 S 0.0 0.0 0:00.06 [migration/1]
6 root 34 19 0 0 0 R 0.0 0.0 0:00.01 [ksoftirqd/1]
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/1]
8 root RT -5 0 0 0 S 0.0 0.0 0:00.12 [migration/2]
9 root 34 19 0 0 0 S 0.0 0.0 0:00.02 [ksoftirqd/2]
top - 08:12:33 up 1 day, 4:11, 1 user, load average: 68.10, 70.86, 72.51
Tasks: 230 total, 68 running, 159 sleeping, 0 stopped, 3 zombie
Cpu0 : 7.5%us, 5.5%sy, 87.1%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 5.5%us, 6.0%sy, 88.5%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 4.0%us, 2.5%sy, 93.5%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 11.4%us, 7.5%sy, 80.1%ni, 0.0%id, 0.0%wa, 0.0%hi, 1.0%si, 0.0%st
Mem: 8103672k total, 7169728k used, 933944k free, 632792k buffers
Swap: 8393952k total, 0k used, 8393952k free, 4961600k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30845 nobody 17 1 115m 11m 5680 R 47.8 0.1 0:59.91 lsphp5
2694 mysql 15 0 904m 324m 4928 S 42.8 4.1 1182:21 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pi
1339 nobody 17 1 114m 9940 4728 R 38.3 0.1 0:11.29 lsphp5
1792 nobody 17 1 115m 10m 4668 R 21.4 0.1 0:05.21 lsphp5
31483 nobody 17 1 115m 10m 5108 R 20.9 0.1 0:48.48 lsphp5
825 nobody 17 1 113m 10m 4608 R 19.9 0.1 0:18.87 lsphp5
1904 nobody 17 1 112m 7844 2988 R 18.4 0.1 0:01.46 lsphp5
998 nobody 17 1 115m 10m 5008 R 16.9 0.1 0:23.12 lsphp5
1408 nobody 17 1 112m 8472 3284 R 16.4 0.1 0:06.03 lsphp5
1788 nobody 17 1 113m 9.9m 4300 R 16.4 0.1 0:03.36 lsphp5
30012 nobody 17 1 115m 11m 5628 R 15.9 0.1 1:23.07 lsphp5
30057 nobody 17 1 115m 10m 5340 R 15.4 0.1 1:29.99 lsphp5
1802 nobody 17 1 113m 9852 4028 R 14.9 0.1 0:01.81 lsphp5
31382 nobody 17 1 115m 10m 5360 R 14.4 0.1 0:46.58 lsphp5
823 nobody 18 1 112m 9292 4424 R 11.9 0.1 0:19.28 lsphp5
1126 nobody 17 1 112m 8908 4044 R 10.5 0.1 0:10.94 lsphp5
29161 nobody 17 1 115m 11m 5712 R 9.5 0.1 1:39.35 lsphp5
31012 nobody 18 1 115m 11m 5680 R 9.5 0.1 0:45.01 lsphp5
1638 nobody 18 1 114m 11m 4460 R 9.0 0.1 0:09.96 lsphp5
29728 nobody 17 1 115m 11m 6000 R 8.0 0.1 1:44.28 lsphp5
1771 nobody 18 1 115m 10m 4512 R 4.5 0.1 0:05.26 lsphp5
31038 nobody 18 1 115m 11m 5568 R 4.5 0.1 0:57.89 lsphp5
1933 nobody 18 1 112m 8048 2896 R 3.0 0.1 0:01.35 lsphp5
2010 nobody 17 1 169m 15m 8424 R 2.0 0.2 0:00.10 lsphp5
1897 nobody 17 1 168m 15m 8760 S 1.5 0.2 0:00.39 lsphp5
14861 nobody 15 0 21428 6656 1692 S 1.5 0.1 0:20.62 lshttpd
14863 nobody 15 0 31780 15m 2688 R 1.5 0.2 5:38.95 lshttpd
1787 nobody 16 1 169m 16m 8808 S 1.0 0.2 0:00.51 lsphp5
1564 nobody 18 1 115m 10m 4892 R 0.5 0.1 0:11.69 lsphp5
1 root 15 0 10348 688 576 S 0.0 0.0 0:01.69 init [3]
2 root RT -5 0 0 0 S 0.0 0.0 0:00.09 [migration/0]
3 root 34 19 0 0 0 R 0.0 0.0 0:00.02 [ksoftirqd/0]
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/0]
5 root RT -5 0 0 0 S 0.0 0.0 0:00.06 [migration/1]
6 root 34 19 0 0 0 R 0.0 0.0 0:00.01 [ksoftirqd/1]
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 [watchdog/1]
8 root RT -5 0 0 0 S 0.0 0.0 0:00.12 [migration/2]
9 root 34 19 0 0 0 S 0.0 0.0 0:00.02 [ksoftirqd/2]
Last edited:
Looks like PHP processes are going into a infinite loop or some thing very CPU intensive.
CPU idle is 0 percent
try strace the PHP processes on top of the "top" output
do
to find out what is the PHP process doing.
Cpu0 : 7.5%us, 5.5%sy, 87.1%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 5.5%us, 6.0%sy, 88.5%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 4.0%us, 2.5%sy, 93.5%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 11.4%us, 7.5%sy, 80.1%ni, 0.0%id, 0.0%wa, 0.0%hi, 1.0%si, 0.0%st
Cpu1 : 5.5%us, 6.0%sy, 88.5%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 4.0%us, 2.5%sy, 93.5%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 11.4%us, 7.5%sy, 80.1%ni, 0.0%id, 0.0%wa, 0.0%hi, 1.0%si, 0.0%st
30845 nobody 17 1 115m 11m 5680 R 47.8 0.1 0:59.91 lsphp5
do
strace -tt -T -p 30845