Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 26 Sep 2016 01:59:13 +0200
From:      =?utf-8?B?U3TDpWxl?= Kristoffersen <chiller@driftfun.com>
To:        Anton Yuzhaninov <citrin+bsd@citrin.ru>
Cc:        "freebsd-questions@freebsd.org" <freebsd-questions@freebsd.org>
Subject:   Re: Server gets a high load, but no CPU use, and then later stops respond on the network
Message-ID:  <20160925235913.GA9921@putsch.kolbu.ws>
In-Reply-To: <68f553b9-8546-7707-df86-88851b3283f8@citrin.ru>
References:  <20160913232351.GA36091@putsch.kolbu.ws> <68f553b9-8546-7707-df86-88851b3283f8@citrin.ru>

next in thread | previous in thread | raw e-mail | index | archive | help
On 2016-09-20 at 16:57, Anton Yuzhaninov wrote:
<snip>
> In top output you should look at WCPU and STATE for kernel threads and 
> for unresponding network daemons.

It had a problem last night, and beneath is the last output in the
logfile. (Also available here: http://pastebin.com/G1t3ayPd )

Unfortunately it did not save anything after that, even if it did echo
back text written in an already opened ssh connection. No errors were
printed on the ipmi kvm console.

2016-09-25 02:27:00
last pid: 86980;  load averages:  0.09,  0.07,  0.08  up 4+02:13:16    02:27:00
627 processes: 24 running, 565 sleeping, 38 waiting

Mem: 4824K Active, 475M Inact, 30G Wired, 58M Cache, 801M Free
ARC: 27G Total, 20G MFU, 5978M MRU, 232K Anon, 152M Header, 1026M Other
Swap: 4096M Total, 4096M Free


  PID USERNAME   PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
   11 root       155 ki31     0K   384K CPU13  13  97.0H 100.00% [idle{idle: cpu13}]
   11 root       155 ki31     0K   384K CPU0    0  96.9H 100.00% [idle{idle: cpu0}]
   11 root       155 ki31     0K   384K CPU7    7  96.7H 100.00% [idle{idle: cpu7}]
   11 root       155 ki31     0K   384K CPU5    5  96.7H 100.00% [idle{idle: cpu5}]
   11 root       155 ki31     0K   384K RUN     4  96.7H 100.00% [idle{idle: cpu4}]
   11 root       155 ki31     0K   384K CPU3    3  96.7H 100.00% [idle{idle: cpu3}]
   11 root       155 ki31     0K   384K CPU1    1  96.7H 100.00% [idle{idle: cpu1}]
   11 root       155 ki31     0K   384K CPU9    9  96.7H 100.00% [idle{idle: cpu9}]
   11 root       155 ki31     0K   384K CPU6    6  96.7H 100.00% [idle{idle: cpu6}]
   11 root       155 ki31     0K   384K CPU10  10  96.7H 100.00% [idle{idle: cpu10}]
   11 root       155 ki31     0K   384K CPU2    2  96.6H 100.00% [idle{idle: cpu2}]
   11 root       155 ki31     0K   384K CPU17  17  96.6H 100.00% [idle{idle: cpu17}]
   11 root       155 ki31     0K   384K CPU19  19  96.6H 100.00% [idle{idle: cpu19}]
   11 root       155 ki31     0K   384K CPU16  16  96.6H 100.00% [idle{idle: cpu16}]
   11 root       155 ki31     0K   384K CPU15  15  96.6H 100.00% [idle{idle: cpu15}]
   11 root       155 ki31     0K   384K CPU18  18  96.6H 100.00% [idle{idle: cpu18}]
   11 root       155 ki31     0K   384K CPU23  23  96.6H 100.00% [idle{idle: cpu23}]
   11 root       155 ki31     0K   384K CPU12  12  96.6H 100.00% [idle{idle: cpu12}]
   11 root       155 ki31     0K   384K CPU22  22  96.6H 100.00% [idle{idle: cpu22}]
   11 root       155 ki31     0K   384K CPU11  11  96.2H 100.00% [idle{idle: cpu11}]
   11 root       155 ki31     0K   384K CPU8    8  96.7H  99.27% [idle{idle: cpu8}]
   11 root       155 ki31     0K   384K CPU21  21  96.6H  99.27% [idle{idle: cpu21}]
   11 root       155 ki31     0K   384K CPU14  14  96.6H  99.27% [idle{idle: cpu14}]
   11 root       155 ki31     0K   384K CPU20  20  96.6H  91.99% [idle{idle: cpu20}]
  931 root        28    0   306M 14036K select 15  20:36  12.60% /usr/local/sbin/smbd --daemon --configfile=/usr/local/etc/smb4.conf{smbd}
    0 root       -92    -     0K  6512K -      11  62:27   0.10% [kernel{igb0 taskq}]
    5 root       -16    -     0K    16K ipmire 11 193:27   0.00% [ipmi0: kcs]
    7 root       -16    -     0K    32K psleep  0  39:15   0.00% [pagedaemon{pagedaemon}]
 1807 root        20    0   298M 12796K select  0  17:12   0.00% /usr/local/sbin/smbd --daemon --configfile=/usr/local/etc/smb4.conf
  771 plex        20    0   484M   288M uwait   7  11:15   0.00% /usr/local/share/plexmediaserver-plexpass/Plex_Media_Server (Plex Media Server){Plex Media Serve}
  771 plex        20    0   484M   288M kqread 19  11:08   0.00% /usr/local/share/plexmediaserver-plexpass/Plex_Media_Server (Plex Media Server){Plex Media Serve}
   17 root        16    -     0K    16K syncer 21   5:25   0.00% [syncer]
   12 root       -88    -     0K   608K WAIT    8   4:54   0.00% [intr{irq19: uhci2 uhc}]
    2 root       -16    -     0K    96K -      12   3:17   0.00% [cam{doneq0}]
   14 root       -16    -     0K    16K -      16   3:04   0.00% [rand_harvestq]
   18 root       -16    -     0K    16K vlruwt 17   2:19   0.00% [vnlru]
   12 root       -88    -     0K   608K WAIT   13   2:18   0.00% [intr{irq258: mps0}]
    2 root       -16    -     0K    96K -      10   2:18   0.00% [cam{doneq3}]
    2 root       -16    -     0K    96K -      15   2:17   0.00% [cam{doneq1}]
  771 plex        20    0   484M   288M uwait  13   2:15   0.00% /usr/local/share/plexmediaserver-plexpass/Plex_Media_Server (Plex Media Server){Plex Media Serve}
    0 root       -16    -     0K  6512K swapin  8   1:48   0.00% [kernel{swapper}]
    3 root        -8    -     0K   176K tx->tx 22   1:32   0.00% [zfskern{txg_thread_enter}]
  843 plex        35   15   339M 93664K select 12   1:31   0.00% [Plex Script Host{Plex Script Host}]
   12 root       -60    -     0K   608K WAIT    0   1:15   0.00% [intr{swi4: clock}]
    2 root       -16    -     0K    96K -      11   1:05   0.00% [cam{doneq2}]
    0 root       -16    -     0K  6512K -      13   1:04   0.00% [kernel{zio_write_intr_3}]
    0 root       -16    -     0K  6512K -      23   1:04   0.00% [kernel{zio_write_intr_2}]
    0 root       -16    -     0K  6512K -      17   1:04   0.00% [kernel{zio_write_intr_4}]
    0 root       -16    -     0K  6512K -      20   1:04   0.00% [kernel{zio_write_intr_0}]
    0 root       -16    -     0K  6512K -       3   1:04   0.00% [kernel{zio_write_intr_7}]
    0 root       -16    -     0K  6512K -       5   1:04   0.00% [kernel{zio_write_intr_5}]
    0 root       -16    -     0K  6512K -      19   1:04   0.00% [kernel{zio_write_intr_1}]
    0 root       -16    -     0K  6512K -      21   1:04   0.00% [kernel{zio_write_intr_6}]
    0 root       -16    -     0K  6512K -       3   1:02   0.00% [kernel{zio_read_intr_5}]
    0 root       -16    -     0K  6512K -      22   1:02   0.00% [kernel{zio_read_intr_11}]
    0 root       -16    -     0K  6512K -      11   1:02   0.00% [kernel{zio_read_intr_12}]
    0 root       -16    -     0K  6512K -      15   1:02   0.00% [kernel{zio_read_intr_6}]
    0 root       -16    -     0K  6512K -       6   1:02   0.00% [kernel{zio_read_intr_10}]
    0 root       -16    -     0K  6512K -       6   1:02   0.00% [kernel{zio_read_intr_17}]
    0 root       -16    -     0K  6512K -      23   1:02   0.00% [kernel{zio_read_intr_0}]
    0 root       -16    -     0K  6512K -      15   1:02   0.00% [kernel{zio_read_intr_2}]
    0 root       -16    -     0K  6512K -       6   1:02   0.00% [kernel{zio_read_intr_9}]
    0 root       -16    -     0K  6512K -      20   1:02   0.00% [kernel{zio_read_intr_13}]
    0 root       -16    -     0K  6512K -       4   1:02   0.00% [kernel{zio_read_intr_15}]
    0 root       -16    -     0K  6512K -      19   1:02   0.00% [kernel{zio_read_intr_1}]
    0 root       -16    -     0K  6512K -       7   1:02   0.00% [kernel{zio_read_intr_7}]
    0 root       -16    -     0K  6512K -       4   1:02   0.00% [kernel{zio_read_intr_8}]
    0 root       -16    -     0K  6512K -       8   1:02   0.00% [kernel{zio_read_intr_16}]
    0 root       -16    -     0K  6512K -      11   1:02   0.00% [kernel{zio_read_intr_4}]
    0 root       -16    -     0K  6512K -       3   1:02   0.00% [kernel{zio_read_intr_14}]
    0 root       -16    -     0K  6512K -       0   1:01   0.00% [kernel{zio_read_intr_3}]
  746 root        20    0 14500K  2032K select  0   0:40   0.00% /usr/sbin/powerd -a hiadaptive
    0 root       -17    -     0K  6512K -       0   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -16    -     0K  6512K -      13   0:40   0.00% [kernel{zio_write_intr_3}]
    0 root       -17    -     0K  6512K -      11   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -16    -     0K  6512K -      19   0:40   0.00% [kernel{zio_write_intr_7}]
    0 root       -17    -     0K  6512K -       7   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -       2   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -       9   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -       4   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -16    -     0K  6512K -      15   0:40   0.00% [kernel{zio_write_intr_1}]
    0 root       -17    -     0K  6512K -       2   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -      18   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -       1   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -      13   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -      14   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -       0   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -      13   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -      21   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -16    -     0K  6512K -       9   0:40   0.00% [kernel{zio_write_intr_5}]
    0 root       -17    -     0K  6512K -      17   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -       9   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -16    -     0K  6512K -       2   0:40   0.00% [kernel{zio_write_intr_4}]
    0 root       -17    -     0K  6512K -       7   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -16    -     0K  6512K -      11   0:40   0.00% [kernel{zio_write_intr_2}]
    0 root       -17    -     0K  6512K -      14   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -16    -     0K  6512K -      15   0:40   0.00% [kernel{zio_write_intr_6}]
    0 root       -16    -     0K  6512K -       0   0:40   0.00% [kernel{zio_write_intr_0}]
    0 root       -17    -     0K  6512K -      17   0:40   0.00% [kernel{zio_write_issue_}]
    0 root       -17    -     0K  6512K -      10   0:40   0.00% [kernel{zio_write_issue_}]

-------------------------------------------------------------------------------


-- 
Ståle Kristoffersen



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160925235913.GA9921>