Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 14 Sep 2016 01:23:51 +0200
From:      "St\xe5le Bordal Kristoffersen" <chiller@putsch.kolbu.ws>
To:        freebsd-questions@freebsd.org
Subject:   Server gets a high load, but no CPU use, and then later stops respond on the network
Message-ID:  <20160913232351.GA36091@putsch.kolbu.ws>

next in thread | raw e-mail | index | archive | help
Hello,
I'm struggeling with finding out what is wrong with my current setup.
It is a supermicro server, running 10.3-RELEASE-p7, and its main job is to
act as a file server (smb, ftp, plex).

about once a day, but not in any pattern, it starts getting a load of 5-10
and usually stops responding over the network before I notice it. On the
IPMI console, it is still alive, but each key I press usualy ends up typing
4-20 identical letters, making it difficult to debug. A quick reboot fixes
it.

>From googling a bit, I have tried to disable msix on the igb network
interface, and increased the nmbclusters with no apparent change in behaviour.
(kern.ipc.nmbclusters="1000000" and hw.igb.enable_msix=0 in loader.conf)


Today I noticed it before it became unresponsive, but I could not figure
out what the cause is. It had a load of 10, and was very laggy over SSH.
I eventually shut down everything else running, and then the load went
down to around 5.

Does anyone have any good input on what could be the cause, or what to do
to debug this problem further?

All I see is that the igb0 taskq pid is almost always in the RUN state when
the machine is having trouble.

Some output from when it was misbehaving:

pusen% uptime
11:24PM  up 2 days,  2:36, 2 users, load averages: 9.07, 9.20, 7.84

pusen# netstat -m
1033/17447/18480 mbufs in use (current/cache/total)
1023/10173/11196/1000000 mbuf clusters in use (current/cache/total/max)
1023/9856 mbuf+clusters out of packet secondary zone in use (current/cache)
0/3344/3344/1530833 4k (page size) jumbo clusters in use
(current/cache/total/max)
0/0/0/453580 9k jumbo clusters in use (current/cache/total/max)
0/0/0/255138 16k jumbo clusters in use (current/cache/total/max)
2304K/38083K/40388K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile

pusen# ps auxH | grep R
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2994:03.50 [idle/idle: cpu0]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2990:04.11 [idle/idle: cpu1]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2990:57.17 [idle/idle: cpu2]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2989:48.37 [idle/idle: cpu3]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2989:53.70 [idle/idle: cpu4]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2989:19.11 [idle/idle: cpu5]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2990:05.88 [idle/idle: cpu6]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2991:36.87 [idle/idle: cpu7]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2991:55.68 [idle/idle: cpu8]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2990:33.11 [idle/idle: cpu9]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2993:17.43 [idle/idle: cpu10]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 2973:36.23 [idle/idle: cpu11]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3035:35.84 [idle/idle: cpu12]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3039:04.19 [idle/idle: cpu13]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3034:43.90 [idle/idle: cpu15]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3034:47.94 [idle/idle: cpu16]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3034:31.73 [idle/idle: cpu19]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3032:37.23 [idle/idle: cpu20]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3033:10.67 [idle/idle: cpu21]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3034:16.99 [idle/idle: cpu22]
root       11 100.0  0.0     0   384  -  RL   Sun08PM 3033:55.10 [idle/idle: cpu23]
root       11  99.6  0.0     0   384  -  RL   Sun08PM 3033:49.85 [idle/idle: cpu14]
root       11  99.3  0.0     0   384  -  RL   Sun08PM 3034:06.59 [idle/idle: cpu17]
root       11  99.3  0.0     0   384  -  RL   Sun08PM 3033:11.53 [idle/idle: cpu18]
root        0   0.0  0.0     0  6512  -  RLs  Sun08PM   28:38.29 [kernel/igb0 taskq]
root        0   0.0  0.0     0  6512  -  RLs  Sun08PM    0:00.00 [kernel/mca taskq]
root       12   0.0  0.0     0   608  -  RL   Sun08PM    0:37.35 [intr/swi4: clock]
root    54877   0.0  0.0 18768  2960  1  R+   11:48PM    0:00.00 ps auxH
root    54879   0.0  0.0 24392  4060  1  R    11:48PM    0:00.00 -su (zsh)

pusen# netstat -w1
            input        (Total)           output
   packets  errs idrops      bytes    packets  errs      bytes colls
         3     0     0        194          4     0        284     0
         0     0     0          0          0     0        182     0
         0     0     0          0          0     0        182     0
         6     0     0        386          7     0        728     0
         2     0     0        126          2     0        236     0

pusen# top -aSP
last pid: 55242;  load averages:  5.12,  4.41,  5.33                                                                                                                                                                 up 2+03:08:35  23:56:22
49 processes:  2 running, 45 sleeping, 1 stopped, 1 waiting
CPU 0:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 1:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 2:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 3:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 4:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 5:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 6:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 7:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 8:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 9:   0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 10:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 11:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 12:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 13:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 14:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 15:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 16:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 17:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 18:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 19:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 20:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 21:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 22:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
CPU 23:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Mem: 1028K Active, 135M Inact, 45G Wired, 1449M Free
ARC: 42G Total, 10G MFU, 30G MRU, 158K Anon, 193M Header, 1177M Other
Swap: 4096M Total, 4096M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
   11 root         24 155 ki31     0K   384K CPU23  23 1206.3 2396.63% [idle]
    5 root          1 -16    -     0K    16K ipmire 14 100:17   0.00% [ipmi0: kcs]
    0 root        407  -8    -     0K  6512K -      22  56:20   0.00% [kernel]
    7 root          2 -16    -     0K    32K umarcl  3   6:21   0.00% [pagedaemon]
   18 root          1  16    -     0K    16K syncer 14   3:37   0.00% [syncer]
   12 root         38 -76    -     0K   608K WAIT   255   3:04   0.00% [intr]
    2 root          6 -16    -     0K    96K -       0   2:41   0.00% [cam]
   14 root          1 -16    -     0K    16K -      16   1:40   0.00% [rand_harvestq]
    3 root          9  -8    -     0K   176K tx->tx 20   1:13   0.00% [zfskern]
   17 root          1 -16    -     0K    16K vlruwt 13   1:10   0.00% [vnlru]
  762 root          1  20    0 50040K 15212K select 18   0:10   0.00% /usr/local/bin/perl -wT /usr/local/sbin/munin-node
  620 root          1  20    0 14520K  2044K select 20   0:06   0.00% /usr/sbin/syslogd -s
   15 root         40 -68    -     0K   640K -       0   0:05   0.00% [usb]
  686 root          1  20    0 26128K 18044K select 15   0:05   0.00% /usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift
  823 root          1  20    0 24156K  5420K select 13   0:02   0.00% sendmail: accepting connections (sendmail)
    6 root          1 -16    -     0K    16K idle   16   0:02   0.00% [enc_daemon0]
   16 root          1 -16    -     0K    16K psleep 19   0:00   0.00% [bufdaemon]
  830 root          1  20    0 16624K   712K nanslp 16   0:00   0.00% /usr/sbin/cron -s
  826 smmsp         1  20    0 24156K  1056K pause  23   0:00   0.00% sendmail: Queue runner@00:30:00 for /var/spool/clientmqueue (sendmail)
52778 chiller       1  20    0 31060K  5356K pause  23   0:00   0.00% -zsh (zsh)
  800 root          1  20    0 61316K  5164K select 17   0:00   0.00% /usr/sbin/sshd
    1 root          1  20    0  9492K   460K wait   22   0:00   0.00% [init]
54395 chiller       1  23    0 31060K  5388K pause  23   0:00   0.00% -zsh (zsh)
52777 chiller       1  20    0 86584K  7576K select 23   0:00   0.00% sshd: chiller@pts/0 (sshd)
54394 chiller       1  20    0 86584K  7616K select 19   0:00   0.00% sshd: chiller@pts/1 (sshd)
  473 root          1  20    0 13628K  4504K select 22   0:00   0.00% /sbin/devd
54441 root          1  20    0 24392K  4064K pause  15   0:00   0.00% -su (zsh)
52774 root          1  20    0 86584K  7532K select 19   0:00   0.00% sshd: chiller [priv] (sshd)
54050 root          1  20    0 24392K  4064K ttyin  20   0:00   0.00% -su (zsh)
   13 root          3  -8    -     0K    48K -       4   0:00   0.00% [geom]
54389 root          1  20    0 86584K  7568K select 13   0:00   0.00% sshd: chiller [priv] (sshd)

pusen# vmstat -i
interrupt                          total       rate
irq1: atkbd0                           6          0
irq18: ehci0 uhci5                     2          0
irq19: uhci2 uhci4*              3871680         21
irq21: uhci1                        1460          0
cpu0:timer                       9994628         54
irq256: igb0                   337772029       1834
irq258: mps0                    11422070         62
cpu1:timer                       5964401         32
cpu22:timer                      2525037         13
cpu11:timer                     29142129        158
cpu16:timer                      2476696         13
cpu5:timer                       6016046         32
cpu19:timer                      2485297         13
cpu4:timer                       5983685         32
cpu21:timer                      2584555         14
cpu10:timer                      5682385         30
cpu14:timer                      2538560         13
cpu8:timer                       6397210         34
cpu13:timer                      2533735         13
cpu9:timer                       5888253         31
cpu18:timer                      2578986         14
cpu2:timer                       5911061         32
cpu20:timer                      2620293         14
cpu3:timer                       5977356         32
cpu12:timer                      2347271         12
cpu7:timer                       5867877         31
cpu17:timer                      2507985         13
cpu6:timer                       5970321         32
cpu23:timer                      2543222         13
cpu15:timer                      2475942         13
Total                          482080178       2618


pusen# vmstat 1
 procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr da0 da1   in   sy   cs us sy id
 0 0 0    858M  1449M   335   0   0   1   355 4954   0   0 1917 4403 5302  0  0 99
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    1  120   80  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    0  124   81  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    1  120   68  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    1  127   92  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    0  120   91  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    1  121   82  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    0  120   75  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    2  121   96  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 265   0   0    1  126   83  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 217   0   0    1  121   68  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 215   0   0    1  120   88  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 215   0   0    0  121   92  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 215   0   0    0  120   83  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 215   0   0    1  127   90  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 196   0   0    5  120   94  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 196   0   0    1  121   80  0  0 100
 0 0 0    858M  1449M     0   0   0   0     0 196   0   0    0  123   79  0  0 100
 1 0 0    858M  1449M     0   0   0   0     0 196   0   0    2  121   76  0  0 100
 1 0 0    858M  1449M     0   0   0   0     0 196   0   0    4  118  106  0  0 100
 1 0 0    858M  1449M     0   0   0   0     0 196   0   0    0  112   87  0  0 100
^C

pusen# zfs-stats -a

------------------------------------------------------------------------
ZFS Subsystem Report                            Wed Sep 14 00:08:39 2016
------------------------------------------------------------------------

System Information:

        Kernel Version:                         1003000 (osreldate)
        Hardware Platform:                      amd64
        Processor Architecture:                 amd64

        ZFS Storage pool Version:               5000
        ZFS Filesystem Version:                 5

FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016 root
12:08AM  up 2 days,  3:21, 3 users, load averages: 5.53, 4.96, 4.94

------------------------------------------------------------------------

System Memory:

        0.00%   2.29    MiB Active,     0.37%   177.05  MiB Inact
        96.69%  45.17   GiB Wired,      0.00%   0 Cache
        2.94%   1.37    GiB Free,       0.00%   0 Gap

        Real Installed:                         48.00   GiB
        Real Available:                 99.93%  47.96   GiB
        Real Managed:                   97.40%  46.72   GiB

        Logical Total:                          48.00   GiB
        Logical Used:                   96.78%  46.46   GiB
        Logical Free:                   3.22%   1.54    GiB

Kernel Memory:                                  448.42  MiB
        Data:                           94.02%  421.59  MiB
        Text:                           5.98%   26.83   MiB

Kernel Memory Map:                              46.72   GiB
        Size:                           90.78%  42.41   GiB
        Free:                           9.22%   4.31    GiB

------------------------------------------------------------------------

ARC Summary: (HEALTHY)
        Memory Throttle Count:                  0

ARC Misc:
        Deleted:                                2.80m
        Recycle Misses:                         0
        Mutex Misses:                           215
        Evict Skips:                            394

ARC Size:                               92.53%  42.30   GiB
        Target Size: (Adaptive)         93.09%  42.56   GiB
        Min Size (Hard Limit):          12.50%  5.71    GiB
        Max Size (High Water):          8:1     45.72   GiB

ARC Size Breakdown:
        Recently Used Cache Size:       78.25%  33.30   GiB
        Frequently Used Cache Size:     21.75%  9.26    GiB

ARC Hash Breakdown:
        Elements Max:                           798.17k
        Elements Current:               99.24%  792.10k
        Collisions:                             545.65k
        Chain Max:                              4
        Chains:                                 35.26k

------------------------------------------------------------------------

ARC Efficiency:                                 99.40m
        Cache Hit Ratio:                94.91%  94.34m
        Cache Miss Ratio:               5.09%   5.06m
        Actual Hit Ratio:               90.03%  89.49m

        Data Demand Efficiency:         98.70%  21.95m
        Data Prefetch Efficiency:       66.68%  7.20m

        CACHE HITS BY CACHE LIST:
          Anonymously Used:             4.61%   4.35m
          Most Recently Used:           6.92%   6.53m
          Most Frequently Used:         87.94%  82.97m
          Most Recently Used Ghost:     0.12%   112.83k
          Most Frequently Used Ghost:   0.41%   386.44k

        CACHE HITS BY DATA TYPE:
          Demand Data:                  22.96%  21.66m
          Prefetch Data:                5.09%   4.80m
          Demand Metadata:              71.90%  67.83m
          Prefetch Metadata:            0.05%   44.80k

        CACHE MISSES BY DATA TYPE:
          Demand Data:                  5.63%   284.71k
          Prefetch Data:                47.43%  2.40m
          Demand Metadata:              46.29%  2.34m
          Prefetch Metadata:            0.65%   33.12k

------------------------------------------------------------------------

L2ARC is disabled

------------------------------------------------------------------------

File-Level Prefetch: (HEALTHY)

DMU Efficiency:                                 200.99m
        Hit Ratio:                      4.81%   9.66m
        Miss Ratio:                     95.19%  191.34m

        Colinear:                               0
          Hit Ratio:                    100.00% 0
          Miss Ratio:                   100.00% 0

        Stride:                                 0
          Hit Ratio:                    100.00% 0
          Miss Ratio:                   100.00% 0

DMU Misc:
        Reclaim:                                0
          Successes:                    100.00% 0
          Failures:                     100.00% 0

        Streams:                                0
          +Resets:                      100.00% 0
          -Resets:                      100.00% 0
          Bogus:                                0

------------------------------------------------------------------------

VDEV cache is disabled

------------------------------------------------------------------------

ZFS Tunables (sysctl):
        kern.maxusers                           3405
        vm.kmem_size                            50162339840
        vm.kmem_size_scale                      1
        vm.kmem_size_min                        0
        vm.kmem_size_max                        1319413950874
        vfs.zfs.trim.max_interval               1
        vfs.zfs.trim.timeout                    30
        vfs.zfs.trim.txg_delay                  32
        vfs.zfs.trim.enabled                    1
        vfs.zfs.vol.unmap_enabled               1
        vfs.zfs.vol.mode                        1
        vfs.zfs.version.zpl                     5
        vfs.zfs.version.spa                     5000
        vfs.zfs.version.acl                     1
        vfs.zfs.version.ioctl                   5
        vfs.zfs.debug                           0
        vfs.zfs.super_owner                     0
        vfs.zfs.sync_pass_rewrite               2
        vfs.zfs.sync_pass_dont_compress         5
        vfs.zfs.sync_pass_deferred_free         2
        vfs.zfs.zio.exclude_metadata            0
        vfs.zfs.zio.use_uma                     1
        vfs.zfs.cache_flush_disable             0
        vfs.zfs.zil_replay_disable              0
        vfs.zfs.min_auto_ashift                 9
        vfs.zfs.max_auto_ashift                 13
        vfs.zfs.vdev.trim_max_pending           10000
        vfs.zfs.vdev.bio_delete_disable         0
        vfs.zfs.vdev.bio_flush_disable          0
        vfs.zfs.vdev.write_gap_limit            4096
        vfs.zfs.vdev.read_gap_limit             32768
        vfs.zfs.vdev.aggregation_limit          131072
        vfs.zfs.vdev.trim_max_active            64
        vfs.zfs.vdev.trim_min_active            1
        vfs.zfs.vdev.scrub_max_active           2
        vfs.zfs.vdev.scrub_min_active           1
        vfs.zfs.vdev.async_write_max_active     10
        vfs.zfs.vdev.async_write_min_active     1
        vfs.zfs.vdev.async_read_max_active      3
        vfs.zfs.vdev.async_read_min_active      1
        vfs.zfs.vdev.sync_write_max_active      10
        vfs.zfs.vdev.sync_write_min_active      10
        vfs.zfs.vdev.sync_read_max_active       10
        vfs.zfs.vdev.sync_read_min_active       10
        vfs.zfs.vdev.max_active                 1000
        vfs.zfs.vdev.async_write_active_max_dirty_percent60
        vfs.zfs.vdev.async_write_active_min_dirty_percent30
        vfs.zfs.vdev.mirror.non_rotating_seek_inc1
        vfs.zfs.vdev.mirror.non_rotating_inc    0
        vfs.zfs.vdev.mirror.rotating_seek_offset1048576
        vfs.zfs.vdev.mirror.rotating_seek_inc   5
        vfs.zfs.vdev.mirror.rotating_inc        0
        vfs.zfs.vdev.trim_on_init               1
        vfs.zfs.vdev.cache.bshift               16
        vfs.zfs.vdev.cache.size                 0
        vfs.zfs.vdev.cache.max                  16384
        vfs.zfs.vdev.metaslabs_per_vdev         200
        vfs.zfs.txg.timeout                     5
        vfs.zfs.space_map_blksz                 4096
        vfs.zfs.spa_slop_shift                  5
        vfs.zfs.spa_asize_inflation             24
        vfs.zfs.deadman_enabled                 1
        vfs.zfs.deadman_checktime_ms            5000
        vfs.zfs.deadman_synctime_ms             1000000
        vfs.zfs.recover                         0
        vfs.zfs.spa_load_verify_data            1
        vfs.zfs.spa_load_verify_metadata        1
        vfs.zfs.spa_load_verify_maxinflight     10000
        vfs.zfs.check_hostid                    1
        vfs.zfs.mg_fragmentation_threshold      85
        vfs.zfs.mg_noalloc_threshold            0
        vfs.zfs.condense_pct                    200
        vfs.zfs.metaslab.bias_enabled           1
        vfs.zfs.metaslab.lba_weighting_enabled  1
        vfs.zfs.metaslab.fragmentation_factor_enabled1
        vfs.zfs.metaslab.preload_enabled        1
        vfs.zfs.metaslab.preload_limit          3
        vfs.zfs.metaslab.unload_delay           8
        vfs.zfs.metaslab.load_pct               50
        vfs.zfs.metaslab.min_alloc_size         33554432
        vfs.zfs.metaslab.df_free_pct            4
        vfs.zfs.metaslab.df_alloc_threshold     131072
        vfs.zfs.metaslab.debug_unload           0
        vfs.zfs.metaslab.debug_load             0
        vfs.zfs.metaslab.fragmentation_threshold70
        vfs.zfs.metaslab.gang_bang              16777217
        vfs.zfs.free_bpobj_enabled              1
        vfs.zfs.free_max_blocks                 -1
        vfs.zfs.no_scrub_prefetch               0
        vfs.zfs.no_scrub_io                     0
        vfs.zfs.resilver_min_time_ms            3000
        vfs.zfs.free_min_time_ms                1000
        vfs.zfs.scan_min_time_ms                1000
        vfs.zfs.scan_idle                       50
        vfs.zfs.scrub_delay                     4
        vfs.zfs.resilver_delay                  2
        vfs.zfs.top_maxinflight                 32
        vfs.zfs.zfetch.array_rd_sz              1048576
        vfs.zfs.zfetch.max_distance             8388608
        vfs.zfs.zfetch.min_sec_reap             2
        vfs.zfs.zfetch.max_streams              8
        vfs.zfs.prefetch_disable                0
        vfs.zfs.delay_scale                     500000
        vfs.zfs.delay_min_dirty_percent         60
        vfs.zfs.dirty_data_sync                 67108864
        vfs.zfs.dirty_data_max_percent          10
        vfs.zfs.dirty_data_max_max              4294967296
        vfs.zfs.dirty_data_max                  4294967296
        vfs.zfs.max_recordsize                  1048576
        vfs.zfs.mdcomp_disable                  0
        vfs.zfs.nopwrite_enabled                1
        vfs.zfs.dedup.prefetch                  1
        vfs.zfs.l2c_only_size                   0
        vfs.zfs.mfu_ghost_data_lsize            32644399616
        vfs.zfs.mfu_ghost_metadata_lsize        0
        vfs.zfs.mfu_ghost_size                  32644399616
        vfs.zfs.mfu_data_lsize                  10069889536
        vfs.zfs.mfu_metadata_lsize              1010720768
        vfs.zfs.mfu_size                        11215540736
        vfs.zfs.mru_ghost_data_lsize            12834271232
        vfs.zfs.mru_ghost_metadata_lsize        90285568
        vfs.zfs.mru_ghost_size                  12924556800
        vfs.zfs.mru_data_lsize                  32342277632
        vfs.zfs.mru_metadata_lsize              48542208
        vfs.zfs.mru_size                        32770899968
        vfs.zfs.anon_data_lsize                 0
        vfs.zfs.anon_metadata_lsize             0
        vfs.zfs.anon_size                       67584
        vfs.zfs.l2arc_norw                      1
        vfs.zfs.l2arc_feed_again                1
        vfs.zfs.l2arc_noprefetch                1
        vfs.zfs.l2arc_feed_min_ms               200
        vfs.zfs.l2arc_feed_secs                 1
        vfs.zfs.l2arc_headroom                  2
        vfs.zfs.l2arc_write_boost               8388608
        vfs.zfs.l2arc_write_max                 8388608
        vfs.zfs.arc_meta_limit                  12272149504
        vfs.zfs.arc_free_target                 84942
        vfs.zfs.arc_shrink_shift                7
        vfs.zfs.arc_average_blocksize           8192
        vfs.zfs.arc_min                         6136074752
        vfs.zfs.arc_max                         49088598016

------------------------------------------------------------------------

pusen# dmesg
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016
    root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Intel(R) Xeon(R) CPU           X5675  @ 3.07GHz (3066.84-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x9ee3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 68723671040 (65540 MB)
avail memory = 49990733824 (47674 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM APIC1635>
FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs
FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 SMT threads
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID: 16
 cpu7 (AP): APIC ID: 17
 cpu8 (AP): APIC ID: 18
 cpu9 (AP): APIC ID: 19
 cpu10 (AP): APIC ID: 20
 cpu11 (AP): APIC ID: 21
 cpu12 (AP): APIC ID: 32
 cpu13 (AP): APIC ID: 33
 cpu14 (AP): APIC ID: 34
 cpu15 (AP): APIC ID: 35
 cpu16 (AP): APIC ID: 36
 cpu17 (AP): APIC ID: 37
 cpu18 (AP): APIC ID: 48
 cpu19 (AP): APIC ID: 49
 cpu20 (AP): APIC ID: 50
 cpu21 (AP): APIC ID: 51
 cpu22 (AP): APIC ID: 52
 cpu23 (AP): APIC ID: 53
random: <Software, Yarrow> initialized
ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe0Block: 128/64 (20150515/tbfadt-644)
ioapic1: Changing APIC ID to 7
ioapic2: Changing APIC ID to 8
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
kbd1 at kbdmux0
acpi0: <SMCI > on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
cpu8: <ACPI CPU> on acpi0
cpu9: <ACPI CPU> on acpi0
cpu10: <ACPI CPU> on acpi0
cpu11: <ACPI CPU> on acpi0
cpu12: <ACPI CPU> on acpi0
cpu13: <ACPI CPU> on acpi0
cpu14: <ACPI CPU> on acpi0
cpu15: <ACPI CPU> on acpi0
cpu16: <ACPI CPU> on acpi0
cpu17: <ACPI CPU> on acpi0
cpu18: <ACPI CPU> on acpi0
cpu19: <ACPI CPU> on acpi0
cpu20: <ACPI CPU> on acpi0
cpu21: <ACPI CPU> on acpi0
cpu22: <ACPI CPU> on acpi0
cpu23: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
Event timer "HPET3" frequency 14318180 Hz quality 340
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xec00-0xec1f mem 0xfbde0000-0xfbdfffff,0xfbdc0000-0xfbddffff,0xfbd9c000-0xfbd9ffff irq 28 at device 0.0 on pci1
igb0:  Using an MSI interrupt
igb0: Ethernet address: 0c:c4:7a:50:05:c8
igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xe800-0xe81f mem 0xfbd60000-0xfbd7ffff,0xfbd40000-0xfbd5ffff,0xfbd1c000-0xfbd1ffff irq 40 at device 0.1 on pci1
igb1:  Using an MSI interrupt
igb1: Ethernet address: 0c:c4:7a:50:05:c9
pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 5.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 7.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 9.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pci0: <base peripheral, interrupt controller> at device 20.0 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.1 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.2 (no driver attached)
pci0: <base peripheral, interrupt controller> at device 20.3 (no driver attached)
uhci0: <Intel 82801JI (ICH10) USB controller USB-D> port 0xcf80-0xcf9f irq 16 at device 26.0 on pci0
uhci0: LegSup = 0x2f00
usbus0 on uhci0
uhci1: <Intel 82801JI (ICH10) USB controller USB-E> port 0xcf40-0xcf5f irq 21 at device 26.1 on pci0
uhci1: LegSup = 0x2f00
usbus1 on uhci1
uhci2: <Intel 82801JI (ICH10) USB controller USB-F> port 0xcf20-0xcf3f irq 19 at device 26.2 on pci0
uhci2: LegSup = 0x2f00
usbus2 on uhci2
ehci0: <Intel 82801JI (ICH10) USB 2.0 controller USB-B> mem 0xfbeda000-0xfbeda3ff irq 18 at device 26.7 on pci0
usbus3: EHCI version 1.0
usbus3 on ehci0
uhci3: <Intel 82801JI (ICH10) USB controller USB-A> port 0xcf00-0xcf1f irq 23 at device 29.0 on pci0
uhci3: LegSup = 0x2f00
usbus4 on uhci3
uhci4: <Intel 82801JI (ICH10) USB controller USB-B> port 0xcec0-0xcedf irq 19 at device 29.1 on pci0
uhci4: LegSup = 0x2f00
usbus5 on uhci4
uhci5: <Intel 82801JI (ICH10) USB controller USB-C> port 0xcea0-0xcebf irq 18 at device 29.2 on pci0
uhci5: LegSup = 0x2f00
usbus6 on uhci5
ehci1: <Intel 82801JI (ICH10) USB 2.0 controller USB-A> mem 0xfbed8000-0xfbed83ff irq 23 at device 29.7 on pci0
usbus7: EHCI version 1.0
usbus7 on ehci1
pcib6: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci6: <ACPI PCI bus> on pcib6
vgapci0: <VGA-compatible display> mem 0xf9000000-0xf9ffffff,0xfaffc000-0xfaffffff,0xfb000000-0xfb7fffff irq 16 at device 4.0 on pci6
vgapci0: Boot video device
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH10 SATA300 controller> port 0xcff0-0xcff7,0xcf7c-0xcf7f,0xcfe0-0xcfe7,0xcef4-0xcef7,0xcfa0-0xcfaf,0xcf60-0xcf6f irq 19 at device 31.2 on pci0
ata2: <ATA channel> at channel 0 on atapci0
ata3: <ATA channel> at channel 1 on atapci0
atapci1: <Intel ICH10 SATA300 controller> port 0xcee0-0xcee7,0xcef0-0xcef3,0xce70-0xce77,0xce7c-0xce7f,0xce60-0xce6f,0xce40-0xce4f irq 19 at device 31.5 on pci0
ata4: <ATA channel> at channel 0 on atapci1
ata5: <ATA channel> at channel 1 on atapci1
pcib7: <ACPI Host-PCI bridge> on acpi0
pci128: <ACPI PCI bus> on pcib7
pcib8: <PCI-PCI bridge> at device 0.0 on pci128
pci129: <PCI bus> on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 1.0 on pci128
pci130: <ACPI PCI bus> on pcib9
pcib10: <ACPI PCI-PCI bridge> at device 3.0 on pci128
pci131: <ACPI PCI bus> on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 5.0 on pci128
pci132: <ACPI PCI bus> on pcib11
pcib12: <ACPI PCI-PCI bridge> at device 7.0 on pci128
pci133: <ACPI PCI bus> on pcib12
mps0: <Avago Technologies (LSI) SAS2008> port 0xf800-0xf8ff mem 0xf8eb0000-0xf8ebffff,0xf8ec0000-0xf8efffff irq 54 at device 0.0 on pci133
mps0: Firmware: 20.00.04.00, Driver: 20.00.00.00-fbsd
mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
pcib13: <ACPI PCI-PCI bridge> at device 9.0 on pci128
pci134: <ACPI PCI bus> on pcib13
pci128: <base peripheral, interrupt controller> at device 20.0 (no driver attached)
pci128: <base peripheral, interrupt controller> at device 20.1 (no driver attached)
pci128: <base peripheral, interrupt controller> at device 20.2 (no driver attached)
pci128: <base peripheral, interrupt controller> at device 20.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
ipmi0: <IPMI System Interface> port 0xca2-0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
qpi0: <QPI system bus> on motherboard
pcib14: <QPI Host-PCI bridge> pcibus 255 on qpi0
pci255: <PCI bus> on pcib14
pcib15: <QPI Host-PCI bridge> pcibus 254 on qpi0
pci254: <PCI bus> on pcib15
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xcefff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est4: <Enhanced SpeedStep Frequency Control> on cpu4
est5: <Enhanced SpeedStep Frequency Control> on cpu5
est6: <Enhanced SpeedStep Frequency Control> on cpu6
est7: <Enhanced SpeedStep Frequency Control> on cpu7
est8: <Enhanced SpeedStep Frequency Control> on cpu8
est9: <Enhanced SpeedStep Frequency Control> on cpu9
est10: <Enhanced SpeedStep Frequency Control> on cpu10
est11: <Enhanced SpeedStep Frequency Control> on cpu11
est12: <Enhanced SpeedStep Frequency Control> on cpu12
est13: <Enhanced SpeedStep Frequency Control> on cpu13
est14: <Enhanced SpeedStep Frequency Control> on cpu14
est15: <Enhanced SpeedStep Frequency Control> on cpu15
est16: <Enhanced SpeedStep Frequency Control> on cpu16
est17: <Enhanced SpeedStep Frequency Control> on cpu17
est18: <Enhanced SpeedStep Frequency Control> on cpu18
est19: <Enhanced SpeedStep Frequency Control> on cpu19
est20: <Enhanced SpeedStep Frequency Control> on cpu20
est21: <Enhanced SpeedStep Frequency Control> on cpu21
est22: <Enhanced SpeedStep Frequency Control> on cpu22
est23: <Enhanced SpeedStep Frequency Control> on cpu23
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 12Mbps Full Speed USB v1.0
usbus6: 12Mbps Full Speed USB v1.0
usbus7: 480Mbps High Speed USB v2.0
ugen1.1: <Intel> at usbus1
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen0.1: <Intel> at usbus0
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen3.1: <Intel> at usbus3
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen2.1: <Intel> at usbus2
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen5.1: <Intel> at usbus5
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
ugen4.1: <Intel> at usbus4
uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen7.1: <Intel> at usbus7
uhub6: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7
ugen6.1: <Intel> at usbus6
uhub7: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
random: unblocking device.
ipmi0: IPMI device rev. 1, firmware rev. 3.03, version 2.0
ipmi0: Number of channels 2
ipmi0: Attached watchdog
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub4: 2 ports with 2 removable, self powered
uhub5: 2 ports with 2 removable, self powered
uhub7: 2 ports with 2 removable, self powered
mps0: SAS Address for SATA device = 243e808aa27ba99c
mps0: SAS Address for SATA device = 382a272f69955241
mps0: SAS Address for SATA device = 243e807fae76849d
mps0: SAS Address for SATA device = 382a272f83843a51
mps0: SAS Address for SATA device = 243c809cac9d7f9c
mps0: SAS Address for SATA device = 242d7e7aa9888190
mps0: SAS Address from SATA device = 243e808aa27ba99c
mps0: SAS Address from SATA device = 382a272f69955241
mps0: SAS Address from SATA device = 243e807fae76849d
mps0: SAS Address from SATA device = 382a272f83843a51
mps0: SAS Address from SATA device = 243c809cac9d7f9c
mps0: SAS Address from SATA device = 242d7e7aa9888190
mps0: SAS Address for SATA device = 243e808aa279869c
mps0: SAS Address for SATA device = 243c809eac76819c
mps0: SAS Address for SATA device = 382a272f7e895842
mps0: SAS Address for SATA device = 243c809e978d879c
mps0: SAS Address for SATA device = 382a272f83843540
mps0: SAS Address for SATA device = 243e8088ae919d9d
mps0: SAS Address from SATA device = 243e808aa279869c
mps0: SAS Address from SATA device = 243c809eac76819c
mps0: SAS Address from SATA device = 382a272f7e895842
mps0: SAS Address from SATA device = 243c809e978d879c
mps0: SAS Address from SATA device = 382a272f83843540
mps0: SAS Address from SATA device = 243e8088ae919d9d
ses0 at mps0 bus 0 scbus4 target 21 lun 0
ses0: <LSI CORP SAS2X36 0717> Fixed Enclosure Services SPC-3 SCSI device
ses0: Serial Number
ses0: 600.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 ENC Device
da1 at mps0 bus 0 scbus4 target 9 lun 0
da4 at mps0 bus 0 scbus4 target 12 lun 0
da7 at mps0 bus 0 scbus4 target 15 lun 0
da1: <ATA ST4000DM000-1F21 CC52> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number             W300JG5S
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 3815447MB (7814037168 512 byte sectors)
da1: quirks=0x8<4K>
da10 at mps0 bus 0 scbus4 target 18 lun 0
da10: <ATA ST4000DM000-1F21 CC52> Fixed Direct Access SPC-4 SCSI device
da10: Serial Number             W300ELSD
da10: 600.000MB/s transfers
da4: <ATA ST4000DM000-1F21 CC52> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number             W3000XMC
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 3815447MB (7814037168 512 byte sectors)
uhub6: 6 ports with 6 removable, self powered
uhub2: 6 ports with 6 removable, self powered
da7: <ATA ST4000DM000-1F21 CC52> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number             W300JG0B
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 3815447MB (7814037168 512 byte sectors)
da7: quirks=0x8<4K>
da2 at mps0 bus 0 scbus4 target 10 lun 0
da2: <ATA HGST HDN724040AL A5E0> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number       PK1334PEJDM8ZS
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 3815447MB (7814037168 512 byte sectors)
da6 at mps0 bus 0 scbus4 target 14 lun 0
da6: <ATA HGST HDN724040AL A5E0> Fixed Direct Access SPC-4 SCSI device
da6: Serial Number       PK1334PCJXW32S
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 3815447MB (7814037168 512 byte sectors)
da10: Command Queueing enabled
da10: 3815447MB (7814037168 512 byte sectors)
da10: quirks=0x8<4K>
da0 at mps0 bus 0 scbus4 target 8 lun 0
da0: <ATA HGST HDN724040AL A5E0> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number       PK1334PEJDM67S
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 3815447MB (7814037168 512 byte sectors)
da5 at mps0 bus 0 scbus4 target 13 lun 0
da5: <ATA HGST HDN724040AL A5E0> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number       PK1334PCJVWZ0S
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 3815447MB (7814037168 512 byte sectors)
da4: quirks=0x8<4K>
da9 at mps0 bus 0 scbus4 target 17 lun 0
da9: <ATA HGST HDN724040AL A5E0> Fixed Direct Access SPC-4 SCSI device
da3 at mps0 bus 0 scbus4 target 11 lun 0
da9: Serial Number       PK2338P4H4SE2C
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 3815447MB (7814037168 512 byte sectors)
da3: <ATA HGST HDN724040AL A5E0> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number       PK1334PCJXBJ8S
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 3815447MB (7814037168 512 byte sectors)
da8 at mps0 bus 0 scbus4 target 16 lun 0
da8: <ATA HGST HDN724040AL A5E0> Fixed Direct Access SPC-4 SCSI device
da8: Serial Number       PK2334PEJ9X35T
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 3815447MB (7814037168 512 byte sectors)
ada0 at ata2 bus 0 scbus0 target 0 lun 0
da11 at mps0 bus 0 scbus4 target 19 lun 0
da11: <ATA HGST HDN724040AL A5E0> Fixed Direct Access SPC-4 SCSI device
da11: Serial Number       PK2334PEJBXNNT
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 3815447MB (7814037168 512 byte sectors)
ada0: <MTFDDAK128MAM-1J1 0809> ACS-2 ATA SATA 3.x device
ada0: Serial Number 1244091BCC3C
ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada0: 122104MB (250069680 512 byte sectors)
ada0: Previously was known as ad4
ada1 at ata3 bus 0 scbus1 target 0 lun 0
ada1: <MTFDDAK128MAM-1J1 0809> ACS-2 ATA SATA 3.x device
ada1: Serial Number 1244091BCC3F
ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada1: 122104MB (250069680 512 byte sectors)
ada1: Previously was known as ad6
SMP: AP CPU #1 Launched!
SMP: AP CPU #22 Launched!
SMP: AP CPU #11 Launched!
SMP: AP CPU #23 Launched!
SMP: AP CPU #10 Launched!
SMP: AP CPU #16 Launched!
SMP: AP CPU #9 Launched!
SMP: AP CPU #12 Launched!
SMP: AP CPU #8 Launched!
SMP: AP CPU #19 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #18 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #20 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #17 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #14 Launched!
SMP: AP CPU #15 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #21 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #13 Launched!
Timecounter "TSC-low" frequency 1533420188 Hz quality 1000
ugen1.2: <American Megatrends Inc.> at usbus1
ukbd0: <Keyboard Interface> on usbus1
kbd2 at ukbd0
Trying to mount root from zfs:zroot/ROOT/default []...
igb1: link state changed to UP
ums0: <Mouse Interface> on usbus1
igb0: link state changed to UP



-- 
Ståle Kristoffersen



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160913232351.GA36091>