Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 22 Aug 2013 10:21:21 +0200
From:      Yamagi Burmeister <lists@yamagi.org>
To:        freebsd-fs@freebsd.org
Subject:   Re: 9.2-RC1: LORs / Deadlock with SU+J on HAST in "memsync" mode
Message-ID:  <20130822102121.a799b116b0f6b0f7c90ba578@yamagi.org>
In-Reply-To: <20130819115101.ae9c0cf788f881dc4de464c5@yamagi.org>
References:  <20130819115101.ae9c0cf788f881dc4de464c5@yamagi.org>

next in thread | previous in thread | raw e-mail | index | archive | help
--Signature=_Thu__22_Aug_2013_10_21_21_+0200_AqUicpZAxs_SMgFu
Content-Type: multipart/mixed;
 boundary="Multipart=_Thu__22_Aug_2013_10_21_21_+0200_nF//5/q+6KpaeERX"


--Multipart=_Thu__22_Aug_2013_10_21_21_+0200_nF//5/q+6KpaeERX
Content-Type: text/plain; charset=US-ASCII
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hello,
attached is a file with the transcript of a dd session containing:
- ps
- trace
- show pcpu
- show allpcpu
- show locks
- show alllocks
- show lockedvnods
- alltrace
If the attachement is eaten by mailman the file can be found here:
http://deponie.yamagi.org/freebsd/debug/lor_hast/ddb2.txt

This is my first deadlock debug session. So if anything is still
missing please tell me.

Ciao,
Yamagi

On Wed, Aug 21, 2013 at 08:03:35PM +0200, Yamagi Burmeister wrote:
> Could the problem be related to this deadlock / LOR? -
> http://lists.freebsd.org/pipermail/freebsd-fs/2013-August/018052.html
This is not related.

>=20
> My test setup is still in place. Will test with r250907 reverted
> tomorrow morning and report back. Additional informations could be
> provided if necessary. I just need to know what exactly.

Just follow the
http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kernel=
debug-deadlocks.html
and collect all the information listed there after the
apache/sendfile/nfs deadlock is reproduced.

----

On Mon, 19 Aug 2013 11:51:01 +0200
Yamagi Burmeister <lists@yamagi.org> wrote:

> Hello,
> in the last week I've seen several deadlocked processes (stucked in
> state "vofflock" or "wswbuf0" on an UFS2 filesystem with SU+J on HAST
> with "memsync" mode. TheThe deadlock disappear if either SU+J is
> disabled (only SU are active) or HAST is switched to "fullsync" mode.
>=20
> The system is:
> FreeBSD tvtransfer.local 9.2-RC1 FreeBSD 9.2-RC1 #0 r254355M:
> Fri Aug 16 12:35:30 UTC 2013
> support@tvtransfer.local:/usr/obj/usr/src/sys/GENERIC  amd64
>=20
> I've build a kernel with full debugging support and seen this two LOR
> beside the known false positive between bufwait and dirhash:
>=20
> lock order reversal:
>  1st 0xfffffe0197310d68 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2223
>  2nd 0xffffff87be8be0f8 bufwait (bufwait)
> @ /usr/src/sys/ufs/ffs/ffs_vnops.c:261 3rd 0xfffffe019746f098 ufs (ufs)
> @ /usr/src/sys/kern/vfs_subr.c:2223 KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame
> 0xffffff88715dcd50 kdb_backtrace() at kdb_backtrace+0x37/frame
> 0xffffff88715dce10 _witness_debugger() at _witness_debugger+0x2c/frame
> 0xffffff88715dce30 witness_checkorder() at witness_checkorder
> +0x875/frame 0xffffff88715dcef0 __lockmgr_args() at __lockmgr_args
> +0x1141/frame 0xffffff88715dcfd0 ffs_lock() at ffs_lock+0x9c/frame
> 0xffffff88715dd020 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe3/frame
> 0xffffff88715dd050 _vn_lock() at _vn_lock+0x55/frame 0xffffff88715dd0b0
> vget() at vget+0x7b/frame 0xffffff88715dd100
> vfs_hash_get() at vfs_hash_get+0xd5/frame 0xffffff88715dd150
> ffs_vgetf() at ffs_vgetf+0x48/frame 0xffffff88715dd1d0
> softdep_sync_buf() at softdep_sync_buf+0x547/frame 0xffffff88715dd2a0
> ffs_syncvnode() at ffs_syncvnode+0x2c1/frame 0xffffff88715dd310
> ffs_truncate() at ffs_truncate+0x10bb/frame 0xffffff88715dd560
> ufs_direnter() at ufs_direnter+0x550/frame 0xffffff88715dd630
> ufs_makeinode() at ufs_makeinode+0x355/frame 0xffffff88715dd7f0
> VOP_CREATE_APV() at VOP_CREATE_APV+0x102/frame 0xffffff88715dd820
> vn_open_cred() at vn_open_cred+0x45e/frame 0xffffff88715dd990
> kern_openat() at kern_openat+0x20a/frame 0xffffff88715ddb10
> amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88715ddc30
> Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88715ddc30
> --- syscall (5, FreeBSD ELF64, sys_open), rip =3D 0x800b38d3c, rsp =3D
> 0x7fffffffd128, rbp =3D 0x7fffffffda40 ---=20
>=20
> lock order reversal:
>  1st 0xfffffe0035dfbf30 so_snd_sx (so_snd_sx)
> @ /usr/src/sys/kern/uipc_sockbuf.c:145 2nd 0xfffffe02ead2b328 ufs (ufs)
> @ /usr/src/sys/kern/uipc_syscalls.c:2062 KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame
> 0xffffff8871515530 kdb_backtrace() at kdb_backtrace+0x37/frame
> 0xffffff88715155f0 _witness_debugger() at _witness_debugger+0x2c/frame
> 0xffffff8871515610 witness_checkorder() at witness_checkorder
> +0x875/frame 0xffffff88715156d0 __lockmgr_args() at __lockmgr_args
> +0x1446/frame 0xffffff88715157b0 ffs_lock() at ffs_lock+0x9c/frame
> 0xffffff8871515800 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe3/frame
> 0xffffff8871515830 _vn_lock() at _vn_lock+0x55/frame 0xffffff8871515890
> kern_sendfile() at kern_sendfile+0x8e7/frame 0xffffff8871515ab0
> do_sendfile() at do_sendfile+0xdc/frame 0xffffff8871515b10
> amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871515c30
> Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871515c30
> --- syscall (393, FreeBSD ELF64, sys_sendfile), rip =3D 0x8022075bc, rsp
> =3D 0x7fffffffb268, rbp =3D 0x7fffffffd570 ---
>=20
> Both LORs can be found in a more readable format here:
> http://deponie.yamagi.org/freebsd/debug/lor_hast/lor.txt
>=20
> The transcript of a DDB session with all informations requested in
> chapter "10.7. Debugging Deadlocks" of the developers handbook can
> be found here:
> http://deponie.yamagi.org/freebsd/debug/lor_hast/ddb.txt
>=20
> Reproducing this problem is easy:
> 1. Create a HAST setup in "memsync" mode with both primary and
>    secondary connected
> 2. "newfs -U -j /dev/hast/$device" on the primary
> 3. mount "mount /dev/hast/$device /mnt"
> 4. Create some load on /mnt. For example copy /usr/ports in
>    an endless loop. Sooner or later processes accessing /mnt
>    will deadlock. This may be easer to reproduce while HAST
>    is performing the initial sync.
>=20
> More information can be provided if necessary.
>=20
> Thanks,
> Yamagi
>=20
> --=20
> Homepage:  www.yamagi.org
> XMPP:      yamagi@yamagi.org
> GnuPG/GPG: 0xEFBCCBCB


--=20
Homepage:  www.yamagi.org
XMPP:      yamagi@yamagi.org
GnuPG/GPG: 0xEFBCCBCB

--Multipart=_Thu__22_Aug_2013_10_21_21_+0200_nF//5/q+6KpaeERX
Content-Type: text/plain;
 name="ddb2.txt"
Content-Disposition: attachment;
 filename="ddb2.txt"
Content-Transfer-Encoding: quoted-printable

db> ps
  pid  ppid  pgrp   uid   state   wmesg         wchan        cmd
42798  1267 42798     0  R+      CPU 6                       sysctl
42785 42780 42780     2  D       flswai   0xffffffff815e7a80 mv
42780 42778 42780     2  Ss      wait     0xfffffe010e60f000 sh
42778  1219  1219     0  S       piperd   0xfffffe000df65b60 cron
42773  1357  1357     0  D+      getblk   0xffffff87bfe6a5f8 mktemp
42646  1359  1359     0  D+      getblk   0xffffff87bfe4bcf8 mktemp
 9787  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 9784  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 9783  1346  1346    80  S       lockf    0xfffffe02f1879700 httpd
 9779  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 9773  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 9769  1346  1346    80  S       lockf    0xfffffe02f1efd580 httpd
 9768  1346  1346    80  S       lockf    0xfffffe02ad5d7400 httpd
 9766  1346  1346    80  S       lockf    0xfffffe02f18b2680 httpd
 9764  1346  1346    80  S       lockf    0xfffffe02ad421600 httpd
 9755  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 9750  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 9749  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 9748  1346  1346    80  S       kqread   0xfffffe0278c7e700 httpd
 8852  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 8851  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 8849  1346  1346    80  D       wswbuf0  0xffffffff81388280 httpd
 8847  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 8842  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 8840  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 8839  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 8837  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 7978  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 7974  1346  1346    80  S       lockf    0xfffffe02ad84f100 httpd
 7971  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 7969  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 7166  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 7164  1346  1346    80  S       lockf    0xfffffe02ad136d00 httpd
 6396  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 3076  1320  3076     0  D+      getblk   0xffffff87bf2f0ff8 cpdup
 2087  1315  2087     0  D+      getblk   0xffffff87bf2f0ff8 cpdup
 1359  1311  1359     0  S+      piperd   0xfffffe010e668b60 sh
 1357  1296  1357     0  S+      piperd   0xfffffe000de7db60 sh
 1356  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 1352  1324  1352     0  S+      select   0xfffffe000dc99640 top
 1351  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 1350  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 1349  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 1348  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 1347  1346  1346    80  D       vofflock 0xfffffe000de479d2 httpd
 1346     1  1346     0  Ss      select   0xfffffe000df7d540 httpd
 1324  1322  1324     0  S+      pause    0xfffffe0028a940a0 tcsh
 1322  1292  1322     0  Ss+     pause    0xfffffe000df340a0 csh
 1320  1318  1320     0  S+      pause    0xfffffe00289b99f0 tcsh
 1318  1292  1318     0  Ss+     pause    0xfffffe0028799548 csh
 1315  1313  1315     0  S+      pause    0xfffffe00287990a0 tcsh
 1313  1292  1313     0  Ss+     pause    0xfffffe000dee6548 csh
 1311  1309  1311     0  S+      pause    0xfffffe000dee69f0 tcsh
 1309  1292  1309     0  Ss+     pause    0xfffffe000dec19f0 csh
 1296  1293  1296     0  S+      pause    0xfffffe00287a40a0 tcsh
 1293  1292  1293     0  Ss+     pause    0xfffffe002879b9f0 csh
 1292     1  1292     0  Rs      CPU 7                       tmux
 1290  1276  1290     0  S+      select   0xfffffe0028eff840 tmux
 1283  1124  1124   845  DLJ     (threaded)                  hastd
100196                   S       uwait    0xfffffe000daf1580 hastd
100195                   S       uwait    0xfffffe000db66180 hastd
100194                   S       uwait    0xfffffe000db66080 hastd
100193                   D       physrd   0xffffff87be46aa60 hastd
100192                   S       ggwait   0xfffffe000de08000 hastd
100191                   S       sbwait   0xfffffe0028645694 hastd
100190                   S       sigwait  0xfffffe000dd37000 hastd
100130                   S       uwait    0xfffffe000df7b080 hastd
 1282  1124  1124   845  DLJ     (threaded)                  hastd
100203                   S       uwait    0xfffffe000dc85900 hastd
100202                   S       uwait    0xfffffe000dd59480 hastd
100201                   S       uwait    0xfffffe000dc85a80 hastd
100200                   S       uwait    0xfffffe000dc85b80 hastd
100199                   S       uwait    0xfffffe000dc85c80 hastd
100198                   S       sbwait   0xfffffe000df54694 hastd
100197                   S       sigwait  0xfffffe000da60000 hastd
100117                   D       physwr   0xffffff87be479160 hastd
 1276  1275  1276     0  S+      pause    0xfffffe000dec10a0 csh
 1275  1272  1275  1001  S+      wait     0xfffffe000df214a8 su
 1272  1271  1272  1001  Ss+     pause    0xfffffe002879b0a0 csh
 1271  1269  1269  1001  S       select   0xfffffe000dc42140 sshd
 1269  1204  1269     0  Ss      select   0xfffffe000db5eac0 sshd
 1267  1261  1267     0  S+      pause    0xfffffe0028a950a0 csh
 1261     1  1261     0  Ss+     wait     0xfffffe0028a954a8 login
 1260     1  1260     0  Ss+     ttyin    0xfffffe000d9e0ca8 getty
 1259     1  1259     0  Ss+     ttyin    0xfffffe000d9e20a8 getty
 1258     1  1258     0  Ss+     ttyin    0xfffffe000d9e24a8 getty
 1257     1  1257     0  Ss+     ttyin    0xfffffe000d9988a8 getty
 1256     1  1256     0  Ss+     ttyin    0xfffffe000d9984a8 getty
 1255     1  1255     0  Ss+     ttyin    0xfffffe000d997ca8 getty
 1254     1  1254     0  Ss+     ttyin    0xfffffe000d9104a8 getty
 1253     1  1253     0  Ss+     ttyin    0xfffffe000d9990a8 getty
 1219     1  1219     0  Ss      nanslp   0xffffffff813fc468 cron
 1204     1  1204     0  Ss      select   0xfffffe000df7d0c0 sshd
 1198     1  1198    26  Ss      select   0xfffffe000dc42540 exim-4.80.1-2
 1185     1  1185   181  Ss      select   0xfffffe000df7c8c0 nrpe2
 1124     1  1124     0  Ss      select   0xfffffe000dc9c440 hastd
 1093     1  1093     0  Ss      select   0xfffffe000df7c5c0 syslogd
  930     1   930     0  Ss      select   0xfffffe000df7c0c0 devd
  135     1   135     0  Ss      pause    0xfffffe000de660a0 adjkerntz
   18     0     0     0  DL      m:w1     0xfffffe000dd07800 [g_mirror gmro=
otfs]
   17     0     0     0  DL      sdflush  0xffffffff815f3778 [softdepflush]
   16     0     0     0  DL      wswbuf0  0xffffffff81388280 [syncer]
    9     0     0     0  DL      vlruwt   0xfffffe000da52000 [vnlru]
    8     0     0     0  DL      wswbuf0  0xffffffff81388280 [bufdaemon]
    7     0     0     0  DL      pgzero   0xffffffff815fd81c [pagezero]
    6     0     0     0  DL      psleep   0xffffffff815fc9e8 [vmdaemon]
    5     0     0     0  DL      psleep   0xffffffff815fc9ac [pagedaemon]
    4     0     0     0  DL      ccb_scan 0xffffffff813bd5e0 [xpt_thrd]
    3     0     0     0  DL      waiting_ 0xffffffff815edac0 [sctp_iterator]
    2     0     0     0  DL      -        0xfffffe000d99b248 [fdc0]
   15     0     0     0  DL      (threaded)                  [usb]
100059                   D       -        0xffffff8000e47e18 [usbus3]
100058                   D       -        0xffffff8000e47dc0 [usbus3]
100057                   D       -        0xffffff8000e47d68 [usbus3]
100056                   D       -        0xffffff8000e47d10 [usbus3]
100055                   D       -        0xffffff8000e3eef0 [usbus2]
100054                   D       -        0xffffff8000e3ee98 [usbus2]
100053                   D       -        0xffffff8000e3ee40 [usbus2]
100052                   D       -        0xffffff8000e3ede8 [usbus2]
100050                   D       -        0xffffff8000e35ef0 [usbus1]
100049                   D       -        0xffffff8000e35e98 [usbus1]
100048                   D       -        0xffffff8000e35e40 [usbus1]
100047                   D       -        0xffffff8000e35de8 [usbus1]
100045                   D       -        0xffffff8000e2cef0 [usbus0]
100044                   D       -        0xffffff8000e2ce98 [usbus0]
100043                   D       -        0xffffff8000e2ce40 [usbus0]
100042                   D       -        0xffffff8000e2cde8 [usbus0]
   14     0     0     0  DL      -        0xffffffff813fb104 [yarrow]
   13     0     0     0  RL      (threaded)                  [geom]
100023                   D       -        0xffffffff813f4350 [g_down]
100022                   Run     CPU 3                       [g_up]
100021                   D       -        0xffffffff813f4338 [g_event]
   12     0     0     0  RL      (threaded)                  [intr]
100064                   I                                   [irq7: ppc0]
100062                   I                                   [swi0: uart ua=
rt]
100061                   I                                   [irq12: psm0]
100060                   I                                   [irq1: atkbd0]
100051                   I                                   [irq18: uhci2]
100046                   CanRun                              [irq19: uhci1 =
ahci0]
100041                   I                                   [irq17: uhci0 =
ehci0]
100040                   I                                   [irq16: twa0]
100032                   I                                   [swi2: cambio]
100029                   I                                   [swi6: task qu=
eue]
100028                   I                                   [swi6: Giant t=
askq]
100026                   I                                   [swi5: fast ta=
skq]
100020                   I                                   [swi1: netisr =
0]
100019                   I                                   [swi4: clock]
100018                   I                                   [swi4: clock]
100017                   I                                   [swi4: clock]
100016                   I                                   [swi4: clock]
100015                   I                                   [swi4: clock]
100014                   I                                   [swi4: clock]
100013                   I                                   [swi4: clock]
100012                   Run     CPU 0                       [swi4: clock]
100011                   I                                   [swi3: vm]
   11     0     0     0  RL      (threaded)                  [idle]
100010                   CanRun                              [idle: cpu7]
100009                   CanRun                              [idle: cpu6]
100008                   Run     CPU 5                       [idle: cpu5]
100007                   Run     CPU 4                       [idle: cpu4]
100006                   CanRun                              [idle: cpu3]
100005                   Run     CPU 2                       [idle: cpu2]
100004                   Run     CPU 1                       [idle: cpu1]
100003                   CanRun                              [idle: cpu0]
    1     0     1     0  SLs     wait     0xfffffe000a2b0950 [init]
   10     0     0     0  DL      audit_wo 0xffffffff815f26f0 [audit]
    0     0     0     0  DLs     (threaded)                  [kernel]
100065                   D       -        0xfffffe000d9fd380 [mca taskq]
100039                   D       -        0xfffffe000a514600 [em1 txq]
100038                   D       -        0xfffffe000a514680 [em1 que]
100037                   D       -        0xfffffe000a50d480 [em0 txq]
100036                   D       -        0xfffffe000a50d500 [em0 que]
100035                   D       -        0xfffffe000a49f500 [acpi_task_2]
100034                   D       -        0xfffffe000a49f500 [acpi_task_1]
100033                   D       -        0xfffffe000a49f500 [acpi_task_0]
100031                   D       -        0xfffffe000a4a0280 [ffs_trim task=
q]
100030                   D       -        0xfffffe000a461400 [kqueue taskq]
100027                   D       -        0xfffffe000a461100 [thread taskq]
100024                   D       -        0xfffffe000a2c2300 [firmware task=
q]
100000                   D       sched    0xffffffff813f4620 [swapper]

-----------------------------------------------------------------------

# httpd stuck in wswbuf
db> trace 8849
Tracing pid 8849 tid 100328 td 0xfffffe02788e8000
sched_switch() at sched_switch+0x194/frame 0xffffff887167d360
mi_switch() at mi_switch+0x208/frame 0xffffff887167d3b0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887167d3e0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887167d410
_sleep() at _sleep+0x3d4/frame 0xffffff887167d4a0
getpbuf() at getpbuf+0x4e/frame 0xffffff887167d4c0
cluster_wbuild_gb() at cluster_wbuild_gb+0x21e/frame 0xffffff887167d570
cluster_write_gb() at cluster_write_gb+0x651/frame 0xffffff887167d670
ffs_write() at ffs_write+0x65e/frame 0xffffff887167d710
VOP_WRITE_APV() at VOP_WRITE_APV+0x16e/frame 0xffffff887167d820
vn_write() at vn_write+0x311/frame 0xffffff887167d8b0
vn_io_fault() at vn_io_fault+0x9e/frame 0xffffff887167da30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887167da80
kern_writev() at kern_writev+0x6c/frame 0xffffff887167dac0
sys_write() at sys_write+0x64/frame 0xffffff887167db10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887167dc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887167dc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

# httpd stuck in vofflo
db> trace 8851
Tracing pid 8851 tid 100240 td 0xfffffe010e5f3490
sched_switch() at sched_switch+0x194/frame 0xffffff88714c5720
mi_switch() at mi_switch+0x208/frame 0xffffff88714c5770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714c57a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88714c57d0
_sleep() at _sleep+0x3d4/frame 0xffffff88714c5860
foffset_lock() at foffset_lock+0x86/frame 0xffffff88714c5890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88714c58b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88714c5a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff88714c5a80
kern_writev() at kern_writev+0x6c/frame 0xffffff88714c5ac0
sys_write() at sys_write+0x64/frame 0xffffff88714c5b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714c5c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714c5c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

-----------------------------------------------------------------------

db> show pcpu
cpuid        =3D 6
dynamic pcpu =3D 0xffffff807eebe900
curthread    =3D 0xfffffe002896a490: pid 42798 "sysctl"
curpcb       =3D 0xffffff8871484d00
fpcurthread  =3D 0xfffffe002896a490: pid 42798 "sysctl"
idlethread   =3D 0xfffffe000a2b3490: tid 100009 "idle: cpu6"
curpmap      =3D 0xfffffe000df87750
tssp         =3D 0xffffffff81601070
commontssp   =3D 0xffffffff81601070
rsp0         =3D 0xffffff8871484d00
gs32p        =3D 0xffffffff815ff1a8
ldt          =3D 0xffffffff815ff1e8
tss          =3D 0xffffffff815ff1d8
spin locks held:

db> show allpcpu
Current CPU: 6

cpuid        =3D 0
dynamic pcpu =3D 0x66b900
curthread    =3D 0xfffffe000a2c4000: pid 12 "swi4: clock"
curpcb       =3D 0xffffff80002bed00
fpcurthread  =3D none
idlethread   =3D 0xfffffe000a2b2490: tid 100003 "idle: cpu0"
curpmap      =3D 0xffffffff813f5070
tssp         =3D 0xffffffff81600e00
commontssp   =3D 0xffffffff81600e00
rsp0         =3D 0xffffff80002bed00
gs32p        =3D 0xffffffff815fef38
ldt          =3D 0xffffffff815fef78
tss          =3D 0xffffffff815fef68
spin locks held:

cpuid        =3D 1
dynamic pcpu =3D 0xffffff807ee9b900
curthread    =3D 0xfffffe000a2b2000: pid 11 "idle: cpu1"
curpcb       =3D 0xffffff8000296d00
fpcurthread  =3D none
idlethread   =3D 0xfffffe000a2b2000: tid 100004 "idle: cpu1"
curpmap      =3D 0xffffffff813f5070
tssp         =3D 0xffffffff81600e68
commontssp   =3D 0xffffffff81600e68
rsp0         =3D 0xffffff8000296d00
gs32p        =3D 0xffffffff815fefa0
ldt          =3D 0xffffffff815fefe0
tss          =3D 0xffffffff815fefd0
spin locks held:

cpuid        =3D 2
dynamic pcpu =3D 0xffffff807eea2900
curthread    =3D 0xfffffe000a2c0920: pid 11 "idle: cpu2"
curpcb       =3D 0xffffff800029bd00
fpcurthread  =3D none
idlethread   =3D 0xfffffe000a2c0920: tid 100005 "idle: cpu2"
curpmap      =3D 0xffffffff813f5070
tssp         =3D 0xffffffff81600ed0
commontssp   =3D 0xffffffff81600ed0
rsp0         =3D 0xffffff800029bd00
gs32p        =3D 0xffffffff815ff008
ldt          =3D 0xffffffff815ff048
tss          =3D 0xffffffff815ff038
spin locks held:

cpuid        =3D 3
dynamic pcpu =3D 0xffffff807eea9900
curthread    =3D 0xfffffe000a2d3000: pid 13 "g_up"
curpcb       =3D 0xffffff80002f0d00
fpcurthread  =3D none
idlethread   =3D 0xfffffe000a2c0490: tid 100006 "idle: cpu3"
curpmap      =3D 0xffffffff813f5070
tssp         =3D 0xffffffff81600f38
commontssp   =3D 0xffffffff81600f38
rsp0         =3D 0xffffff80002f0d00
gs32p        =3D 0xffffffff815ff070
ldt          =3D 0xffffffff815ff0b0
tss          =3D 0xffffffff815ff0a0
spin locks held:
exclusive spin mutex sched lock 3 (sched lock) r =3D 0 (0xffffffff81414cc0)=
 locked @ /usr/src/sys/kern/kern_intr.c:916

cpuid        =3D 4
dynamic pcpu =3D 0xffffff807eeb0900
curthread    =3D 0xfffffe000a2c0000: pid 11 "idle: cpu4"
curpcb       =3D 0xffffff80002a5d00
fpcurthread  =3D none
idlethread   =3D 0xfffffe000a2c0000: tid 100007 "idle: cpu4"
curpmap      =3D 0xffffffff813f5070
tssp         =3D 0xffffffff81600fa0
commontssp   =3D 0xffffffff81600fa0
rsp0         =3D 0xffffff80002a5d00
gs32p        =3D 0xffffffff815ff0d8
ldt          =3D 0xffffffff815ff118
tss          =3D 0xffffffff815ff108
spin locks held:

cpuid        =3D 5
dynamic pcpu =3D 0xffffff807eeb7900
curthread    =3D 0xfffffe000a2b3920: pid 11 "idle: cpu5"
curpcb       =3D 0xffffff80002aad00
fpcurthread  =3D none
idlethread   =3D 0xfffffe000a2b3920: tid 100008 "idle: cpu5"
curpmap      =3D 0xffffffff813f5070
tssp         =3D 0xffffffff81601008
commontssp   =3D 0xffffffff81601008
rsp0         =3D 0xffffff80002aad00
gs32p        =3D 0xffffffff815ff140
ldt          =3D 0xffffffff815ff180
tss          =3D 0xffffffff815ff170
spin locks held:

cpuid        =3D 6
dynamic pcpu =3D 0xffffff807eebe900
curthread    =3D 0xfffffe002896a490: pid 42798 "sysctl"
curpcb       =3D 0xffffff8871484d00
fpcurthread  =3D 0xfffffe002896a490: pid 42798 "sysctl"
idlethread   =3D 0xfffffe000a2b3490: tid 100009 "idle: cpu6"
curpmap      =3D 0xfffffe000df87750
tssp         =3D 0xffffffff81601070
commontssp   =3D 0xffffffff81601070
rsp0         =3D 0xffffff8871484d00
gs32p        =3D 0xffffffff815ff1a8
ldt          =3D 0xffffffff815ff1e8
tss          =3D 0xffffffff815ff1d8
spin locks held:

cpuid        =3D 7
dynamic pcpu =3D 0xffffff807eec5900
curthread    =3D 0xfffffe000de70490: pid 1292 "tmux"
curpcb       =3D 0xffffff8871209d00
fpcurthread  =3D 0xfffffe000de70490: pid 1292 "tmux"
idlethread   =3D 0xfffffe000a2c4920: tid 100010 "idle: cpu7"
curpmap      =3D 0xfffffe002856ca60
tssp         =3D 0xffffffff816010d8
commontssp   =3D 0xffffffff816010d8
rsp0         =3D 0xffffff8871209d00
gs32p        =3D 0xffffffff815ff210
ldt          =3D 0xffffffff815ff250
tss          =3D 0xffffffff815ff240
spin locks held:

-----------------------------------------------------------------------

db> show locks
exclusive sleep mutex Giant (Giant) r =3D 0 (0xffffffff813f7780) locked @ /=
usr/src/sys/kern/kern_sysctl.c:1492

-----------------------------------------------------------------------

db> show alllocks
Process 42798 (sysctl) thread 0xfffffe002896a490 (100227)
exclusive sleep mutex Giant (Giant) r =3D 0 (0xffffffff813f7780) locked @ /=
usr/src/sys/kern/kern_sysctl.c:1492
Process 42773 (mktemp) thread 0xfffffe000ddbb920 (100131)
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02ad8c0ad8) locked @ /usr/src/=
sys/ufs/ffs/ffs_vfsops.c:1721
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02ad5f8098) locked @ /usr/src/=
sys/kern/vfs_subr.c:2223
Process 42646 (mktemp) thread 0xfffffe010e5df920 (100260)
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02ad91e098) locked @ /usr/src/=
sys/ufs/ffs/ffs_vfsops.c:1721
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02ad5f6ad8) locked @ /usr/src/=
sys/kern/vfs_subr.c:2223
Process 8849 (httpd) thread 0xfffffe02788e8000 (100328)
exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87bfaf67f8) locked @ /=
usr/src/sys/kern/vfs_cluster.c:816
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe000dfabd68) locked @ /usr/src/=
sys/kern/vfs_vnops.c:762
Process 3076 (cpdup) thread 0xfffffe000ddbb000 (100157)
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02480265b8) locked @ /usr/src/=
sys/ufs/ffs/ffs_vfsops.c:1721
shared lockmgr ufs (ufs) r =3D 0 (0xfffffe010ebcb848) locked @ /usr/src/sys=
/kern/vfs_subr.c:2223
Process 2087 (cpdup) thread 0xfffffe002896a000 (100228)
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe022cfec5b8) locked @ /usr/src/=
sys/kern/vfs_vnops.c:319
Process 1283 (hastd) thread 0xfffffe000de97920 (100193)
exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87be46aaf8) locked @ /=
usr/src/sys/vm/vm_pager.c:311
Process 1283 (hastd) thread 0xfffffe000de76490 (100191)
exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xfffffe0028645648) locked @ /u=
sr/src/sys/kern/uipc_sockbuf.c:148
Process 1282 (hastd) thread 0xfffffe0028a15490 (100198)
exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xfffffe000df54648) locked @ /u=
sr/src/sys/kern/uipc_sockbuf.c:148
Process 1282 (hastd) thread 0xfffffe000daee920 (100117)
exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87be4791f8) locked @ /=
usr/src/sys/vm/vm_pager.c:311
Process 16 (syncer) thread 0xfffffe000daf2000 (100073)
exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87bf1a4bf8) locked @ /=
usr/src/sys/kern/vfs_cluster.c:816
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02f180f098) locked @ /usr/src/=
sys/kern/vfs_subr.c:1823
Process 8 (bufdaemon) thread 0xfffffe000d98f490 (100071)
exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87bf1490f8) locked @ /=
usr/src/sys/kern/vfs_cluster.c:816
exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02f13f9098) locked @ /usr/src/=
sys/kern/vfs_bio.c:2753

-----------------------------------------------------------------------

db> show lockedvnods
Locked vnodes


0xfffffe000dfabcd0: 0xfffffe000dfabcd0: tag ufs, type VREG
tag ufs, type VREG
    usecount 1, writecount 1, refcount 102 mountedhere 0
    usecount 1, writecount 1, refcount 102 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
    v_object 0xfffffe00289e5570 ref 0 pages 800
    v_object 0xfffffe00289e5570 ref 0 pages 800
        lock type ufs: EXCL by thread 0xfffffe02788e8000 (pid 8849)
lock type ufs: EXCL by thread 0xfffffe02788e8000 (pid 8849)
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#1 0xffffffff80b520fc at ffs_lock+0x9c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#3 0xffffffff809b62e5 at _vn_lock+0x55
#3 0xffffffff809b62e5 at _vn_lock+0x55
#4 0xffffffff809b74b6 at vn_write+0x136
#4 0xffffffff809b74b6 at vn_write+0x136
#5 0xffffffff809b4f7e at vn_io_fault+0x9e
#5 0xffffffff809b4f7e at vn_io_fault+0x9e
#6 0xffffffff80954bc5 at dofilewrite+0x85
#6 0xffffffff80954bc5 at dofilewrite+0x85
#7 0xffffffff8095651c at kern_writev+0x6c
#7 0xffffffff8095651c at kern_writev+0x6c
#8 0xffffffff80956634 at sys_write+0x64
#8 0xffffffff80956634 at sys_write+0x64
#9 0xffffffff80ceb239 at amd64_syscall+0x2f9
#9 0xffffffff80ceb239 at amd64_syscall+0x2f9
#10 0xffffffff80cd57e7 at Xfast_syscall+0xf7
#10 0xffffffff80cd57e7 at Xfast_syscall+0xf7
	ino 1446711, on dev mirror/gmrootfs	ino 1446711, on dev mirror/gmrootfs



0xfffffe010ebcb7b0: 0xfffffe010ebcb7b0: tag ufs, type VDIR
tag ufs, type VDIR
    usecount 1, writecount 0, refcount 4 mountedhere 0
    usecount 1, writecount 0, refcount 4 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
    v_object 0xfffffe010eb691d0 ref 0 pages 4
    v_object 0xfffffe010eb691d0 ref 0 pages 4
        lock type ufs: SHARED (count 1)
lock type ufs: SHARED (count 1)
#0 0xffffffff808e827c at __lockmgr_args+0xd5c
#0 0xffffffff808e827c at __lockmgr_args+0xd5c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#3 0xffffffff809b62e5 at _vn_lock+0x55
#3 0xffffffff809b62e5 at _vn_lock+0x55
#4 0xffffffff809a87cb at vget+0x7b
#4 0xffffffff809a87cb at vget+0x7b
#5 0xffffffff80990df3 at cache_lookup_times+0x553
#5 0xffffffff80990df3 at cache_lookup_times+0x553
#6 0xffffffff80991298 at vfs_cache_lookup+0xc8
#6 0xffffffff80991298 at vfs_cache_lookup+0xc8
#7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102
#7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102
#8 0xffffffff80998bdf at lookup+0x32f
#8 0xffffffff80998bdf at lookup+0x32f
#9 0xffffffff80999c9f at namei+0x40f
#9 0xffffffff80999c9f at namei+0x40f
#10 0xffffffff809ac5b6 at kern_statat_vnhook+0xb6
#10 0xffffffff809ac5b6 at kern_statat_vnhook+0xb6
#11 0xffffffff809ac7d5 at kern_statat+0x15
#11 0xffffffff809ac7d5 at kern_statat+0x15
#12 0xffffffff809ac89a at sys_lstat+0x2a
#12 0xffffffff809ac89a at sys_lstat+0x2a
#13 0xffffffff80ceb239 at amd64_syscall+0x2f9
#13 0xffffffff80ceb239 at amd64_syscall+0x2f9
#14 0xffffffff80cd57e7 at Xfast_syscall+0xf7
#14 0xffffffff80cd57e7 at Xfast_syscall+0xf7
	ino 1203840, on dev hast/links	ino 1203840, on dev hast/links



0xfffffe02f13f9000: 0xfffffe02f13f9000: tag ufs, type VREG
tag ufs, type VREG
    usecount 0, writecount 0, refcount 5 mountedhere 0
    usecount 0, writecount 0, refcount 5 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
    v_object 0xfffffe02f16cc2b8 ref 0 pages 29
    v_object 0xfffffe02f16cc2b8 ref 0 pages 29
        lock type ufs: EXCL by thread 0xfffffe000d98f490 (pid 8)
lock type ufs: EXCL by thread 0xfffffe000d98f490 (pid 8)
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#1 0xffffffff80b520fc at ffs_lock+0x9c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#3 0xffffffff809b62e5 at _vn_lock+0x55
#3 0xffffffff809b62e5 at _vn_lock+0x55
#4 0xffffffff8098b590 at flushbufqueues+0x4f0
#4 0xffffffff8098b590 at flushbufqueues+0x4f0
#5 0xffffffff8098b816 at buf_do_flush+0x16
#5 0xffffffff8098b816 at buf_do_flush+0x16
#6 0xffffffff8098b9c4 at buf_daemon+0xe4
#6 0xffffffff8098b9c4 at buf_daemon+0xe4
#7 0xffffffff808d2e55 at fork_exit+0x135
#7 0xffffffff808d2e55 at fork_exit+0x135
#8 0xffffffff80cd5a2e at fork_trampoline+0xe
#8 0xffffffff80cd5a2e at fork_trampoline+0xe
	ino 228168177, on dev hast/links	ino 228168177, on dev hast/links



0xfffffe02f180f000: 0xfffffe02f180f000: tag ufs, type VREG
tag ufs, type VREG
    usecount 0, writecount 0, refcount 6 mountedhere 0
    usecount 0, writecount 0, refcount 6 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
    v_object 0xfffffe02f16902b8 ref 0 pages 30
    v_object 0xfffffe02f16902b8 ref 0 pages 30
        lock type ufs: EXCL by thread 0xfffffe000daf2000 (pid 16)
lock type ufs: EXCL by thread 0xfffffe000daf2000 (pid 16)
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#1 0xffffffff80b520fc at ffs_lock+0x9c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#3 0xffffffff809b62e5 at _vn_lock+0x55
#3 0xffffffff809b62e5 at _vn_lock+0x55
#4 0xffffffff809aac70 at sync_vnode+0x130
#4 0xffffffff809aac70 at sync_vnode+0x130
#5 0xffffffff809aaef7 at sched_sync+0x1c7
#5 0xffffffff809aaef7 at sched_sync+0x1c7
#6 0xffffffff808d2e55 at fork_exit+0x135
#6 0xffffffff808d2e55 at fork_exit+0x135
#7 0xffffffff80cd5a2e at fork_trampoline+0xe
#7 0xffffffff80cd5a2e at fork_trampoline+0xe
	ino 209869753, on dev hast/links	ino 209869753, on dev hast/links



0xfffffe02ad5f6a40: 0xfffffe02ad5f6a40: tag ufs, type VDIR
tag ufs, type VDIR
    usecount 1, writecount 0, refcount 3 mountedhere 0
    usecount 1, writecount 0, refcount 3 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
    v_object 0xfffffe02ad5c7cb0 ref 0 pages 0
    v_object 0xfffffe02ad5c7cb0 ref 0 pages 0
        lock type ufs: EXCL by thread 0xfffffe010e5df920 (pid 42646)
lock type ufs: EXCL by thread 0xfffffe010e5df920 (pid 42646)
#0 0xffffffff808e827c at __lockmgr_args+0xd5c
#0 0xffffffff808e827c at __lockmgr_args+0xd5c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#3 0xffffffff809b62e5 at _vn_lock+0x55
#3 0xffffffff809b62e5 at _vn_lock+0x55
#4 0xffffffff809a87cb at vget+0x7b
#4 0xffffffff809a87cb at vget+0x7b
#5 0xffffffff80990df3 at cache_lookup_times+0x553
#5 0xffffffff80990df3 at cache_lookup_times+0x553
#6 0xffffffff80991298 at vfs_cache_lookup+0xc8
#6 0xffffffff80991298 at vfs_cache_lookup+0xc8
#7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102
#7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102
#8 0xffffffff80998bdf at lookup+0x32f
#8 0xffffffff80998bdf at lookup+0x32f
#9 0xffffffff80999c9f at namei+0x40f
#9 0xffffffff80999c9f at namei+0x40f
#10 0xffffffff809b5991 at vn_open_cred+0xc1
#10 0xffffffff809b5991 at vn_open_cred+0xc1
#11 0xffffffff809b209a at kern_openat+0x20a
#11 0xffffffff809b209a at kern_openat+0x20a
#12 0xffffffff80ceb239 at amd64_syscall+0x2f9
#12 0xffffffff80ceb239 at amd64_syscall+0x2f9
#13 0xffffffff80cd57e7 at Xfast_syscall+0xf7
#13 0xffffffff80cd57e7 at Xfast_syscall+0xf7
	ino 69421440, on dev hast/links	ino 69421440, on dev hast/links



0xfffffe02ad5f8000: 0xfffffe02ad5f8000: tag ufs, type VDIR
tag ufs, type VDIR
    usecount 1, writecount 0, refcount 3 mountedhere 0
    usecount 1, writecount 0, refcount 3 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
    v_object 0xfffffe02ad51a2b8 ref 0 pages 0
    v_object 0xfffffe02ad51a2b8 ref 0 pages 0
        lock type ufs: EXCL by thread 0xfffffe000ddbb920 (pid 42773)
lock type ufs: EXCL by thread 0xfffffe000ddbb920 (pid 42773)
#0 0xffffffff808e827c at __lockmgr_args+0xd5c
#0 0xffffffff808e827c at __lockmgr_args+0xd5c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#3 0xffffffff809b62e5 at _vn_lock+0x55
#3 0xffffffff809b62e5 at _vn_lock+0x55
#4 0xffffffff809a87cb at vget+0x7b
#4 0xffffffff809a87cb at vget+0x7b
#5 0xffffffff80990df3 at cache_lookup_times+0x553
#5 0xffffffff80990df3 at cache_lookup_times+0x553
#6 0xffffffff80991298 at vfs_cache_lookup+0xc8
#6 0xffffffff80991298 at vfs_cache_lookup+0xc8
#7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102
#7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102
#8 0xffffffff80998bdf at lookup+0x32f
#8 0xffffffff80998bdf at lookup+0x32f
#9 0xffffffff80999c9f at namei+0x40f
#9 0xffffffff80999c9f at namei+0x40f
#10 0xffffffff809b5991 at vn_open_cred+0xc1
#10 0xffffffff809b5991 at vn_open_cred+0xc1
#11 0xffffffff809b209a at kern_openat+0x20a
#11 0xffffffff809b209a at kern_openat+0x20a
#12 0xffffffff80ceb239 at amd64_syscall+0x2f9
#12 0xffffffff80ceb239 at amd64_syscall+0x2f9
#13 0xffffffff80cd57e7 at Xfast_syscall+0xf7
#13 0xffffffff80cd57e7 at Xfast_syscall+0xf7
	ino 53932032, on dev hast/links	ino 53932032, on dev hast/links



0xfffffe02ad91e000: 0xfffffe02ad91e000: tag ufs, type VNON
tag ufs, type VNON
    usecount 1, writecount 0, refcount 1 mountedhere 0
    usecount 1, writecount 0, refcount 1 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
        lock type ufs: EXCL by thread 0xfffffe010e5df920 (pid 42646)
lock type ufs: EXCL by thread 0xfffffe010e5df920 (pid 42646)
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#1 0xffffffff80b4c593 at ffs_vgetf+0x163
#1 0xffffffff80b4c593 at ffs_vgetf+0x163
#2 0xffffffff80b24341 at ffs_valloc+0x4a1
#2 0xffffffff80b24341 at ffs_valloc+0x4a1
#3 0xffffffff80b5f5af at ufs_makeinode+0xaf
#3 0xffffffff80b5f5af at ufs_makeinode+0xaf
#4 0xffffffff80d95262 at VOP_CREATE_APV+0x102
#4 0xffffffff80d95262 at VOP_CREATE_APV+0x102
#5 0xffffffff809b5d2e at vn_open_cred+0x45e
#5 0xffffffff809b5d2e at vn_open_cred+0x45e
#6 0xffffffff809b209a at kern_openat+0x20a
#6 0xffffffff809b209a at kern_openat+0x20a
#7 0xffffffff80ceb239 at amd64_syscall+0x2f9
#7 0xffffffff80ceb239 at amd64_syscall+0x2f9
#8 0xffffffff80cd57e7 at Xfast_syscall+0xf7
#8 0xffffffff80cd57e7 at Xfast_syscall+0xf7
	ino 69421952, on dev hast/links	ino 69421952, on dev hast/links



0xfffffe02ad8c0a40: 0xfffffe02ad8c0a40: tag ufs, type VNON
tag ufs, type VNON
    usecount 1, writecount 0, refcount 1 mountedhere 0
    usecount 1, writecount 0, refcount 1 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
        lock type ufs: EXCL by thread 0xfffffe000ddbb920 (pid 42773)
lock type ufs: EXCL by thread 0xfffffe000ddbb920 (pid 42773)
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#1 0xffffffff80b4c593 at ffs_vgetf+0x163
#1 0xffffffff80b4c593 at ffs_vgetf+0x163
#2 0xffffffff80b24341 at ffs_valloc+0x4a1
#2 0xffffffff80b24341 at ffs_valloc+0x4a1
#3 0xffffffff80b5f5af at ufs_makeinode+0xaf
#3 0xffffffff80b5f5af at ufs_makeinode+0xaf
#4 0xffffffff80d95262 at VOP_CREATE_APV+0x102
#4 0xffffffff80d95262 at VOP_CREATE_APV+0x102
#5 0xffffffff809b5d2e at vn_open_cred+0x45e
#5 0xffffffff809b5d2e at vn_open_cred+0x45e
#6 0xffffffff809b209a at kern_openat+0x20a
#6 0xffffffff809b209a at kern_openat+0x20a
#7 0xffffffff80ceb239 at amd64_syscall+0x2f9
#7 0xffffffff80ceb239 at amd64_syscall+0x2f9
#8 0xffffffff80cd57e7 at Xfast_syscall+0xf7
#8 0xffffffff80cd57e7 at Xfast_syscall+0xf7
	ino 53932416, on dev hast/links	ino 53932416, on dev hast/links



0xfffffe022cfec520: 0xfffffe022cfec520: tag ufs, type VREG
tag ufs, type VREG
    usecount 0, writecount 0, refcount 4 mountedhere 0
    usecount 0, writecount 0, refcount 4 mountedhere 0
    flags (VI_ACTIVE|VI_DOINGINACT)
    flags (VI_ACTIVE|VI_DOINGINACT)
    v_object 0xfffffe024803c570 ref 0 pages 11
    v_object 0xfffffe024803c570 ref 0 pages 11
        lock type ufs: EXCL by thread 0xfffffe002896a000 (pid 2087)
lock type ufs: EXCL by thread 0xfffffe002896a000 (pid 2087)
#0 0xffffffff808e827c at __lockmgr_args+0xd5c
#0 0xffffffff808e827c at __lockmgr_args+0xd5c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#1 0xffffffff80b520fc at ffs_lock+0x9c
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3
#3 0xffffffff809b62e5 at _vn_lock+0x55
#3 0xffffffff809b62e5 at _vn_lock+0x55
#4 0xffffffff809b7f4a at vn_close+0xca
#4 0xffffffff809b7f4a at vn_close+0xca
#5 0xffffffff809b80cd at vn_closefile+0x5d
#5 0xffffffff809b80cd at vn_closefile+0x5d
#6 0xffffffff808c2583 at _fdrop+0x23
#6 0xffffffff808c2583 at _fdrop+0x23
#7 0xffffffff808c4adc at closef+0x5c
#7 0xffffffff808c4adc at closef+0x5c
#8 0xffffffff808c50e1 at kern_close+0x121
#8 0xffffffff808c50e1 at kern_close+0x121
#9 0xffffffff80ceb239 at amd64_syscall+0x2f9
#9 0xffffffff80ceb239 at amd64_syscall+0x2f9
#10 0xffffffff80cd57e7 at Xfast_syscall+0xf7
#10 0xffffffff80cd57e7 at Xfast_syscall+0xf7
	ino 1204245, on dev hast/links	ino 1204245, on dev hast/links



0xfffffe0248026520: 0xfffffe0248026520: tag ufs, type VNON
tag ufs, type VNON
    usecount 1, writecount 0, refcount 1 mountedhere 0
    usecount 1, writecount 0, refcount 1 mountedhere 0
    flags (VI_ACTIVE)
    flags (VI_ACTIVE)
        lock type ufs: EXCL by thread 0xfffffe000ddbb000 (pid 3076)
lock type ufs: EXCL by thread 0xfffffe000ddbb000 (pid 3076)
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#0 0xffffffff808e7e2e at __lockmgr_args+0x90e
#1 0xffffffff80b4c593 at ffs_vgetf+0x163
#1 0xffffffff80b4c593 at ffs_vgetf+0x163
#2 0xffffffff80b58d8c at ufs_lookup_ino+0xcec
#2 0xffffffff80b58d8c at ufs_lookup_ino+0xcec
#3 0xffffffff80d925b2 at VOP_CACHEDLOOKUP_APV+0x102
#3 0xffffffff80d925b2 at VOP_CACHEDLOOKUP_APV+0x102
#4 0xffffffff809912c8 at vfs_cache_lookup+0xf8
#4 0xffffffff809912c8 at vfs_cache_lookup+0xf8
#5 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102
#5 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102
#6 0xffffffff80998bdf at lookup+0x32f
#6 0xffffffff80998bdf at lookup+0x32f
#7 0xffffffff80999c9f at namei+0x40f
#7 0xffffffff80999c9f at namei+0x40f
#8 0xffffffff809ac5b6 at kern_statat_vnhook+0xb6
#8 0xffffffff809ac5b6 at kern_statat_vnhook+0xb6
#9 0xffffffff809ac7d5 at kern_statat+0x15
#9 0xffffffff809ac7d5 at kern_statat+0x15
#10 0xffffffff809ac89a at sys_lstat+0x2a
#10 0xffffffff809ac89a at sys_lstat+0x2a
#11 0xffffffff80ceb239 at amd64_syscall+0x2f9
#11 0xffffffff80ceb239 at amd64_syscall+0x2f9
#12 0xffffffff80cd57e7 at Xfast_syscall+0xf7
#12 0xffffffff80cd57e7 at Xfast_syscall+0xf7
	ino 1204244, on dev hast/links	ino 1204244, on dev hast/links

-----------------------------------------------------------------------

db> alltrace

Tracing command sysctl pid 42798 tid 100227 td 0xfffffe002896a490
kdb_enter() at kdb_enter+0x3b/frame 0xffffff8871484900
kdb_sysctl_enter() at kdb_sysctl_enter+0x89/frame 0xffffff8871484940
sysctl_root() at sysctl_root+0x147/frame 0xffffff8871484990
userland_sysctl() at userland_sysctl+0x158/frame 0xffffff8871484a40
sys___sysctl() at sys___sysctl+0xaa/frame 0xffffff8871484b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871484c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871484c30
--- syscall (202, FreeBSD ELF64, sys___sysctl), rip =3D 0x80092713c, rsp =
=3D 0x7fffffffd278, rbp =3D 0x7fffffffd760 ---

Tracing command mv pid 42785 tid 100292 td 0xfffffe010e5db000
sched_switch() at sched_switch+0x194/frame 0xffffff88715c97a0
mi_switch() at mi_switch+0x208/frame 0xffffff88715c97f0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88715c9820
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88715c9850
_sleep() at _sleep+0x3d4/frame 0xffffff88715c98e0
bwillwrite() at bwillwrite+0xc9/frame 0xffffff88715c98f0
kern_renameat() at kern_renameat+0x4c/frame 0xffffff88715c9b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88715c9c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88715c9c30
--- syscall (128, FreeBSD ELF64, sys_rename), rip =3D 0x80088069c, rsp =3D =
0x7fffffffd0d8, rbp =3D 0x7fffffffdf0b ---

Tracing command sh pid 42780 tid 100249 td 0xfffffe010e5bc920
sched_switch() at sched_switch+0x194/frame 0xffffff88714f26d0
mi_switch() at mi_switch+0x208/frame 0xffffff88714f2720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714f2750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714f27=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88714f27d0
_sleep() at _sleep+0x368/frame 0xffffff88714f2860
kern_wait6() at kern_wait6+0x5e2/frame 0xffffff88714f28f0
kern_wait() at kern_wait+0x9c/frame 0xffffff88714f2a50
sys_wait4() at sys_wait4+0x35/frame 0xffffff88714f2b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714f2c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714f2c30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x800d2e4ac, rsp =3D 0x7=
fffffffd838, rbp =3D 0x7fffffffda08 ---

Tracing command cron pid 42778 tid 100332 td 0xfffffe0028efa490
sched_switch() at sched_switch+0x194/frame 0xffffff8871691850
mi_switch() at mi_switch+0x208/frame 0xffffff88716918a0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716918d0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88716919=
30
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871691950
_sleep() at _sleep+0x368/frame 0xffffff88716919e0
pipe_read() at pipe_read+0x4a2/frame 0xffffff8871691a30
dofileread() at dofileread+0xa1/frame 0xffffff8871691a80
kern_readv() at kern_readv+0x6c/frame 0xffffff8871691ac0
sys_read() at sys_read+0x64/frame 0xffffff8871691b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871691c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871691c30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800d4bb1c, rsp =3D 0x7f=
ffffffd478, rbp =3D 0x800f9a560 ---

Tracing command mktemp pid 42773 tid 100131 td 0xfffffe000ddbb920
sched_switch() at sched_switch+0x194/frame 0xffffff88712a4270
mi_switch() at mi_switch+0x208/frame 0xffffff88712a42c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712a42f0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712a4320
__lockmgr_args() at __lockmgr_args+0x921/frame 0xffffff88712a4400
getblk() at getblk+0x119/frame 0xffffff88712a4490
breadn_flags() at breadn_flags+0x40/frame 0xffffff88712a44e0
bread() at bread+0x26/frame 0xffffff88712a4510
ffs_vgetf() at ffs_vgetf+0x2e9/frame 0xffffff88712a4590
ffs_valloc() at ffs_valloc+0x4a1/frame 0xffffff88712a4630
ufs_makeinode() at ufs_makeinode+0xaf/frame 0xffffff88712a47f0
VOP_CREATE_APV() at VOP_CREATE_APV+0x102/frame 0xffffff88712a4820
vn_open_cred() at vn_open_cred+0x45e/frame 0xffffff88712a4990
kern_openat() at kern_openat+0x20a/frame 0xffffff88712a4b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712a4c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712a4c30
--- syscall (5, FreeBSD ELF64, sys_open), rip =3D 0x80091cd3c, rsp =3D 0x7f=
ffffffd538, rbp =3D 0x800c0704f ---

Tracing command mktemp pid 42646 tid 100260 td 0xfffffe010e5df920
sched_switch() at sched_switch+0x194/frame 0xffffff8871529270
mi_switch() at mi_switch+0x208/frame 0xffffff88715292c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88715292f0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8871529320
__lockmgr_args() at __lockmgr_args+0x921/frame 0xffffff8871529400
getblk() at getblk+0x119/frame 0xffffff8871529490
breadn_flags() at breadn_flags+0x40/frame 0xffffff88715294e0
bread() at bread+0x26/frame 0xffffff8871529510
ffs_vgetf() at ffs_vgetf+0x2e9/frame 0xffffff8871529590
ffs_valloc() at ffs_valloc+0x4a1/frame 0xffffff8871529630
ufs_makeinode() at ufs_makeinode+0xaf/frame 0xffffff88715297f0
VOP_CREATE_APV() at VOP_CREATE_APV+0x102/frame 0xffffff8871529820
vn_open_cred() at vn_open_cred+0x45e/frame 0xffffff8871529990
kern_openat() at kern_openat+0x20a/frame 0xffffff8871529b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871529c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871529c30
--- syscall (5, FreeBSD ELF64, sys_open), rip =3D 0x80091cd3c, rsp =3D 0x7f=
ffffffd538, rbp =3D 0x800c0704f ---

Tracing command httpd pid 9787 tid 100085 td 0xfffffe000df3b920
sched_switch() at sched_switch+0x194/frame 0xffffff88711be720
mi_switch() at mi_switch+0x208/frame 0xffffff88711be770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711be7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88711be7d0
_sleep() at _sleep+0x3d4/frame 0xffffff88711be860
foffset_lock() at foffset_lock+0x86/frame 0xffffff88711be890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88711be8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88711bea30
dofilewrite() at dofilewrite+0x85/frame 0xffffff88711bea80
kern_writev() at kern_writev+0x6c/frame 0xffffff88711beac0
sys_write() at sys_write+0x64/frame 0xffffff88711beb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711bec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711bec30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 9784 tid 100245 td 0xfffffe010e5f2490
sched_switch() at sched_switch+0x194/frame 0xffffff88714de720
mi_switch() at mi_switch+0x208/frame 0xffffff88714de770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714de7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88714de7d0
_sleep() at _sleep+0x3d4/frame 0xffffff88714de860
foffset_lock() at foffset_lock+0x86/frame 0xffffff88714de890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88714de8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88714dea30
dofilewrite() at dofilewrite+0x85/frame 0xffffff88714dea80
kern_writev() at kern_writev+0x6c/frame 0xffffff88714deac0
sys_write() at sys_write+0x64/frame 0xffffff88714deb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714dec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714dec30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 9783 tid 100099 td 0xfffffe000df39000
sched_switch() at sched_switch+0x194/frame 0xffffff88712046d0
mi_switch() at mi_switch+0x208/frame 0xffffff8871204720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871204750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712047=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712047d0
_sleep() at _sleep+0x368/frame 0xffffff8871204860
lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88712048f0
lf_advlock() at lf_advlock+0x47/frame 0xffffff8871204940
vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff8871204a40
VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff8871204a70
sys_flock() at sys_flock+0x13e/frame 0xffffff8871204b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871204c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871204c30
--- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0=
x7fffffffd9d8, rbp =3D 0x7fffffffda5c ---

Tracing command httpd pid 9779 tid 100315 td 0xfffffe0028575490
sched_switch() at sched_switch+0x194/frame 0xffffff887163c720
mi_switch() at mi_switch+0x208/frame 0xffffff887163c770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887163c7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887163c7d0
_sleep() at _sleep+0x3d4/frame 0xffffff887163c860
foffset_lock() at foffset_lock+0x86/frame 0xffffff887163c890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887163c8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887163ca30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887163ca80
kern_writev() at kern_writev+0x6c/frame 0xffffff887163cac0
sys_write() at sys_write+0x64/frame 0xffffff887163cb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887163cc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887163cc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 9773 tid 100276 td 0xfffffe010e702920
sched_switch() at sched_switch+0x194/frame 0xffffff8871579720
mi_switch() at mi_switch+0x208/frame 0xffffff8871579770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88715797a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88715797d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871579860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871579890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88715798b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871579a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871579a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871579ac0
sys_write() at sys_write+0x64/frame 0xffffff8871579b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871579c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871579c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 9769 tid 100181 td 0xfffffe0028a7d920
sched_switch() at sched_switch+0x194/frame 0xffffff887139e6d0
mi_switch() at mi_switch+0x208/frame 0xffffff887139e720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887139e750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887139e7=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887139e7d0
_sleep() at _sleep+0x368/frame 0xffffff887139e860
lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff887139e8f0
lf_advlock() at lf_advlock+0x47/frame 0xffffff887139e940
vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff887139ea40
VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff887139ea70
sys_flock() at sys_flock+0x13e/frame 0xffffff887139eb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887139ec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887139ec30
--- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0=
x7fffffffd9d8, rbp =3D 0x7fffffffda5c ---

Tracing command httpd pid 9768 tid 100087 td 0xfffffe000ddbc920
sched_switch() at sched_switch+0x194/frame 0xffffff88711c86d0
mi_switch() at mi_switch+0x208/frame 0xffffff88711c8720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711c8750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711c87=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711c87d0
_sleep() at _sleep+0x368/frame 0xffffff88711c8860
lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88711c88f0
lf_advlock() at lf_advlock+0x47/frame 0xffffff88711c8940
vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff88711c8a40
VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff88711c8a70
sys_flock() at sys_flock+0x13e/frame 0xffffff88711c8b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711c8c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711c8c30
--- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0=
x7fffffffd9d8, rbp =3D 0x7fffffffda5c ---

Tracing command httpd pid 9766 tid 100140 td 0xfffffe000df5e920
sched_switch() at sched_switch+0x194/frame 0xffffff88712d16d0
mi_switch() at mi_switch+0x208/frame 0xffffff88712d1720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712d1750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712d17=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712d17d0
_sleep() at _sleep+0x368/frame 0xffffff88712d1860
lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88712d18f0
lf_advlock() at lf_advlock+0x47/frame 0xffffff88712d1940
vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff88712d1a40
VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff88712d1a70
sys_flock() at sys_flock+0x13e/frame 0xffffff88712d1b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712d1c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712d1c30
--- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0=
x7fffffffd9d8, rbp =3D 0x7fffffffda5c ---

Tracing command httpd pid 9764 tid 100233 td 0xfffffe000dfdb490
sched_switch() at sched_switch+0x194/frame 0xffffff88714a26d0
mi_switch() at mi_switch+0x208/frame 0xffffff88714a2720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714a2750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714a27=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88714a27d0
_sleep() at _sleep+0x368/frame 0xffffff88714a2860
lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88714a28f0
lf_advlock() at lf_advlock+0x47/frame 0xffffff88714a2940
vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff88714a2a40
VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff88714a2a70
sys_flock() at sys_flock+0x13e/frame 0xffffff88714a2b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714a2c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714a2c30
--- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0=
x7fffffffd9d8, rbp =3D 0x7fffffffda5c ---

Tracing command httpd pid 9755 tid 100126 td 0xfffffe000df9f920
sched_switch() at sched_switch+0x194/frame 0xffffff887128b720
mi_switch() at mi_switch+0x208/frame 0xffffff887128b770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887128b7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887128b7d0
_sleep() at _sleep+0x3d4/frame 0xffffff887128b860
foffset_lock() at foffset_lock+0x86/frame 0xffffff887128b890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887128b8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887128ba30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887128ba80
kern_writev() at kern_writev+0x6c/frame 0xffffff887128bac0
sys_write() at sys_write+0x64/frame 0xffffff887128bb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887128bc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887128bc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 9750 tid 100120 td 0xfffffe000df5f920
sched_switch() at sched_switch+0x194/frame 0xffffff887126d720
mi_switch() at mi_switch+0x208/frame 0xffffff887126d770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887126d7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887126d7d0
_sleep() at _sleep+0x3d4/frame 0xffffff887126d860
foffset_lock() at foffset_lock+0x86/frame 0xffffff887126d890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887126d8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887126da30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887126da80
kern_writev() at kern_writev+0x6c/frame 0xffffff887126dac0
sys_write() at sys_write+0x64/frame 0xffffff887126db10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887126dc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887126dc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 9749 tid 100142 td 0xfffffe000df9e920
sched_switch() at sched_switch+0x194/frame 0xffffff88712db720
mi_switch() at mi_switch+0x208/frame 0xffffff88712db770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712db7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712db7d0
_sleep() at _sleep+0x3d4/frame 0xffffff88712db860
foffset_lock() at foffset_lock+0x86/frame 0xffffff88712db890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88712db8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88712dba30
dofilewrite() at dofilewrite+0x85/frame 0xffffff88712dba80
kern_writev() at kern_writev+0x6c/frame 0xffffff88712dbac0
sys_write() at sys_write+0x64/frame 0xffffff88712dbb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712dbc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712dbc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 9748 tid 100127 td 0xfffffe000df5f000
sched_switch() at sched_switch+0x194/frame 0xffffff88712906d0
mi_switch() at mi_switch+0x208/frame 0xffffff8871290720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871290750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712907=
b0
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88712907e0
_sleep() at _sleep+0x3ae/frame 0xffffff8871290870
kern_kevent() at kern_kevent+0x331/frame 0xffffff8871290a50
sys_kevent() at sys_kevent+0x90/frame 0xffffff8871290b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871290c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871290c30
--- syscall (363, FreeBSD ELF64, sys_kevent), rip =3D 0x80228ed8c, rsp =3D =
0x7fffffffd958, rbp =3D 0x8029112f0 ---

Tracing command httpd pid 8852 tid 100307 td 0xfffffe0278833000
sched_switch() at sched_switch+0x194/frame 0xffffff8871614720
mi_switch() at mi_switch+0x208/frame 0xffffff8871614770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716147a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88716147d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871614860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871614890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88716148b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871614a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871614a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871614ac0
sys_write() at sys_write+0x64/frame 0xffffff8871614b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871614c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871614c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 8851 tid 100240 td 0xfffffe010e5f3490
sched_switch() at sched_switch+0x194/frame 0xffffff88714c5720
mi_switch() at mi_switch+0x208/frame 0xffffff88714c5770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714c57a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88714c57d0
_sleep() at _sleep+0x3d4/frame 0xffffff88714c5860
foffset_lock() at foffset_lock+0x86/frame 0xffffff88714c5890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88714c58b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88714c5a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff88714c5a80
kern_writev() at kern_writev+0x6c/frame 0xffffff88714c5ac0
sys_write() at sys_write+0x64/frame 0xffffff88714c5b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714c5c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714c5c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 8849 tid 100328 td 0xfffffe02788e8000
sched_switch() at sched_switch+0x194/frame 0xffffff887167d360
mi_switch() at mi_switch+0x208/frame 0xffffff887167d3b0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887167d3e0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887167d410
_sleep() at _sleep+0x3d4/frame 0xffffff887167d4a0
getpbuf() at getpbuf+0x4e/frame 0xffffff887167d4c0
cluster_wbuild_gb() at cluster_wbuild_gb+0x21e/frame 0xffffff887167d570
cluster_write_gb() at cluster_write_gb+0x651/frame 0xffffff887167d670
ffs_write() at ffs_write+0x65e/frame 0xffffff887167d710
VOP_WRITE_APV() at VOP_WRITE_APV+0x16e/frame 0xffffff887167d820
vn_write() at vn_write+0x311/frame 0xffffff887167d8b0
vn_io_fault() at vn_io_fault+0x9e/frame 0xffffff887167da30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887167da80
kern_writev() at kern_writev+0x6c/frame 0xffffff887167dac0
sys_write() at sys_write+0x64/frame 0xffffff887167db10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887167dc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887167dc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 8847 tid 100327 td 0xfffffe02788e8490
sched_switch() at sched_switch+0x194/frame 0xffffff8871678720
mi_switch() at mi_switch+0x208/frame 0xffffff8871678770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716787a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88716787d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871678860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871678890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88716788b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871678a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871678a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871678ac0
sys_write() at sys_write+0x64/frame 0xffffff8871678b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871678c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871678c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 8842 tid 100326 td 0xfffffe02788e8920
sched_switch() at sched_switch+0x194/frame 0xffffff8871673720
mi_switch() at mi_switch+0x208/frame 0xffffff8871673770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716737a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88716737d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871673860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871673890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88716738b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871673a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871673a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871673ac0
sys_write() at sys_write+0x64/frame 0xffffff8871673b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871673c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871673c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 8840 tid 100325 td 0xfffffe02788ee000
sched_switch() at sched_switch+0x194/frame 0xffffff887166e720
mi_switch() at mi_switch+0x208/frame 0xffffff887166e770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887166e7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887166e7d0
_sleep() at _sleep+0x3d4/frame 0xffffff887166e860
foffset_lock() at foffset_lock+0x86/frame 0xffffff887166e890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887166e8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887166ea30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887166ea80
kern_writev() at kern_writev+0x6c/frame 0xffffff887166eac0
sys_write() at sys_write+0x64/frame 0xffffff887166eb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887166ec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887166ec30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 8839 tid 100324 td 0xfffffe0028efd490
sched_switch() at sched_switch+0x194/frame 0xffffff8871669720
mi_switch() at mi_switch+0x208/frame 0xffffff8871669770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716697a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88716697d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871669860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871669890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88716698b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871669a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871669a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871669ac0
sys_write() at sys_write+0x64/frame 0xffffff8871669b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871669c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871669c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 8837 tid 100132 td 0xfffffe000ddbb490
sched_switch() at sched_switch+0x194/frame 0xffffff88712a9720
mi_switch() at mi_switch+0x208/frame 0xffffff88712a9770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712a97a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712a97d0
_sleep() at _sleep+0x3d4/frame 0xffffff88712a9860
foffset_lock() at foffset_lock+0x86/frame 0xffffff88712a9890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88712a98b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88712a9a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff88712a9a80
kern_writev() at kern_writev+0x6c/frame 0xffffff88712a9ac0
sys_write() at sys_write+0x64/frame 0xffffff88712a9b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712a9c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712a9c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 7978 tid 100141 td 0xfffffe000de77000
sched_switch() at sched_switch+0x194/frame 0xffffff88712d6720
mi_switch() at mi_switch+0x208/frame 0xffffff88712d6770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712d67a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712d67d0
_sleep() at _sleep+0x3d4/frame 0xffffff88712d6860
foffset_lock() at foffset_lock+0x86/frame 0xffffff88712d6890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88712d68b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88712d6a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff88712d6a80
kern_writev() at kern_writev+0x6c/frame 0xffffff88712d6ac0
sys_write() at sys_write+0x64/frame 0xffffff88712d6b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712d6c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712d6c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 7974 tid 100262 td 0xfffffe010e5df000
sched_switch() at sched_switch+0x194/frame 0xffffff88715336d0
mi_switch() at mi_switch+0x208/frame 0xffffff8871533720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871533750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88715337=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88715337d0
_sleep() at _sleep+0x368/frame 0xffffff8871533860
lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88715338f0
lf_advlock() at lf_advlock+0x47/frame 0xffffff8871533940
vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff8871533a40
VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff8871533a70
sys_flock() at sys_flock+0x13e/frame 0xffffff8871533b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871533c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871533c30
--- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0=
x7fffffffd9d8, rbp =3D 0x7fffffffda5c ---

Tracing command httpd pid 7971 tid 100208 td 0xfffffe0028eea920
sched_switch() at sched_switch+0x194/frame 0xffffff8871425720
mi_switch() at mi_switch+0x208/frame 0xffffff8871425770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714257a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88714257d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871425860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871425890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88714258b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871425a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871425a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871425ac0
sys_write() at sys_write+0x64/frame 0xffffff8871425b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871425c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871425c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 7969 tid 100098 td 0xfffffe000df39490
sched_switch() at sched_switch+0x194/frame 0xffffff88711ff720
mi_switch() at mi_switch+0x208/frame 0xffffff88711ff770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711ff7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88711ff7d0
_sleep() at _sleep+0x3d4/frame 0xffffff88711ff860
foffset_lock() at foffset_lock+0x86/frame 0xffffff88711ff890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88711ff8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88711ffa30
dofilewrite() at dofilewrite+0x85/frame 0xffffff88711ffa80
kern_writev() at kern_writev+0x6c/frame 0xffffff88711ffac0
sys_write() at sys_write+0x64/frame 0xffffff88711ffb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711ffc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711ffc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 7166 tid 100278 td 0xfffffe010e702000
sched_switch() at sched_switch+0x194/frame 0xffffff8871583720
mi_switch() at mi_switch+0x208/frame 0xffffff8871583770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88715837a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88715837d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871583860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871583890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88715838b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871583a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871583a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871583ac0
sys_write() at sys_write+0x64/frame 0xffffff8871583b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871583c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871583c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command httpd pid 7164 tid 100175 td 0xfffffe0028a7e490
sched_switch() at sched_switch+0x194/frame 0xffffff88713806d0
mi_switch() at mi_switch+0x208/frame 0xffffff8871380720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871380750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713807=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713807d0
_sleep() at _sleep+0x368/frame 0xffffff8871380860
lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88713808f0
lf_advlock() at lf_advlock+0x47/frame 0xffffff8871380940
vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff8871380a40
VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff8871380a70
sys_flock() at sys_flock+0x13e/frame 0xffffff8871380b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871380c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871380c30
--- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0=
x7fffffffd9d8, rbp =3D 0x7fffffffda5c ---

Tracing command httpd pid 6396 tid 100115 td 0xfffffe000daef000
sched_switch() at sched_switch+0x194/frame 0xffffff8871254720
mi_switch() at mi_switch+0x208/frame 0xffffff8871254770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712547a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712547d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871254860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871254890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88712548b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871254a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871254a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871254ac0
sys_write() at sys_write+0x64/frame 0xffffff8871254b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871254c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871254c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command cpdup pid 3076 tid 100157 td 0xfffffe000ddbb000
sched_switch() at sched_switch+0x194/frame 0xffffff8871326280
mi_switch() at mi_switch+0x208/frame 0xffffff88713262d0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871326300
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8871326330
__lockmgr_args() at __lockmgr_args+0x921/frame 0xffffff8871326410
getblk() at getblk+0x119/frame 0xffffff88713264a0
breadn_flags() at breadn_flags+0x40/frame 0xffffff88713264f0
bread() at bread+0x26/frame 0xffffff8871326520
ffs_vgetf() at ffs_vgetf+0x2e9/frame 0xffffff88713265a0
ufs_lookup_ino() at ufs_lookup_ino+0xcec/frame 0xffffff88713266b0
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x102/frame 0xffffff88713266=
e0
vfs_cache_lookup() at vfs_cache_lookup+0xf8/frame 0xffffff8871326750
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x102/frame 0xffffff8871326780
lookup() at lookup+0x32f/frame 0xffffff8871326820
namei() at namei+0x40f/frame 0xffffff88713268d0
kern_statat_vnhook() at kern_statat_vnhook+0xb6/frame 0xffffff8871326a50
kern_statat() at kern_statat+0x15/frame 0xffffff8871326a70
sys_lstat() at sys_lstat+0x2a/frame 0xffffff8871326b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871326c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871326c30
--- syscall (190, FreeBSD ELF64, sys_lstat), rip =3D 0x800b314ac, rsp =3D 0=
x7fffffffd128, rbp =3D 0x7fffffffda40 ---

Tracing command cpdup pid 2087 tid 100228 td 0xfffffe002896a000
sched_switch() at sched_switch+0x194/frame 0xffffff88714894c0
mi_switch() at mi_switch+0x208/frame 0xffffff8871489510
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871489540
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8871489570
__lockmgr_args() at __lockmgr_args+0x921/frame 0xffffff8871489650
getblk() at getblk+0x119/frame 0xffffff88714896e0
breadn_flags() at breadn_flags+0x40/frame 0xffffff8871489730
ffs_update() at ffs_update+0x117/frame 0xffffff88714897c0
ufs_inactive() at ufs_inactive+0x178/frame 0xffffff8871489810
VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0x105/frame 0xffffff8871489840
vinactive() at vinactive+0xa4/frame 0xffffff8871489890
vputx() at vputx+0x3e1/frame 0xffffff88714898f0
vn_close() at vn_close+0x13a/frame 0xffffff8871489970
vn_closefile() at vn_closefile+0x5d/frame 0xffffff88714899f0
_fdrop() at _fdrop+0x23/frame 0xffffff8871489a10
closef() at closef+0x5c/frame 0xffffff8871489ac0
kern_close() at kern_close+0x121/frame 0xffffff8871489b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871489c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871489c30
--- syscall (6, FreeBSD ELF64, sys_close), rip =3D 0x800b40adc, rsp =3D 0x7=
fffffffd128, rbp =3D 0 ---

Tracing command sh pid 1359 tid 100184 td 0xfffffe0028a16920
sched_switch() at sched_switch+0x194/frame 0xffffff88713ad850
mi_switch() at mi_switch+0x208/frame 0xffffff88713ad8a0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713ad8d0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713ad9=
30
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713ad950
_sleep() at _sleep+0x368/frame 0xffffff88713ad9e0
pipe_read() at pipe_read+0x4a2/frame 0xffffff88713ada30
dofileread() at dofileread+0xa1/frame 0xffffff88713ada80
kern_readv() at kern_readv+0x6c/frame 0xffffff88713adac0
sys_read() at sys_read+0x64/frame 0xffffff88713adb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713adc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713adc30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800dbbb1c, rsp =3D 0x7f=
ffffffd3d8, rbp =3D 0x80140a88e ---

Tracing command sh pid 1357 tid 100151 td 0xfffffe000df5e490
sched_switch() at sched_switch+0x194/frame 0xffffff8871308850
mi_switch() at mi_switch+0x208/frame 0xffffff88713088a0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713088d0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713089=
30
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871308950
_sleep() at _sleep+0x368/frame 0xffffff88713089e0
pipe_read() at pipe_read+0x4a2/frame 0xffffff8871308a30
dofileread() at dofileread+0xa1/frame 0xffffff8871308a80
kern_readv() at kern_readv+0x6c/frame 0xffffff8871308ac0
sys_read() at sys_read+0x64/frame 0xffffff8871308b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871308c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871308c30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800dbbb1c, rsp =3D 0x7f=
ffffffd3d8, rbp =3D 0x80140a88e ---

Tracing command httpd pid 1356 tid 100153 td 0xfffffe000df89920
sched_switch() at sched_switch+0x194/frame 0xffffff8871312720
mi_switch() at mi_switch+0x208/frame 0xffffff8871312770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713127a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88713127d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871312860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871312890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88713128b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871312a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871312a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871312ac0
sys_write() at sys_write+0x64/frame 0xffffff8871312b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871312c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871312c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7f8, rbp =3D 0x13 ---

Tracing command top pid 1352 tid 100217 td 0xfffffe0028efd920
sched_switch() at sched_switch+0x194/frame 0xffffff8871452700
mi_switch() at mi_switch+0x208/frame 0xffffff8871452750
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871452780
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714527=
e0
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff8871452810
_cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff8871452870
seltdwait() at seltdwait+0x56/frame 0xffffff88714528a0
kern_select() at kern_select+0x6f0/frame 0xffffff8871452ac0
sys_select() at sys_select+0x5d/frame 0xffffff8871452b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871452c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871452c30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x800fa6a9c, rsp =3D 0=
x7fffffffd8c8, rbp =3D 0x2f ---

Tracing command httpd pid 1351 tid 100160 td 0xfffffe000ddb8000
sched_switch() at sched_switch+0x194/frame 0xffffff8871335720
mi_switch() at mi_switch+0x208/frame 0xffffff8871335770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713357a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88713357d0
_sleep() at _sleep+0x3d4/frame 0xffffff8871335860
foffset_lock() at foffset_lock+0x86/frame 0xffffff8871335890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88713358b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871335a30
dofilewrite() at dofilewrite+0x85/frame 0xffffff8871335a80
kern_writev() at kern_writev+0x6c/frame 0xffffff8871335ac0
sys_write() at sys_write+0x64/frame 0xffffff8871335b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871335c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871335c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7d8, rbp =3D 0x13 ---

Tracing command httpd pid 1350 tid 100162 td 0xfffffe000daf2490
sched_switch() at sched_switch+0x194/frame 0xffffff887133f720
mi_switch() at mi_switch+0x208/frame 0xffffff887133f770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887133f7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887133f7d0
_sleep() at _sleep+0x3d4/frame 0xffffff887133f860
foffset_lock() at foffset_lock+0x86/frame 0xffffff887133f890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887133f8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887133fa30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887133fa80
kern_writev() at kern_writev+0x6c/frame 0xffffff887133fac0
sys_write() at sys_write+0x64/frame 0xffffff887133fb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887133fc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887133fc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7d8, rbp =3D 0x13 ---

Tracing command httpd pid 1349 tid 100114 td 0xfffffe000dec8490
sched_switch() at sched_switch+0x194/frame 0xffffff887124f720
mi_switch() at mi_switch+0x208/frame 0xffffff887124f770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887124f7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887124f7d0
_sleep() at _sleep+0x3d4/frame 0xffffff887124f860
foffset_lock() at foffset_lock+0x86/frame 0xffffff887124f890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887124f8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887124fa30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887124fa80
kern_writev() at kern_writev+0x6c/frame 0xffffff887124fac0
sys_write() at sys_write+0x64/frame 0xffffff887124fb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887124fc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887124fc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7d8, rbp =3D 0x13 ---

Tracing command httpd pid 1348 tid 100165 td 0xfffffe000dfca920
sched_switch() at sched_switch+0x194/frame 0xffffff887134e720
mi_switch() at mi_switch+0x208/frame 0xffffff887134e770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887134e7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887134e7d0
_sleep() at _sleep+0x3d4/frame 0xffffff887134e860
foffset_lock() at foffset_lock+0x86/frame 0xffffff887134e890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887134e8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887134ea30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887134ea80
kern_writev() at kern_writev+0x6c/frame 0xffffff887134eac0
sys_write() at sys_write+0x64/frame 0xffffff887134eb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887134ec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887134ec30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7d8, rbp =3D 0x13 ---

Tracing command httpd pid 1347 tid 100104 td 0xfffffe000de80920
sched_switch() at sched_switch+0x194/frame 0xffffff887121d720
mi_switch() at mi_switch+0x208/frame 0xffffff887121d770
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887121d7a0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887121d7d0
_sleep() at _sleep+0x3d4/frame 0xffffff887121d860
foffset_lock() at foffset_lock+0x86/frame 0xffffff887121d890
foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887121d8b0
vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887121da30
dofilewrite() at dofilewrite+0x85/frame 0xffffff887121da80
kern_writev() at kern_writev+0x6c/frame 0xffffff887121dac0
sys_write() at sys_write+0x64/frame 0xffffff887121db10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887121dc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887121dc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7=
fffffffd7d8, rbp =3D 0x13 ---

Tracing command httpd pid 1346 tid 100095 td 0xfffffe000df3a490
sched_switch() at sched_switch+0x194/frame 0xffffff88711f0700
mi_switch() at mi_switch+0x208/frame 0xffffff88711f0750
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711f0780
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711f07=
e0
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88711f0810
_cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff88711f0870
seltdwait() at seltdwait+0x56/frame 0xffffff88711f08a0
kern_select() at kern_select+0x6f0/frame 0xffffff88711f0ac0
sys_select() at sys_select+0x5d/frame 0xffffff88711f0b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711f0c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711f0c30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x8022ada9c, rsp =3D 0=
x7fffffffda28, rbp =3D 0 ---

Tracing command tcsh pid 1324 tid 100204 td 0xfffffe0028eeb490
sched_switch() at sched_switch+0x194/frame 0xffffff8871411900
mi_switch() at mi_switch+0x208/frame 0xffffff8871411950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871411980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714119=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871411a00
_sleep() at _sleep+0x368/frame 0xffffff8871411a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871411ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871411b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871411c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871411c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce68, rbp =3D 0x801408500 ---

Tracing command csh pid 1322 tid 100089 td 0xfffffe000df3b000
sched_switch() at sched_switch+0x194/frame 0xffffff88711d2900
mi_switch() at mi_switch+0x208/frame 0xffffff88711d2950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711d2980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711d29=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711d2a00
_sleep() at _sleep+0x368/frame 0xffffff88711d2a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff88711d2ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff88711d2b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711d2c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711d2c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce58, rbp =3D 0x801407500 ---

Tracing command tcsh pid 1320 tid 100173 td 0xfffffe000de82920
sched_switch() at sched_switch+0x194/frame 0xffffff8871376900
mi_switch() at mi_switch+0x208/frame 0xffffff8871376950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871376980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713769=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871376a00
_sleep() at _sleep+0x368/frame 0xffffff8871376a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871376ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871376b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871376c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871376c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce68, rbp =3D 0x801408500 ---

Tracing command csh pid 1318 tid 100212 td 0xfffffe0028938920
sched_switch() at sched_switch+0x194/frame 0xffffff8871439900
mi_switch() at mi_switch+0x208/frame 0xffffff8871439950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871439980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714399=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871439a00
_sleep() at _sleep+0x368/frame 0xffffff8871439a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871439ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871439b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871439c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871439c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce58, rbp =3D 0x801407500 ---

Tracing command tcsh pid 1315 tid 100213 td 0xfffffe0028938490
sched_switch() at sched_switch+0x194/frame 0xffffff887143e900
mi_switch() at mi_switch+0x208/frame 0xffffff887143e950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887143e980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887143e9=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887143ea00
_sleep() at _sleep+0x368/frame 0xffffff887143ea90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff887143eae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff887143eb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887143ec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887143ec30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce68, rbp =3D 0x801408500 ---

Tracing command csh pid 1313 tid 100124 td 0xfffffe000df68490
sched_switch() at sched_switch+0x194/frame 0xffffff8871281900
mi_switch() at mi_switch+0x208/frame 0xffffff8871281950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871281980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712819=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871281a00
_sleep() at _sleep+0x368/frame 0xffffff8871281a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871281ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871281b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871281c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871281c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce58, rbp =3D 0x801407500 ---

Tracing command tcsh pid 1311 tid 100123 td 0xfffffe000df68920
sched_switch() at sched_switch+0x194/frame 0xffffff887127c900
mi_switch() at mi_switch+0x208/frame 0xffffff887127c950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887127c980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887127c9=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887127ca00
_sleep() at _sleep+0x368/frame 0xffffff887127ca90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff887127cae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff887127cb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887127cc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887127cc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce68, rbp =3D 0x801408500 ---

Tracing command csh pid 1309 tid 100093 td 0xfffffe000dec9920
sched_switch() at sched_switch+0x194/frame 0xffffff88711e6900
mi_switch() at mi_switch+0x208/frame 0xffffff88711e6950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711e6980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711e69=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711e6a00
_sleep() at _sleep+0x368/frame 0xffffff88711e6a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff88711e6ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff88711e6b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711e6c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711e6c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce58, rbp =3D 0x801407500 ---

Tracing command tcsh pid 1296 tid 100206 td 0xfffffe002896b000
sched_switch() at sched_switch+0x194/frame 0xffffff887141b900
mi_switch() at mi_switch+0x208/frame 0xffffff887141b950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887141b980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887141b9=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887141ba00
_sleep() at _sleep+0x368/frame 0xffffff887141ba90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff887141bae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff887141bb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887141bc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887141bc30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce68, rbp =3D 0x801408800 ---

Tracing command csh pid 1293 tid 100163 td 0xfffffe000dfdb000
sched_switch() at sched_switch+0x194/frame 0xffffff8871344900
mi_switch() at mi_switch+0x208/frame 0xffffff8871344950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871344980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713449=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871344a00
_sleep() at _sleep+0x368/frame 0xffffff8871344a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871344ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871344b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871344c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871344c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffce58, rbp =3D 0x801407500 ---

Tracing command tmux pid 1292 tid 100100 td 0xfffffe000de70490
cpustop_handler() at cpustop_handler+0x28/frame 0xffffff8000280d00
ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff8000280d20
trap() at trap+0x355/frame 0xffffff8000280f20
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff8000280f20
--- trap 0x13, rip =3D 0xffffffff808f01ad, rsp =3D 0xffffff8000280fe0, rbp =
=3D 0xffffff8871209910 ---
_mtx_lock_sleep() at _mtx_lock_sleep+0x17d/frame 0xffffff8871209910
_mtx_lock_flags() at _mtx_lock_flags+0x106/frame 0xffffff8871209940
sysctl_root() at sysctl_root+0x255/frame 0xffffff8871209990
userland_sysctl() at userland_sysctl+0x158/frame 0xffffff8871209a40
sys___sysctl() at sys___sysctl+0xaa/frame 0xffffff8871209b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871209c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871209c30
--- syscall (202, FreeBSD ELF64, sys___sysctl), rip =3D 0x800ff913c, rsp =
=3D 0x7fffffffcae8, rbp =3D 0x7fffffffcb00 ---

Tracing command tmux pid 1290 tid 100112 td 0xfffffe000df38920
sched_switch() at sched_switch+0x194/frame 0xffffff88712457c0
mi_switch() at mi_switch+0x208/frame 0xffffff8871245810
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871245840
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712458=
a0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712458c0
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff8871245920
seltdwait() at seltdwait+0xac/frame 0xffffff8871245950
sys_poll() at sys_poll+0x421/frame 0xffffff8871245b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871245c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871245c30
--- syscall (209, FreeBSD ELF64, sys_poll), rip =3D 0x800f92b3c, rsp =3D 0x=
7fffffffd0e8, rbp =3D 0x80144f100 ---

Tracing command hastd pid 1283 tid 100196 td 0xfffffe0028fda000
sched_switch() at sched_switch+0x194/frame 0xffffff88713e98b0
mi_switch() at mi_switch+0x208/frame 0xffffff88713e9900
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713e9930
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713e99=
90
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713e99b0
_sleep() at _sleep+0x368/frame 0xffffff88713e9a40
do_wait() at do_wait+0x3fd/frame 0xffffff88713e9ad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff88713e9b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713e9c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713e9c30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7ffffeff7ef8, rbp =3D 0 ---

Tracing command hastd pid 1283 tid 100195 td 0xfffffe000de97000
sched_switch() at sched_switch+0x194/frame 0xffffff88713e48b0
mi_switch() at mi_switch+0x208/frame 0xffffff88713e4900
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713e4930
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713e49=
90
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713e49b0
_sleep() at _sleep+0x368/frame 0xffffff88713e4a40
do_wait() at do_wait+0x3fd/frame 0xffffff88713e4ad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff88713e4b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713e4c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713e4c30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7fffff1f8ed8, rbp =3D 0 ---

Tracing command hastd pid 1283 tid 100194 td 0xfffffe000de97490
sched_switch() at sched_switch+0x194/frame 0xffffff88713df8a0
mi_switch() at mi_switch+0x208/frame 0xffffff88713df8f0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713df920
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713df9=
80
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88713df9b0
_sleep() at _sleep+0x3ae/frame 0xffffff88713dfa40
do_wait() at do_wait+0x53a/frame 0xffffff88713dfad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff88713dfb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713dfc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713dfc30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7fffff3f9eb8, rbp =3D 0 ---

Tracing command hastd pid 1283 tid 100193 td 0xfffffe000de97920
sched_switch() at sched_switch+0x194/frame 0xffffff88713da7f0
mi_switch() at mi_switch+0x208/frame 0xffffff88713da840
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713da870
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88713da8a0
_sleep() at _sleep+0x3d4/frame 0xffffff88713da930
bwait() at bwait+0x64/frame 0xffffff88713da960
physio() at physio+0x210/frame 0xffffff88713da9b0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff88713daa20
dofileread() at dofileread+0xa1/frame 0xffffff88713daa70
kern_preadv() at kern_preadv+0x68/frame 0xffffff88713daac0
sys_pread() at sys_pread+0x68/frame 0xffffff88713dab10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713dac30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713dac30
--- syscall (475, FreeBSD ELF64, sys_pread), rip =3D 0x80175596c, rsp =3D 0=
x7fffff5faf98, rbp =3D 0x801cb3700 ---

Tracing command hastd pid 1283 tid 100192 td 0xfffffe000de76000
sched_switch() at sched_switch+0x194/frame 0xffffff88713d56d0
mi_switch() at mi_switch+0x208/frame 0xffffff88713d5720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713d5750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713d57=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713d57d0
_sleep() at _sleep+0x368/frame 0xffffff88713d5860
g_gate_ioctl() at g_gate_ioctl+0x5d6/frame 0xffffff88713d5a00
devfs_ioctl_f() at devfs_ioctl_f+0x7a/frame 0xffffff88713d5a60
kern_ioctl() at kern_ioctl+0xcf/frame 0xffffff88713d5ab0
sys_ioctl() at sys_ioctl+0xfd/frame 0xffffff88713d5b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713d5c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713d5c30
--- syscall (54, FreeBSD ELF64, sys_ioctl), rip =3D 0x801756abc, rsp =3D 0x=
7fffff7fbf88, rbp =3D 0x801d00800 ---

Tracing command hastd pid 1283 tid 100191 td 0xfffffe000de76490
sched_switch() at sched_switch+0x194/frame 0xffffff88713d0780
mi_switch() at mi_switch+0x208/frame 0xffffff88713d07d0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713d0800
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713d08=
60
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713d0880
_sleep() at _sleep+0x368/frame 0xffffff88713d0910
soreceive_generic() at soreceive_generic+0xe5f/frame 0xffffff88713d09c0
kern_recvit() at kern_recvit+0x1c4/frame 0xffffff88713d0a80
recvit() at recvit+0x21/frame 0xffffff88713d0aa0
sys_recvfrom() at sys_recvfrom+0x82/frame 0xffffff88713d0b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713d0c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713d0c30
--- syscall (29, FreeBSD ELF64, sys_recvfrom), rip =3D 0x8016ffe0c, rsp =3D=
 0x7fffff9fcec8, rbp =3D 0x7fffff9fcf60 ---
       =20
Tracing command hastd pid 1283 tid 100190 td 0xfffffe000de76920
sched_switch() at sched_switch+0x194/frame 0xffffff88713cb7c0
mi_switch() at mi_switch+0x208/frame 0xffffff88713cb810
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713cb840
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713cb8=
a0
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88713cb8d0
_sleep() at _sleep+0x3ae/frame 0xffffff88713cb960
kern_sigtimedwait() at kern_sigtimedwait+0x280/frame 0xffffff88713cba40
sys_sigtimedwait() at sys_sigtimedwait+0x95/frame 0xffffff88713cbb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713cbc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713cbc30
--- syscall (345, FreeBSD ELF64, sys_sigtimedwait), rip =3D 0x8016b091c, rs=
p =3D 0x7fffffbfdf08, rbp =3D 0x5215a7e6 ---

Tracing command hastd pid 1283 tid 100130 td 0xfffffe000df9f000
sched_switch() at sched_switch+0x194/frame 0xffffff887129f8b0
mi_switch() at mi_switch+0x208/frame 0xffffff887129f900
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887129f930
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887129f9=
90
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887129f9b0
_sleep() at _sleep+0x368/frame 0xffffff887129fa40
do_wait() at do_wait+0x3fd/frame 0xffffff887129fad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff887129fb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887129fc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887129fc30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7fffffffcb68, rbp =3D 0 ---

Tracing command hastd pid 1282 tid 100203 td 0xfffffe002896b920
sched_switch() at sched_switch+0x194/frame 0xffffff887140c8b0
mi_switch() at mi_switch+0x208/frame 0xffffff887140c900
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887140c930
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887140c9=
90
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887140c9b0
_sleep() at _sleep+0x368/frame 0xffffff887140ca40
do_wait() at do_wait+0x3fd/frame 0xffffff887140cad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff887140cb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887140cc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887140cc30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7ffffeff7ef8, rbp =3D 0 ---

Tracing command hastd pid 1282 tid 100202 td 0xfffffe0028a14000
sched_switch() at sched_switch+0x194/frame 0xffffff88714078b0
mi_switch() at mi_switch+0x208/frame 0xffffff8871407900
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871407930
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714079=
90
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88714079b0
_sleep() at _sleep+0x368/frame 0xffffff8871407a40
do_wait() at do_wait+0x3fd/frame 0xffffff8871407ad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff8871407b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871407c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871407c30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7fffff1f8ed8, rbp =3D 0 ---

Tracing command hastd pid 1282 tid 100201 td 0xfffffe0028a14490
sched_switch() at sched_switch+0x194/frame 0xffffff88714028a0
mi_switch() at mi_switch+0x208/frame 0xffffff88714028f0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871402920
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714029=
80
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88714029b0
_sleep() at _sleep+0x3ae/frame 0xffffff8871402a40
do_wait() at do_wait+0x53a/frame 0xffffff8871402ad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff8871402b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871402c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871402c30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7fffff3f9eb8, rbp =3D 0 ---

Tracing command hastd pid 1282 tid 100200 td 0xfffffe0028a14920
sched_switch() at sched_switch+0x194/frame 0xffffff88713fd8b0
mi_switch() at mi_switch+0x208/frame 0xffffff88713fd900
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713fd930
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713fd9=
90
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713fd9b0
_sleep() at _sleep+0x368/frame 0xffffff88713fda40
do_wait() at do_wait+0x3fd/frame 0xffffff88713fdad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff88713fdb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713fdc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713fdc30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7fffff5faef8, rbp =3D 0 ---

Tracing command hastd pid 1282 tid 100199 td 0xfffffe0028a15000
sched_switch() at sched_switch+0x194/frame 0xffffff88713f88b0
mi_switch() at mi_switch+0x208/frame 0xffffff88713f8900
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713f8930
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713f89=
90
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713f89b0
_sleep() at _sleep+0x368/frame 0xffffff88713f8a40
do_wait() at do_wait+0x3fd/frame 0xffffff88713f8ad0
__umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf=
fffff88713f8b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713f8c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713f8c30
--- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp =
=3D 0x7fffff7fbee8, rbp =3D 0 ---

Tracing command hastd pid 1282 tid 100198 td 0xfffffe0028a15490
sched_switch() at sched_switch+0x194/frame 0xffffff88713f3780
mi_switch() at mi_switch+0x208/frame 0xffffff88713f37d0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713f3800
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713f38=
60
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713f3880
_sleep() at _sleep+0x368/frame 0xffffff88713f3910
soreceive_generic() at soreceive_generic+0xe5f/frame 0xffffff88713f39c0
kern_recvit() at kern_recvit+0x1c4/frame 0xffffff88713f3a80
recvit() at recvit+0x21/frame 0xffffff88713f3aa0
sys_recvfrom() at sys_recvfrom+0x82/frame 0xffffff88713f3b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713f3c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713f3c30
--- syscall (29, FreeBSD ELF64, sys_recvfrom), rip =3D 0x8016ffe0c, rsp =3D=
 0x7fffff9fcec8, rbp =3D 0x7fffff9fcf60 ---

Tracing command hastd pid 1282 tid 100197 td 0xfffffe0028a15920
sched_switch() at sched_switch+0x194/frame 0xffffff88713ee7c0
mi_switch() at mi_switch+0x208/frame 0xffffff88713ee810
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713ee840
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713ee8=
a0
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88713ee8d0
_sleep() at _sleep+0x3ae/frame 0xffffff88713ee960
kern_sigtimedwait() at kern_sigtimedwait+0x280/frame 0xffffff88713eea40
sys_sigtimedwait() at sys_sigtimedwait+0x95/frame 0xffffff88713eeb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713eec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713eec30
--- syscall (345, FreeBSD ELF64, sys_sigtimedwait), rip =3D 0x8016b091c, rs=
p =3D 0x7fffffbfdf08, rbp =3D 0x5215a7e6 ---

Tracing command hastd pid 1282 tid 100117 td 0xfffffe000daee920
sched_switch() at sched_switch+0x194/frame 0xffffff887125e7f0
mi_switch() at mi_switch+0x208/frame 0xffffff887125e840
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887125e870
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887125e8a0
_sleep() at _sleep+0x3d4/frame 0xffffff887125e930
bwait() at bwait+0x64/frame 0xffffff887125e960
physio() at physio+0x2b4/frame 0xffffff887125e9b0
devfs_write_f() at devfs_write_f+0xb1/frame 0xffffff887125ea20
dofilewrite() at dofilewrite+0x85/frame 0xffffff887125ea70
kern_pwritev() at kern_pwritev+0x68/frame 0xffffff887125eac0
sys_pwrite() at sys_pwrite+0x68/frame 0xffffff887125eb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887125ec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887125ec30
--- syscall (476, FreeBSD ELF64, sys_pwrite), rip =3D 0x80175594c, rsp =3D =
0x7fffffffcbd8, rbp =3D 0x29f8800000 ---

Tracing command csh pid 1276 tid 100111 td 0xfffffe000dec9000
sched_switch() at sched_switch+0x194/frame 0xffffff8871240900
mi_switch() at mi_switch+0x208/frame 0xffffff8871240950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871240980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712409=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871240a00
_sleep() at _sleep+0x368/frame 0xffffff8871240a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871240ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871240b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871240c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871240c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffcea8, rbp =3D 0x80140b600 ---

Tracing command su pid 1275 tid 100129 td 0xfffffe000df9f490
sched_switch() at sched_switch+0x194/frame 0xffffff887129a6d0
mi_switch() at mi_switch+0x208/frame 0xffffff887129a720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887129a750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887129a7=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887129a7d0
_sleep() at _sleep+0x368/frame 0xffffff887129a860
kern_wait6() at kern_wait6+0x5e2/frame 0xffffff887129a8f0
kern_wait() at kern_wait+0x9c/frame 0xffffff887129aa50
sys_wait4() at sys_wait4+0x35/frame 0xffffff887129ab10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887129ac30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887129ac30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x800ed34ac, rsp =3D 0x7=
fffffffd578, rbp =3D 0x4fc ---

Tracing command csh pid 1272 tid 100166 td 0xfffffe000dfca490
sched_switch() at sched_switch+0x194/frame 0xffffff8871353900
mi_switch() at mi_switch+0x208/frame 0xffffff8871353950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871353980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713539=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871353a00
_sleep() at _sleep+0x368/frame 0xffffff8871353a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871353ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871353b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871353c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871353c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffcfa8, rbp =3D 0x80141c900 ---

Tracing command sshd pid 1271 tid 100116 td 0xfffffe000df69490
sched_switch() at sched_switch+0x194/frame 0xffffff8871259710
mi_switch() at mi_switch+0x208/frame 0xffffff8871259760
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871259790
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712597=
f0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871259810
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff8871259870
seltdwait() at seltdwait+0xac/frame 0xffffff88712598a0
kern_select() at kern_select+0x6f0/frame 0xffffff8871259ac0
sys_select() at sys_select+0x5d/frame 0xffffff8871259b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871259c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871259c30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x8024f8a9c, rsp =3D 0=
x7fffffffcc68, rbp =3D 0x7fffffffccf0 ---

Tracing command sshd pid 1269 tid 100187 td 0xfffffe000daec000
sched_switch() at sched_switch+0x194/frame 0xffffff88713bc7c0
mi_switch() at mi_switch+0x208/frame 0xffffff88713bc810
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713bc840
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713bc8=
a0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713bc8c0
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88713bc920
seltdwait() at seltdwait+0xac/frame 0xffffff88713bc950
sys_poll() at sys_poll+0x421/frame 0xffffff88713bcb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713bcc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713bcc30
--- syscall (209, FreeBSD ELF64, sys_poll), rip =3D 0x802492b3c, rsp =3D 0x=
7fffffffcd08, rbp =3D 0x803017190 ---

Tracing command csh pid 1267 tid 100185 td 0xfffffe000daec490
sched_switch() at sched_switch+0x194/frame 0xffffff88713b2900
mi_switch() at mi_switch+0x208/frame 0xffffff88713b2950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713b2980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713b29=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713b2a00
_sleep() at _sleep+0x368/frame 0xffffff88713b2a90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff88713b2ae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff88713b2b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713b2c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713b2c30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7=
fffffffd048, rbp =3D 0x801460400 ---

Tracing command login pid 1261 tid 100179 td 0xfffffe000daec920
sched_switch() at sched_switch+0x194/frame 0xffffff88713946d0
mi_switch() at mi_switch+0x208/frame 0xffffff8871394720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871394750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713947=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713947d0
_sleep() at _sleep+0x368/frame 0xffffff8871394860
kern_wait6() at kern_wait6+0x5e2/frame 0xffffff88713948f0
kern_wait() at kern_wait+0x9c/frame 0xffffff8871394a50
sys_wait4() at sys_wait4+0x35/frame 0xffffff8871394b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871394c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871394c30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x800ed54ac, rsp =3D 0x7=
fffffffdc08, rbp =3D 0 ---

Tracing command getty pid 1260 tid 100178 td 0xfffffe000daed000
sched_switch() at sched_switch+0x194/frame 0xffffff887138f770
mi_switch() at mi_switch+0x208/frame 0xffffff887138f7c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887138f7f0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887138f8=
50
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887138f870
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff887138f8d0
tty_wait() at tty_wait+0x48/frame 0xffffff887138f900
ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff887138f990
ttydev_read() at ttydev_read+0x8e/frame 0xffffff887138f9c0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff887138fa30
dofileread() at dofileread+0xa1/frame 0xffffff887138fa80
kern_readv() at kern_readv+0x6c/frame 0xffffff887138fac0
sys_read() at sys_read+0x64/frame 0xffffff887138fb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887138fc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887138fc30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f=
ffffffdca8, rbp =3D 0 ---

Tracing command getty pid 1259 tid 100177 td 0xfffffe000daed490
sched_switch() at sched_switch+0x194/frame 0xffffff887138a770
mi_switch() at mi_switch+0x208/frame 0xffffff887138a7c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887138a7f0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887138a8=
50
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887138a870
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff887138a8d0
tty_wait() at tty_wait+0x48/frame 0xffffff887138a900
ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff887138a990
ttydev_read() at ttydev_read+0x8e/frame 0xffffff887138a9c0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff887138aa30
dofileread() at dofileread+0xa1/frame 0xffffff887138aa80
kern_readv() at kern_readv+0x6c/frame 0xffffff887138aac0
sys_read() at sys_read+0x64/frame 0xffffff887138ab10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887138ac30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887138ac30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f=
ffffffdca8, rbp =3D 0 ---

Tracing command getty pid 1258 tid 100176 td 0xfffffe000daed920
sched_switch() at sched_switch+0x194/frame 0xffffff8871385770
mi_switch() at mi_switch+0x208/frame 0xffffff88713857c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713857f0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713858=
50
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871385870
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88713858d0
tty_wait() at tty_wait+0x48/frame 0xffffff8871385900
ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff8871385990
ttydev_read() at ttydev_read+0x8e/frame 0xffffff88713859c0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff8871385a30
dofileread() at dofileread+0xa1/frame 0xffffff8871385a80
kern_readv() at kern_readv+0x6c/frame 0xffffff8871385ac0
sys_read() at sys_read+0x64/frame 0xffffff8871385b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871385c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871385c30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f=
ffffffdca8, rbp =3D 0 ---

Tracing command getty pid 1257 tid 100134 td 0xfffffe000de80490
sched_switch() at sched_switch+0x194/frame 0xffffff88712b3770
mi_switch() at mi_switch+0x208/frame 0xffffff88712b37c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712b37f0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712b38=
50
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712b3870
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88712b38d0
tty_wait() at tty_wait+0x48/frame 0xffffff88712b3900
ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff88712b3990
ttydev_read() at ttydev_read+0x8e/frame 0xffffff88712b39c0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff88712b3a30
dofileread() at dofileread+0xa1/frame 0xffffff88712b3a80
kern_readv() at kern_readv+0x6c/frame 0xffffff88712b3ac0
sys_read() at sys_read+0x64/frame 0xffffff88712b3b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712b3c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712b3c30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f=
ffffffdca8, rbp =3D 0 ---

Tracing command getty pid 1256 tid 100107 td 0xfffffe000df1f000
sched_switch() at sched_switch+0x194/frame 0xffffff887122c770
mi_switch() at mi_switch+0x208/frame 0xffffff887122c7c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887122c7f0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887122c8=
50
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887122c870
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff887122c8d0
tty_wait() at tty_wait+0x48/frame 0xffffff887122c900
ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff887122c990
ttydev_read() at ttydev_read+0x8e/frame 0xffffff887122c9c0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff887122ca30
dofileread() at dofileread+0xa1/frame 0xffffff887122ca80
kern_readv() at kern_readv+0x6c/frame 0xffffff887122cac0
sys_read() at sys_read+0x64/frame 0xffffff887122cb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887122cc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887122cc30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f=
ffffffdca8, rbp =3D 0 ---

Tracing command getty pid 1255 tid 100128 td 0xfffffe000daee490
sched_switch() at sched_switch+0x194/frame 0xffffff8871295770
mi_switch() at mi_switch+0x208/frame 0xffffff88712957c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712957f0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712958=
50
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871295870
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88712958d0
tty_wait() at tty_wait+0x48/frame 0xffffff8871295900
ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff8871295990
ttydev_read() at ttydev_read+0x8e/frame 0xffffff88712959c0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff8871295a30
dofileread() at dofileread+0xa1/frame 0xffffff8871295a80
kern_readv() at kern_readv+0x6c/frame 0xffffff8871295ac0
sys_read() at sys_read+0x64/frame 0xffffff8871295b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871295c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871295c30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f=
ffffffdca8, rbp =3D 0 ---

Tracing command getty pid 1254 tid 100076 td 0xfffffe000daef490
sched_switch() at sched_switch+0x194/frame 0xffffff8871191770
mi_switch() at mi_switch+0x208/frame 0xffffff88711917c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711917f0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711918=
50
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871191870
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88711918d0
tty_wait() at tty_wait+0x48/frame 0xffffff8871191900
ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff8871191990
ttydev_read() at ttydev_read+0x8e/frame 0xffffff88711919c0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff8871191a30
dofileread() at dofileread+0xa1/frame 0xffffff8871191a80
kern_readv() at kern_readv+0x6c/frame 0xffffff8871191ac0
sys_read() at sys_read+0x64/frame 0xffffff8871191b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871191c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871191c30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f=
ffffffdca8, rbp =3D 0 ---

Tracing command getty pid 1253 tid 100084 td 0xfffffe000decb000
sched_switch() at sched_switch+0x194/frame 0xffffff88711b9770
mi_switch() at mi_switch+0x208/frame 0xffffff88711b97c0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711b97f0
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711b98=
50
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711b9870
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88711b98d0
tty_wait() at tty_wait+0x48/frame 0xffffff88711b9900
ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff88711b9990
ttydev_read() at ttydev_read+0x8e/frame 0xffffff88711b99c0
devfs_read_f() at devfs_read_f+0x90/frame 0xffffff88711b9a30
dofileread() at dofileread+0xa1/frame 0xffffff88711b9a80
kern_readv() at kern_readv+0x6c/frame 0xffffff88711b9ac0
sys_read() at sys_read+0x64/frame 0xffffff88711b9b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711b9c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711b9c30
--- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f=
ffffffdca8, rbp =3D 0 ---

Tracing command cron pid 1219 tid 100167 td 0xfffffe000dec4920
sched_switch() at sched_switch+0x194/frame 0xffffff88713588a0
mi_switch() at mi_switch+0x208/frame 0xffffff88713588f0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871358920
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713589=
80
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88713589b0
_sleep() at _sleep+0x3ae/frame 0xffffff8871358a40
kern_nanosleep() at kern_nanosleep+0x118/frame 0xffffff8871358ab0
sys_nanosleep() at sys_nanosleep+0x6e/frame 0xffffff8871358b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871358c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871358c30
--- syscall (240, FreeBSD ELF64, sys_nanosleep), rip =3D 0x800d2be2c, rsp =
=3D 0x7fffffffdae8, rbp =3D 0x3c ---

Tracing command sshd pid 1204 tid 100090 td 0xfffffe000deca920
sched_switch() at sched_switch+0x194/frame 0xffffff88711d7710
mi_switch() at mi_switch+0x208/frame 0xffffff88711d7760
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711d7790
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711d77=
f0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711d7810
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88711d7870
seltdwait() at seltdwait+0xac/frame 0xffffff88711d78a0
kern_select() at kern_select+0x6f0/frame 0xffffff88711d7ac0
sys_select() at sys_select+0x5d/frame 0xffffff88711d7b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711d7c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711d7c30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x8024f8a9c, rsp =3D 0=
x7fffffffcd88, rbp =3D 0x2 ---
       =20
Tracing command exim-4.80.1-2 pid 1198 tid 100135 td 0xfffffe000de6f000
sched_switch() at sched_switch+0x194/frame 0xffffff88712b8710
mi_switch() at mi_switch+0x208/frame 0xffffff88712b8760
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712b8790
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712b87=
f0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712b8810
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88712b8870
seltdwait() at seltdwait+0xac/frame 0xffffff88712b88a0
kern_select() at kern_select+0x6f0/frame 0xffffff88712b8ac0
sys_select() at sys_select+0x5d/frame 0xffffff88712b8b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712b8c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712b8c30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x80234ba9c, rsp =3D 0=
x7fffffffc6b8, rbp =3D 0x5 ---

Tracing command nrpe2 pid 1185 tid 100102 td 0xfffffe000ddbc490
sched_switch() at sched_switch+0x194/frame 0xffffff8871213700
mi_switch() at mi_switch+0x208/frame 0xffffff8871213750
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871213780
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712137=
e0
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff8871213810
_cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff8871213870
seltdwait() at seltdwait+0x56/frame 0xffffff88712138a0
kern_select() at kern_select+0x6f0/frame 0xffffff8871213ac0
sys_select() at sys_select+0x5d/frame 0xffffff8871213b10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871213c30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871213c30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x800b34a9c, rsp =3D 0=
x7fffffffd038, rbp =3D 0x7fffffffdd10 ---

Tracing command hastd pid 1124 tid 100133 td 0xfffffe000de6f490
sched_switch() at sched_switch+0x194/frame 0xffffff88712ae700
mi_switch() at mi_switch+0x208/frame 0xffffff88712ae750
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712ae780
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712ae7=
e0
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88712ae810
_cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff88712ae870
seltdwait() at seltdwait+0x56/frame 0xffffff88712ae8a0
kern_select() at kern_select+0x6f0/frame 0xffffff88712aeac0
sys_select() at sys_select+0x5d/frame 0xffffff88712aeb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712aec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712aec30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x801756a9c, rsp =3D 0=
x7fffffffd898, rbp =3D 0x12 ---

Tracing command syslogd pid 1093 tid 100094 td 0xfffffe000df3a920
sched_switch() at sched_switch+0x194/frame 0xffffff88711eb710
mi_switch() at mi_switch+0x208/frame 0xffffff88711eb760
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711eb790
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711eb7=
f0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711eb810
_cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88711eb870
seltdwait() at seltdwait+0xac/frame 0xffffff88711eb8a0
kern_select() at kern_select+0x6f0/frame 0xffffff88711ebac0
sys_select() at sys_select+0x5d/frame 0xffffff88711ebb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711ebc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711ebc30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x800b3ea9c, rsp =3D 0=
x7fffffffd128, rbp =3D 0x8010a00c8 ---

Tracing command devd pid 930 tid 100155 td 0xfffffe000df89000
sched_switch() at sched_switch+0x194/frame 0xffffff887131c700
mi_switch() at mi_switch+0x208/frame 0xffffff887131c750
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887131c780
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887131c7=
e0
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff887131c810
_cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff887131c870
seltdwait() at seltdwait+0x56/frame 0xffffff887131c8a0
kern_select() at kern_select+0x6f0/frame 0xffffff887131cac0
sys_select() at sys_select+0x5d/frame 0xffffff887131cb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887131cc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887131cc30
--- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x447f1c, rsp =3D 0x7f=
ffffffbc38, rbp =3D 0x10 ---

Tracing command adjkerntz pid 135 tid 100149 td 0xfffffe000de6e000
sched_switch() at sched_switch+0x194/frame 0xffffff88712fe900
mi_switch() at mi_switch+0x208/frame 0xffffff88712fe950
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712fe980
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712fe9=
e0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712fea00
_sleep() at _sleep+0x368/frame 0xffffff88712fea90
kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff88712feae0
sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff88712feb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712fec30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712fec30
--- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x80087e99c, rsp =3D 0x7=
fffffffdc28, rbp =3D 0x1 ---

Tracing command g_mirror gmrootfs pid 18 tid 100075 td 0xfffffe000ddbd920
sched_switch() at sched_switch+0x194/frame 0xffffff887118c9d0
mi_switch() at mi_switch+0x208/frame 0xffffff887118ca20
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887118ca50
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887118ca80
_sleep() at _sleep+0x3d4/frame 0xffffff887118cb10
g_mirror_worker() at g_mirror_worker+0xb4d/frame 0xffffff887118cbe0
fork_exit() at fork_exit+0x135/frame 0xffffff887118cc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff887118cc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff887118ccf0, rbp =3D 0 ---

Tracing command softdepflush pid 17 tid 100074 td 0xfffffe000daef920
sched_switch() at sched_switch+0x194/frame 0xffffff884afbba60
mi_switch() at mi_switch+0x208/frame 0xffffff884afbbab0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afbbae0
sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884afbbb10
_sleep() at _sleep+0x26d/frame 0xffffff884afbbba0
softdep_flush() at softdep_flush+0x26a/frame 0xffffff884afbbbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884afbbc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afbbc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884afbbcf0, rbp =3D 0 ---

Tracing command syncer pid 16 tid 100073 td 0xfffffe000daf2000
sched_switch() at sched_switch+0x194/frame 0xffffff884afb6800
mi_switch() at mi_switch+0x208/frame 0xffffff884afb6850
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afb6880
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884afb68b0
_sleep() at _sleep+0x3d4/frame 0xffffff884afb6940
getpbuf() at getpbuf+0x4e/frame 0xffffff884afb6960
cluster_wbuild_gb() at cluster_wbuild_gb+0x21e/frame 0xffffff884afb6a10
ffs_syncvnode() at ffs_syncvnode+0x2fa/frame 0xffffff884afb6a80
ffs_fsync() at ffs_fsync+0x45/frame 0xffffff884afb6ac0
VOP_FSYNC_APV() at VOP_FSYNC_APV+0x105/frame 0xffffff884afb6af0
sync_vnode() at sync_vnode+0x157/frame 0xffffff884afb6b70
sched_sync() at sched_sync+0x1c7/frame 0xffffff884afb6be0
fork_exit() at fork_exit+0x135/frame 0xffffff884afb6c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afb6c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884afb6cf0, rbp =3D 0 ---

Tracing command vnlru pid 9 tid 100072 td 0xfffffe000d98f000
sched_switch() at sched_switch+0x194/frame 0xffffff884afb1a10
mi_switch() at mi_switch+0x208/frame 0xffffff884afb1a60
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afb1a90
sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884afb1ac0
_sleep() at _sleep+0x26d/frame 0xffffff884afb1b50
vnlru_proc() at vnlru_proc+0x58a/frame 0xffffff884afb1be0
fork_exit() at fork_exit+0x135/frame 0xffffff884afb1c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afb1c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884afb1cf0, rbp =3D 0 ---

Tracing command bufdaemon pid 8 tid 100071 td 0xfffffe000d98f490
sched_switch() at sched_switch+0x194/frame 0xffffff884afac8f0
mi_switch() at mi_switch+0x208/frame 0xffffff884afac940
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afac970
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884afac9a0
_sleep() at _sleep+0x3d4/frame 0xffffff884afaca30
getpbuf() at getpbuf+0x4e/frame 0xffffff884afaca50
cluster_wbuild_gb() at cluster_wbuild_gb+0x21e/frame 0xffffff884afacb00
flushbufqueues() at flushbufqueues+0x72e/frame 0xffffff884afacba0
buf_do_flush() at buf_do_flush+0x16/frame 0xffffff884afacbc0
buf_daemon() at buf_daemon+0xe4/frame 0xffffff884afacbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884afacc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afacc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884afaccf0, rbp =3D 0 ---

Tracing command pagezero pid 7 tid 100070 td 0xfffffe000d98f920
sched_switch() at sched_switch+0x194/frame 0xffffff884afa7a80
mi_switch() at mi_switch+0x208/frame 0xffffff884afa7ad0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afa7b00
sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884afa7b30
_sleep() at _sleep+0x26d/frame 0xffffff884afa7bc0
vm_pagezero() at vm_pagezero+0x73/frame 0xffffff884afa7be0
fork_exit() at fork_exit+0x135/frame 0xffffff884afa7c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afa7c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884afa7cf0, rbp =3D 0 ---

Tracing command vmdaemon pid 6 tid 100069 td 0xfffffe000d991000
sched_switch() at sched_switch+0x194/frame 0xffffff884afa2a40
mi_switch() at mi_switch+0x208/frame 0xffffff884afa2a90
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afa2ac0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884afa2af0
_sleep() at _sleep+0x3d4/frame 0xffffff884afa2b80
vm_daemon() at vm_daemon+0x4d/frame 0xffffff884afa2be0
fork_exit() at fork_exit+0x135/frame 0xffffff884afa2c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afa2c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884afa2cf0, rbp =3D 0 ---

Tracing command pagedaemon pid 5 tid 100068 td 0xfffffe000d991490
sched_switch() at sched_switch+0x194/frame 0xffffff884af9d980
mi_switch() at mi_switch+0x208/frame 0xffffff884af9d9d0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af9da00
sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884af9da30
_sleep() at _sleep+0x26d/frame 0xffffff884af9dac0
vm_pageout() at vm_pageout+0x269/frame 0xffffff884af9dbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884af9dc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af9dc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884af9dcf0, rbp =3D 0 ---

Tracing command xpt_thrd pid 4 tid 100067 td 0xfffffe000d991920
sched_switch() at sched_switch+0x194/frame 0xffffff884af98a80
mi_switch() at mi_switch+0x208/frame 0xffffff884af98ad0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af98b00
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884af98b30
_sleep() at _sleep+0x3d4/frame 0xffffff884af98bc0
xpt_scanner_thread() at xpt_scanner_thread+0xd5/frame 0xffffff884af98be0
fork_exit() at fork_exit+0x135/frame 0xffffff884af98c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af98c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884af98cf0, rbp =3D 0 ---

Tracing command sctp_iterator pid 3 tid 100066 td 0xfffffe000d993000
sched_switch() at sched_switch+0x194/frame 0xffffff884af93a90
mi_switch() at mi_switch+0x208/frame 0xffffff884af93ae0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af93b10
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884af93b40
_sleep() at _sleep+0x3d4/frame 0xffffff884af93bd0
sctp_iterator_thread() at sctp_iterator_thread+0x3f/frame 0xffffff884af93be0
fork_exit() at fork_exit+0x135/frame 0xffffff884af93c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af93c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884af93cf0, rbp =3D 0 ---

Tracing command fdc0 pid 2 tid 100063 td 0xfffffe000d994000
sched_switch() at sched_switch+0x194/frame 0xffffff884af7e9f0
mi_switch() at mi_switch+0x208/frame 0xffffff884af7ea40
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af7ea70
sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884af7eaa0
_sleep() at _sleep+0x26d/frame 0xffffff884af7eb30
fdc_thread() at fdc_thread+0x74a/frame 0xffffff884af7ebe0
fork_exit() at fork_exit+0x135/frame 0xffffff884af7ec30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af7ec30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884af7ecf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100059 td 0xfffffe000a532920
sched_switch() at sched_switch+0x194/frame 0xffffff884aee2aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aee2af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aee2b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aee2b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aee2bb0
usb_process() at usb_process+0x16b/frame 0xffffff884aee2be0
fork_exit() at fork_exit+0x135/frame 0xffffff884aee2c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aee2c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aee2cf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100058 td 0xfffffe000d700000
sched_switch() at sched_switch+0x194/frame 0xffffff884aeddaa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aeddaf0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aeddb20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aeddb50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aeddbb0
usb_process() at usb_process+0x16b/frame 0xffffff884aeddbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884aeddc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aeddc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aeddcf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100057 td 0xfffffe000d700490
sched_switch() at sched_switch+0x194/frame 0xffffff884aed8aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aed8af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aed8b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aed8b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aed8bb0
usb_process() at usb_process+0x16b/frame 0xffffff884aed8be0
fork_exit() at fork_exit+0x135/frame 0xffffff884aed8c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aed8c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aed8cf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100056 td 0xfffffe000d700920
sched_switch() at sched_switch+0x194/frame 0xffffff884aed3aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aed3af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aed3b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aed3b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aed3bb0
usb_process() at usb_process+0x16b/frame 0xffffff884aed3be0
fork_exit() at fork_exit+0x135/frame 0xffffff884aed3c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aed3c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aed3cf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100055 td 0xfffffe000d701000
sched_switch() at sched_switch+0x194/frame 0xffffff884ad4baa0
mi_switch() at mi_switch+0x208/frame 0xffffff884ad4baf0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ad4bb20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ad4bb50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884ad4bbb0
usb_process() at usb_process+0x16b/frame 0xffffff884ad4bbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884ad4bc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ad4bc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884ad4bcf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100054 td 0xfffffe000d701490
sched_switch() at sched_switch+0x194/frame 0xffffff884ad46aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884ad46af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ad46b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ad46b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884ad46bb0
usb_process() at usb_process+0x16b/frame 0xffffff884ad46be0
fork_exit() at fork_exit+0x135/frame 0xffffff884ad46c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ad46c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884ad46cf0, rbp =3D 0 ---
       =20
Tracing command usb pid 15 tid 100053 td 0xfffffe000d701920
sched_switch() at sched_switch+0x194/frame 0xffffff884ad41aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884ad41af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ad41b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ad41b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884ad41bb0
usb_process() at usb_process+0x16b/frame 0xffffff884ad41be0
fork_exit() at fork_exit+0x135/frame 0xffffff884ad41c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ad41c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884ad41cf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100052 td 0xfffffe000d702000
sched_switch() at sched_switch+0x194/frame 0xffffff884ad3caa0
mi_switch() at mi_switch+0x208/frame 0xffffff884ad3caf0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ad3cb20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ad3cb50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884ad3cbb0
usb_process() at usb_process+0x16b/frame 0xffffff884ad3cbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884ad3cc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ad3cc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884ad3ccf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100050 td 0xfffffe000d702920
sched_switch() at sched_switch+0x194/frame 0xffffff884abacaa0
mi_switch() at mi_switch+0x208/frame 0xffffff884abacaf0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884abacb20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884abacb50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884abacbb0
usb_process() at usb_process+0x16b/frame 0xffffff884abacbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884abacc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884abacc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884abaccf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100049 td 0xfffffe000a52e000
sched_switch() at sched_switch+0x194/frame 0xffffff884aba7aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aba7af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aba7b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aba7b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aba7bb0
usb_process() at usb_process+0x16b/frame 0xffffff884aba7be0
fork_exit() at fork_exit+0x135/frame 0xffffff884aba7c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aba7c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aba7cf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100048 td 0xfffffe000a52e490
sched_switch() at sched_switch+0x194/frame 0xffffff884aba2aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aba2af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aba2b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aba2b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aba2bb0
usb_process() at usb_process+0x16b/frame 0xffffff884aba2be0
fork_exit() at fork_exit+0x135/frame 0xffffff884aba2c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aba2c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aba2cf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100047 td 0xfffffe000a52e920
sched_switch() at sched_switch+0x194/frame 0xffffff884ab9daa0
mi_switch() at mi_switch+0x208/frame 0xffffff884ab9daf0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ab9db20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ab9db50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884ab9dbb0
usb_process() at usb_process+0x16b/frame 0xffffff884ab9dbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884ab9dc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ab9dc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884ab9dcf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100045 td 0xfffffe000a52f490
sched_switch() at sched_switch+0x194/frame 0xffffff884aa0daa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aa0daf0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aa0db20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aa0db50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aa0dbb0
usb_process() at usb_process+0x16b/frame 0xffffff884aa0dbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884aa0dc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aa0dc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aa0dcf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100044 td 0xfffffe000a52f920
sched_switch() at sched_switch+0x194/frame 0xffffff884aa08aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aa08af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aa08b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aa08b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aa08bb0
usb_process() at usb_process+0x16b/frame 0xffffff884aa08be0
fork_exit() at fork_exit+0x135/frame 0xffffff884aa08c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aa08c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aa08cf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100043 td 0xfffffe000a531000
sched_switch() at sched_switch+0x194/frame 0xffffff884aa03aa0
mi_switch() at mi_switch+0x208/frame 0xffffff884aa03af0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aa03b20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aa03b50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884aa03bb0
usb_process() at usb_process+0x16b/frame 0xffffff884aa03be0
fork_exit() at fork_exit+0x135/frame 0xffffff884aa03c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aa03c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884aa03cf0, rbp =3D 0 ---

Tracing command usb pid 15 tid 100042 td 0xfffffe000a531490
sched_switch() at sched_switch+0x194/frame 0xffffff884a9feaa0
mi_switch() at mi_switch+0x208/frame 0xffffff884a9feaf0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884a9feb20
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884a9feb50
_cv_wait() at _cv_wait+0x17d/frame 0xffffff884a9febb0
usb_process() at usb_process+0x16b/frame 0xffffff884a9febe0
fork_exit() at fork_exit+0x135/frame 0xffffff884a9fec30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884a9fec30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884a9fecf0, rbp =3D 0 ---

Tracing command yarrow pid 14 tid 100025 td 0xfffffe000a2dd000
sched_switch() at sched_switch+0x194/frame 0xffffff8000300a60
mi_switch() at mi_switch+0x208/frame 0xffffff8000300ab0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000300ae0
sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff8000300b10
_sleep() at _sleep+0x26d/frame 0xffffff8000300ba0
random_kthread() at random_kthread+0x1ad/frame 0xffffff8000300be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000300c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000300c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000300cf0, rbp =3D 0 ---

Tracing command geom pid 13 tid 100023 td 0xfffffe000a2dd920
sched_switch() at sched_switch+0x194/frame 0xffffff80002f5a30
mi_switch() at mi_switch+0x208/frame 0xffffff80002f5a80
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff80002f5ab0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff80002f5ae0
_sleep() at _sleep+0x3d4/frame 0xffffff80002f5b70
g_io_schedule_down() at g_io_schedule_down+0x2bc/frame 0xffffff80002f5bd0
g_down_procbody() at g_down_procbody+0x72/frame 0xffffff80002f5be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002f5c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002f5c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002f5cf0, rbp =3D 0 ---

Tracing command geom pid 13 tid 100022 td 0xfffffe000a2d3000
cpustop_handler() at cpustop_handler+0x28/frame 0xffffff8000264d00
ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff8000264d20
trap() at trap+0x355/frame 0xffffff8000264f20
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff8000264f20
--- trap 0x13, rip =3D 0xffffffff8092c60f, rsp =3D 0xffffff8000264fe0, rbp =
=3D 0xffffff80002f0850 ---
cpu_search_lowest() at cpu_search_lowest+0x8f/frame 0xffffff80002f0850
cpu_search_lowest() at cpu_search_lowest+0x18e/frame 0xffffff80002f08e0
sched_pickcpu() at sched_pickcpu+0x2b4/frame 0xffffff80002f0950
sched_add() at sched_add+0x65/frame 0xffffff80002f0990
intr_event_schedule_thread() at intr_event_schedule_thread+0xac/frame 0xfff=
fff80002f09d0
intr_event_handle() at intr_event_handle+0xe8/frame 0xffffff80002f0a10
intr_execute_handlers() at intr_execute_handlers+0x5f/frame 0xffffff80002f0=
a40
lapic_handle_intr() at lapic_handle_intr+0x32/frame 0xffffff80002f0a70
Xapic_isr1() at Xapic_isr1+0xa0/frame 0xffffff80002f0a70
--- interrupt, rip =3D 0xffffffff80cda633, rsp =3D 0xffffff80002f0b30, rbp =
=3D 0xffffff80002f0b50 ---
spinlock_exit() at spinlock_exit+0x33/frame 0xffffff80002f0b50
pmap_invalidate_range() at pmap_invalidate_range+0xc9/frame 0xffffff80002f0=
b70
biodone() at biodone+0xaa/frame 0xffffff80002f0bb0
g_io_schedule_up() at g_io_schedule_up+0xcd/frame 0xffffff80002f0bd0
g_up_procbody() at g_up_procbody+0x72/frame 0xffffff80002f0be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002f0c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002f0c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002f0cf0, rbp =3D 0 ---

Tracing command geom pid 13 tid 100021 td 0xfffffe000a2d3490
sched_switch() at sched_switch+0x194/frame 0xffffff80002eba70
mi_switch() at mi_switch+0x208/frame 0xffffff80002ebac0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff80002ebaf0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff80002ebb20
_sleep() at _sleep+0x3d4/frame 0xffffff80002ebbb0
g_run_events() at g_run_events+0x4c1/frame 0xffffff80002ebbe0
fork_exit() at fork_exit+0x135/frame 0xffffff80002ebc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002ebc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002ebcf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100064 td 0xfffffe000d993920
fork_trampoline() at fork_trampoline

Tracing command intr pid 12 tid 100062 td 0xfffffe000d994490
sched_switch() at sched_switch+0x194/frame 0xffffff884af6fb40
mi_switch() at mi_switch+0x208/frame 0xffffff884af6fb90
ithread_loop() at ithread_loop+0x243/frame 0xffffff884af6fbe0
fork_exit() at fork_exit+0x135/frame 0xffffff884af6fc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af6fc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884af6fcf0, rbp =3D 0 ---
       =20
Tracing command intr pid 12 tid 100061 td 0xfffffe000d994920
fork_trampoline() at fork_trampoline

Tracing command intr pid 12 tid 100060 td 0xfffffe000a532490
sched_switch() at sched_switch+0x194/frame 0xffffff884af65b40
mi_switch() at mi_switch+0x208/frame 0xffffff884af65b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff884af65be0
fork_exit() at fork_exit+0x135/frame 0xffffff884af65c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af65c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884af65cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100051 td 0xfffffe000d702490
fork_trampoline() at fork_trampoline

Tracing command intr pid 12 tid 100046 td 0xfffffe000a52f000
sched_switch() at sched_switch+0x194/frame 0xffffff884ab98b40
mi_switch() at mi_switch+0x208/frame 0xffffff884ab98b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff884ab98be0
fork_exit() at fork_exit+0x135/frame 0xffffff884ab98c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ab98c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884ab98cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100041 td 0xfffffe000a531920
fork_trampoline() at fork_trampoline

Tracing command intr pid 12 tid 100040 td 0xfffffe000a532000
sched_switch() at sched_switch+0x194/frame 0xffffff800035bb40
mi_switch() at mi_switch+0x208/frame 0xffffff800035bb90
ithread_loop() at ithread_loop+0x243/frame 0xffffff800035bbe0
fork_exit() at fork_exit+0x135/frame 0xffffff800035bc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800035bc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff800035bcf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100032 td 0xfffffe000a4a3490
sched_switch() at sched_switch+0x194/frame 0xffffff8000323b40
mi_switch() at mi_switch+0x208/frame 0xffffff8000323b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff8000323be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000323c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000323c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000323cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100029 td 0xfffffe000a2d7920
sched_switch() at sched_switch+0x194/frame 0xffffff8000314b40
mi_switch() at mi_switch+0x208/frame 0xffffff8000314b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff8000314be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000314c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000314c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000314cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100028 td 0xfffffe000a2dc000
sched_switch() at sched_switch+0x194/frame 0xffffff800030fb40
mi_switch() at mi_switch+0x208/frame 0xffffff800030fb90
ithread_loop() at ithread_loop+0x243/frame 0xffffff800030fbe0
fork_exit() at fork_exit+0x135/frame 0xffffff800030fc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800030fc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff800030fcf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100026 td 0xfffffe000a2dc920
sched_switch() at sched_switch+0x194/frame 0xffffff8000305b40
mi_switch() at mi_switch+0x208/frame 0xffffff8000305b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff8000305be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000305c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000305c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000305cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100020 td 0xfffffe000a2d3920
sched_switch() at sched_switch+0x194/frame 0xffffff80002e6b40
mi_switch() at mi_switch+0x208/frame 0xffffff80002e6b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff80002e6be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002e6c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002e6c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002e6cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100019 td 0xfffffe000a2d4000
sched_switch() at sched_switch+0x194/frame 0xffffff80002e1b40
mi_switch() at mi_switch+0x208/frame 0xffffff80002e1b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff80002e1be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002e1c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002e1c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002e1cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100018 td 0xfffffe000a2d4490
sched_switch() at sched_switch+0x194/frame 0xffffff80002dcb40
mi_switch() at mi_switch+0x208/frame 0xffffff80002dcb90
ithread_loop() at ithread_loop+0x243/frame 0xffffff80002dcbe0
fork_exit() at fork_exit+0x135/frame 0xffffff80002dcc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002dcc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002dccf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100017 td 0xfffffe000a2d4920
sched_switch() at sched_switch+0x194/frame 0xffffff80002d7b40
mi_switch() at mi_switch+0x208/frame 0xffffff80002d7b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff80002d7be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002d7c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002d7c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002d7cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100016 td 0xfffffe000a2d7000
sched_switch() at sched_switch+0x194/frame 0xffffff80002d2b40
mi_switch() at mi_switch+0x208/frame 0xffffff80002d2b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff80002d2be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002d2c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002d2c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002d2cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100015 td 0xfffffe000a2c3000
sched_switch() at sched_switch+0x194/frame 0xffffff80002cdb40
mi_switch() at mi_switch+0x208/frame 0xffffff80002cdb90
ithread_loop() at ithread_loop+0x243/frame 0xffffff80002cdbe0
fork_exit() at fork_exit+0x135/frame 0xffffff80002cdc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002cdc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002cdcf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100014 td 0xfffffe000a2c3490
sched_switch() at sched_switch+0x194/frame 0xffffff80002c8b40
mi_switch() at mi_switch+0x208/frame 0xffffff80002c8b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff80002c8be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002c8c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002c8c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002c8cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100013 td 0xfffffe000a2c3920
sched_switch() at sched_switch+0x194/frame 0xffffff80002c3b40
mi_switch() at mi_switch+0x208/frame 0xffffff80002c3b90
ithread_loop() at ithread_loop+0x243/frame 0xffffff80002c3be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002c3c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002c3c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002c3cf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100012 td 0xfffffe000a2c4000
cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff816105e0
ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffffff81610600
trap() at trap+0x355/frame 0xffffffff81610800
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81610800
--- trap 0x13, rip =3D 0xffffffff808f019c, rsp =3D 0xffffffff816108c0, rbp =
=3D 0xffffff80002bea70 ---
_mtx_lock_sleep() at _mtx_lock_sleep+0x16c/frame 0xffffff80002bea70
_mtx_lock_flags() at _mtx_lock_flags+0x106/frame 0xffffff80002beaa0
softclock() at softclock+0x28f/frame 0xffffff80002beb60
intr_event_execute_handlers() at intr_event_execute_handlers+0x6a/frame 0xf=
fffff80002beb90
ithread_loop() at ithread_loop+0xac/frame 0xffffff80002bebe0
fork_exit() at fork_exit+0x135/frame 0xffffff80002bec30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002bec30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002becf0, rbp =3D 0 ---

Tracing command intr pid 12 tid 100011 td 0xfffffe000a2c4490
fork_trampoline() at fork_trampoline

Tracing command idle pid 11 tid 100010 td 0xfffffe000a2c4920
sched_switch() at sched_switch+0x194/frame 0xffffff80002b4ae0
mi_switch() at mi_switch+0x208/frame 0xffffff80002b4b30
critical_exit() at critical_exit+0xb0/frame 0xffffff80002b4b50
sched_idletd() at sched_idletd+0x11e/frame 0xffffff80002b4be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002b4c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002b4c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002b4cf0, rbp =3D 0 ---

Tracing command idle pid 11 tid 100009 td 0xfffffe000a2b3490
sched_switch() at sched_switch+0x194/frame 0xffffff80002afb00
mi_switch() at mi_switch+0x208/frame 0xffffff80002afb50
sched_idletd() at sched_idletd+0x17a/frame 0xffffff80002afbe0
fork_exit() at fork_exit+0x135/frame 0xffffff80002afc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002afc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002afcf0, rbp =3D 0 ---

Tracing command idle pid 11 tid 100008 td 0xfffffe000a2b3920
cpustop_handler() at cpustop_handler+0x28/frame 0xffffff8000272d00
ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff8000272d20
trap() at trap+0x355/frame 0xffffff8000272f20
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff8000272f20
--- trap 0x13, rip =3D 0xffffffff80ccff06, rsp =3D 0xffffff8000272fe0, rbp =
=3D 0xffffff80002aaad0 ---
acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xffffff80002aaad0
acpi_cpu_idle() at acpi_cpu_idle+0x2ac/frame 0xffffff80002aab10
cpu_idle_acpi() at cpu_idle_acpi+0x45/frame 0xffffff80002aab30
cpu_idle() at cpu_idle+0x78/frame 0xffffff80002aab50
sched_idletd() at sched_idletd+0x11e/frame 0xffffff80002aabe0
fork_exit() at fork_exit+0x135/frame 0xffffff80002aac30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002aac30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002aacf0, rbp =3D 0 ---

Tracing command idle pid 11 tid 100007 td 0xfffffe000a2c0000
cpustop_handler() at cpustop_handler+0x28/frame 0xffffff800026bd00
ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff800026bd20
trap() at trap+0x355/frame 0xffffff800026bf20
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff800026bf20
--- trap 0x13, rip =3D 0xffffffff80ccff06, rsp =3D 0xffffff800026bfe0, rbp =
=3D 0xffffff80002a5ad0 ---
acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xffffff80002a5ad0
acpi_cpu_idle() at acpi_cpu_idle+0x2ac/frame 0xffffff80002a5b10
cpu_idle_acpi() at cpu_idle_acpi+0x45/frame 0xffffff80002a5b30
cpu_idle() at cpu_idle+0x78/frame 0xffffff80002a5b50
sched_idletd() at sched_idletd+0x11e/frame 0xffffff80002a5be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002a5c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002a5c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002a5cf0, rbp =3D 0 ---

Tracing command idle pid 11 tid 100006 td 0xfffffe000a2c0490
sched_switch() at sched_switch+0x194/frame 0xffffff80002a0ae0
mi_switch() at mi_switch+0x208/frame 0xffffff80002a0b30
critical_exit() at critical_exit+0xb0/frame 0xffffff80002a0b50
sched_idletd() at sched_idletd+0x11e/frame 0xffffff80002a0be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002a0c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002a0c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002a0cf0, rbp =3D 0 ---

Tracing command idle pid 11 tid 100005 td 0xfffffe000a2c0920
cpustop_handler() at cpustop_handler+0x28/frame 0xffffff800025dd00
ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff800025dd20
trap() at trap+0x355/frame 0xffffff800025df20
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff800025df20
--- trap 0x13, rip =3D 0xffffffff80ccff06, rsp =3D 0xffffff800025dfe0, rbp =
=3D 0xffffff800029bad0 ---
acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xffffff800029bad0
acpi_cpu_idle() at acpi_cpu_idle+0x2ac/frame 0xffffff800029bb10
cpu_idle_acpi() at cpu_idle_acpi+0x45/frame 0xffffff800029bb30
cpu_idle() at cpu_idle+0x78/frame 0xffffff800029bb50
sched_idletd() at sched_idletd+0x11e/frame 0xffffff800029bbe0
fork_exit() at fork_exit+0x135/frame 0xffffff800029bc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800029bc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff800029bcf0, rbp =3D 0 ---

Tracing command idle pid 11 tid 100004 td 0xfffffe000a2b2000
cpustop_handler() at cpustop_handler+0x28/frame 0xffffff8000256d00
ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff8000256d20
trap() at trap+0x355/frame 0xffffff8000256f20
nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff8000256f20
--- trap 0x13, rip =3D 0xffffffff80ccff06, rsp =3D 0xffffff8000256fe0, rbp =
=3D 0xffffff8000296ad0 ---
acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xffffff8000296ad0
acpi_cpu_idle() at acpi_cpu_idle+0x2ac/frame 0xffffff8000296b10
cpu_idle_acpi() at cpu_idle_acpi+0x45/frame 0xffffff8000296b30
cpu_idle() at cpu_idle+0x78/frame 0xffffff8000296b50
sched_idletd() at sched_idletd+0x11e/frame 0xffffff8000296be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000296c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000296c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000296cf0, rbp =3D 0 ---

Tracing command idle pid 11 tid 100003 td 0xfffffe000a2b2490
sched_switch() at sched_switch+0x194/frame 0xffffff80002919c0
mi_switch() at mi_switch+0x208/frame 0xffffff8000291a10
critical_exit() at critical_exit+0xb0/frame 0xffffff8000291a30
lapic_handle_timer() at lapic_handle_timer+0x79/frame 0xffffff8000291a70
Xtimerint() at Xtimerint+0x88/frame 0xffffff8000291a70
--- interrupt, rip =3D 0xffffffff80cd9cc9, rsp =3D 0xffffff8000291b30, rbp =
=3D 0xffffff8000291b50 ---
cpu_idle_mwait() at cpu_idle_mwait+0x49/frame 0xffffff8000291b50
sched_idletd() at sched_idletd+0x11e/frame 0xffffff8000291be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000291c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000291c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000291cf0, rbp =3D 0 ---

Tracing command init pid 1 tid 100002 td 0xfffffe000a2b2920
sched_switch() at sched_switch+0x194/frame 0xffffff800028c6d0
mi_switch() at mi_switch+0x208/frame 0xffffff800028c720
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800028c750
sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff800028c7=
b0
sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff800028c7d0
_sleep() at _sleep+0x368/frame 0xffffff800028c860
kern_wait6() at kern_wait6+0x5e2/frame 0xffffff800028c8f0
kern_wait() at kern_wait+0x9c/frame 0xffffff800028ca50
sys_wait4() at sys_wait4+0x35/frame 0xffffff800028cb10
amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff800028cc30
Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff800028cc30
--- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x412c8c, rsp =3D 0x7fff=
ffffd7a8, rbp =3D 0x7d ---
       =20
Tracing command audit pid 10 tid 100001 td 0xfffffe000a2b3000
sched_switch() at sched_switch+0x194/frame 0xffffff8000287a50
mi_switch() at mi_switch+0x208/frame 0xffffff8000287aa0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000287ad0
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000287b00
_cv_wait() at _cv_wait+0x17d/frame 0xffffff8000287b60
audit_worker() at audit_worker+0x77/frame 0xffffff8000287be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000287c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000287c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000287cf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100065 td 0xfffffe000d993490
sched_switch() at sched_switch+0x194/frame 0xffffff884af8dab0
mi_switch() at mi_switch+0x208/frame 0xffffff884af8db00
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af8db30
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884af8db60
msleep_spin() at msleep_spin+0x21c/frame 0xffffff884af8dbc0
taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff884af8d=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff884af8dc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af8dc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff884af8dcf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100039 td 0xfffffe000a4a1000
sched_switch() at sched_switch+0x194/frame 0xffffff8000356ab0
mi_switch() at mi_switch+0x208/frame 0xffffff8000356b00
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000356b30
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000356b60
msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000356bc0
taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000356=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000356c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000356c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000356cf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100038 td 0xfffffe000a4a1490
sched_switch() at sched_switch+0x194/frame 0xffffff8000351ab0
mi_switch() at mi_switch+0x208/frame 0xffffff8000351b00
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000351b30
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000351b60
msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000351bc0
taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000351=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000351c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000351c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000351cf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100037 td 0xfffffe000a4a1920
sched_switch() at sched_switch+0x194/frame 0xffffff8000344ab0
mi_switch() at mi_switch+0x208/frame 0xffffff8000344b00
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000344b30
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000344b60
msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000344bc0
taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000344=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000344c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000344c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000344cf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100036 td 0xfffffe000a4a2000
sched_switch() at sched_switch+0x194/frame 0xffffff800033fab0
mi_switch() at mi_switch+0x208/frame 0xffffff800033fb00
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800033fb30
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff800033fb60
msleep_spin() at msleep_spin+0x21c/frame 0xffffff800033fbc0
taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff800033f=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff800033fc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800033fc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff800033fcf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100035 td 0xfffffe000a4a2490
sched_switch() at sched_switch+0x194/frame 0xffffff8000332ab0
mi_switch() at mi_switch+0x208/frame 0xffffff8000332b00
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000332b30
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000332b60
msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000332bc0
taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000332=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000332c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000332c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000332cf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100034 td 0xfffffe000a4a2920
sched_switch() at sched_switch+0x194/frame 0xffffff800032dab0
mi_switch() at mi_switch+0x208/frame 0xffffff800032db00
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800032db30
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff800032db60
msleep_spin() at msleep_spin+0x21c/frame 0xffffff800032dbc0
taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff800032d=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff800032dc30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800032dc30
--- trap 0, rip =3D 0, rsp =3D 0xffffff800032dcf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100033 td 0xfffffe000a4a3000
sched_switch() at sched_switch+0x194/frame 0xffffff8000328ab0
mi_switch() at mi_switch+0x208/frame 0xffffff8000328b00
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000328b30
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000328b60
msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000328bc0
taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000328=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000328c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000328c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000328cf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100031 td 0xfffffe000a4a3920
sched_switch() at sched_switch+0x194/frame 0xffffff800031ea80
mi_switch() at mi_switch+0x208/frame 0xffffff800031ead0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800031eb00
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff800031eb30
_sleep() at _sleep+0x3d4/frame 0xffffff800031ebc0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb7/frame 0xffffff800031e=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff800031ec30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800031ec30
--- trap 0, rip =3D 0, rsp =3D 0xffffff800031ecf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100030 td 0xfffffe000a2d7490
sched_switch() at sched_switch+0x194/frame 0xffffff8000319a80
mi_switch() at mi_switch+0x208/frame 0xffffff8000319ad0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000319b00
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000319b30
_sleep() at _sleep+0x3d4/frame 0xffffff8000319bc0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb7/frame 0xffffff8000319=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff8000319c30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000319c30
--- trap 0, rip =3D 0, rsp =3D 0xffffff8000319cf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100027 td 0xfffffe000a2dc490
sched_switch() at sched_switch+0x194/frame 0xffffff800030aa80
mi_switch() at mi_switch+0x208/frame 0xffffff800030aad0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800030ab00
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff800030ab30
_sleep() at _sleep+0x3d4/frame 0xffffff800030abc0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb7/frame 0xffffff800030a=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff800030ac30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800030ac30
--- trap 0, rip =3D 0, rsp =3D 0xffffff800030acf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100024 td 0xfffffe000a2dd490
sched_switch() at sched_switch+0x194/frame 0xffffff80002faa80
mi_switch() at mi_switch+0x208/frame 0xffffff80002faad0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff80002fab00
sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff80002fab30
_sleep() at _sleep+0x3d4/frame 0xffffff80002fabc0
taskqueue_thread_loop() at taskqueue_thread_loop+0xb7/frame 0xffffff80002fa=
be0
fork_exit() at fork_exit+0x135/frame 0xffffff80002fac30
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002fac30
--- trap 0, rip =3D 0, rsp =3D 0xffffff80002facf0, rbp =3D 0 ---

Tracing command kernel pid 0 tid 100000 td 0xffffffff813f4ad0
sched_switch() at sched_switch+0x194/frame 0xffffffff81a26b50
mi_switch() at mi_switch+0x208/frame 0xffffffff81a26ba0
sleepq_switch() at sleepq_switch+0xfc/frame 0xffffffff81a26bd0
sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffffff81a26c00
_sleep() at _sleep+0x26d/frame 0xffffffff81a26c90
scheduler() at scheduler+0x2a1/frame 0xffffffff81a26cd0
mi_startup() at mi_startup+0x77/frame 0xffffffff81a26cf0
btext() at btext+0x2c


--Multipart=_Thu__22_Aug_2013_10_21_21_+0200_nF//5/q+6KpaeERX--

--Signature=_Thu__22_Aug_2013_10_21_21_+0200_AqUicpZAxs_SMgFu
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (FreeBSD)

iEYEARECAAYFAlIVygoACgkQWTjlg++8y8uuiQCgy5zRNmWDac8q0CsZNZyhis7U
QsoAn2CicaTnt81kdlRhuOCiyLFV+ECX
=C514
-----END PGP SIGNATURE-----

--Signature=_Thu__22_Aug_2013_10_21_21_+0200_AqUicpZAxs_SMgFu--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130822102121.a799b116b0f6b0f7c90ba578>