Date: Thu, 22 Aug 2013 10:21:21 +0200 From: Yamagi Burmeister <lists@yamagi.org> To: freebsd-fs@freebsd.org Subject: Re: 9.2-RC1: LORs / Deadlock with SU+J on HAST in "memsync" mode Message-ID: <20130822102121.a799b116b0f6b0f7c90ba578@yamagi.org> In-Reply-To: <20130819115101.ae9c0cf788f881dc4de464c5@yamagi.org> References: <20130819115101.ae9c0cf788f881dc4de464c5@yamagi.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Signature=_Thu__22_Aug_2013_10_21_21_+0200_AqUicpZAxs_SMgFu Content-Type: multipart/mixed; boundary="Multipart=_Thu__22_Aug_2013_10_21_21_+0200_nF//5/q+6KpaeERX" --Multipart=_Thu__22_Aug_2013_10_21_21_+0200_nF//5/q+6KpaeERX Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, attached is a file with the transcript of a dd session containing: - ps - trace - show pcpu - show allpcpu - show locks - show alllocks - show lockedvnods - alltrace If the attachement is eaten by mailman the file can be found here: http://deponie.yamagi.org/freebsd/debug/lor_hast/ddb2.txt This is my first deadlock debug session. So if anything is still missing please tell me. Ciao, Yamagi On Wed, Aug 21, 2013 at 08:03:35PM +0200, Yamagi Burmeister wrote: > Could the problem be related to this deadlock / LOR? - > http://lists.freebsd.org/pipermail/freebsd-fs/2013-August/018052.html This is not related. >=20 > My test setup is still in place. Will test with r250907 reverted > tomorrow morning and report back. Additional informations could be > provided if necessary. I just need to know what exactly. Just follow the http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kernel= debug-deadlocks.html and collect all the information listed there after the apache/sendfile/nfs deadlock is reproduced. ---- On Mon, 19 Aug 2013 11:51:01 +0200 Yamagi Burmeister <lists@yamagi.org> wrote: > Hello, > in the last week I've seen several deadlocked processes (stucked in > state "vofflock" or "wswbuf0" on an UFS2 filesystem with SU+J on HAST > with "memsync" mode. TheThe deadlock disappear if either SU+J is > disabled (only SU are active) or HAST is switched to "fullsync" mode. >=20 > The system is: > FreeBSD tvtransfer.local 9.2-RC1 FreeBSD 9.2-RC1 #0 r254355M: > Fri Aug 16 12:35:30 UTC 2013 > support@tvtransfer.local:/usr/obj/usr/src/sys/GENERIC amd64 >=20 > I've build a kernel with full debugging support and seen this two LOR > beside the known false positive between bufwait and dirhash: >=20 > lock order reversal: > 1st 0xfffffe0197310d68 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2223 > 2nd 0xffffff87be8be0f8 bufwait (bufwait) > @ /usr/src/sys/ufs/ffs/ffs_vnops.c:261 3rd 0xfffffe019746f098 ufs (ufs) > @ /usr/src/sys/kern/vfs_subr.c:2223 KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame > 0xffffff88715dcd50 kdb_backtrace() at kdb_backtrace+0x37/frame > 0xffffff88715dce10 _witness_debugger() at _witness_debugger+0x2c/frame > 0xffffff88715dce30 witness_checkorder() at witness_checkorder > +0x875/frame 0xffffff88715dcef0 __lockmgr_args() at __lockmgr_args > +0x1141/frame 0xffffff88715dcfd0 ffs_lock() at ffs_lock+0x9c/frame > 0xffffff88715dd020 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe3/frame > 0xffffff88715dd050 _vn_lock() at _vn_lock+0x55/frame 0xffffff88715dd0b0 > vget() at vget+0x7b/frame 0xffffff88715dd100 > vfs_hash_get() at vfs_hash_get+0xd5/frame 0xffffff88715dd150 > ffs_vgetf() at ffs_vgetf+0x48/frame 0xffffff88715dd1d0 > softdep_sync_buf() at softdep_sync_buf+0x547/frame 0xffffff88715dd2a0 > ffs_syncvnode() at ffs_syncvnode+0x2c1/frame 0xffffff88715dd310 > ffs_truncate() at ffs_truncate+0x10bb/frame 0xffffff88715dd560 > ufs_direnter() at ufs_direnter+0x550/frame 0xffffff88715dd630 > ufs_makeinode() at ufs_makeinode+0x355/frame 0xffffff88715dd7f0 > VOP_CREATE_APV() at VOP_CREATE_APV+0x102/frame 0xffffff88715dd820 > vn_open_cred() at vn_open_cred+0x45e/frame 0xffffff88715dd990 > kern_openat() at kern_openat+0x20a/frame 0xffffff88715ddb10 > amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88715ddc30 > Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88715ddc30 > --- syscall (5, FreeBSD ELF64, sys_open), rip =3D 0x800b38d3c, rsp =3D > 0x7fffffffd128, rbp =3D 0x7fffffffda40 ---=20 >=20 > lock order reversal: > 1st 0xfffffe0035dfbf30 so_snd_sx (so_snd_sx) > @ /usr/src/sys/kern/uipc_sockbuf.c:145 2nd 0xfffffe02ead2b328 ufs (ufs) > @ /usr/src/sys/kern/uipc_syscalls.c:2062 KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a/frame > 0xffffff8871515530 kdb_backtrace() at kdb_backtrace+0x37/frame > 0xffffff88715155f0 _witness_debugger() at _witness_debugger+0x2c/frame > 0xffffff8871515610 witness_checkorder() at witness_checkorder > +0x875/frame 0xffffff88715156d0 __lockmgr_args() at __lockmgr_args > +0x1446/frame 0xffffff88715157b0 ffs_lock() at ffs_lock+0x9c/frame > 0xffffff8871515800 VOP_LOCK1_APV() at VOP_LOCK1_APV+0xe3/frame > 0xffffff8871515830 _vn_lock() at _vn_lock+0x55/frame 0xffffff8871515890 > kern_sendfile() at kern_sendfile+0x8e7/frame 0xffffff8871515ab0 > do_sendfile() at do_sendfile+0xdc/frame 0xffffff8871515b10 > amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871515c30 > Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871515c30 > --- syscall (393, FreeBSD ELF64, sys_sendfile), rip =3D 0x8022075bc, rsp > =3D 0x7fffffffb268, rbp =3D 0x7fffffffd570 --- >=20 > Both LORs can be found in a more readable format here: > http://deponie.yamagi.org/freebsd/debug/lor_hast/lor.txt >=20 > The transcript of a DDB session with all informations requested in > chapter "10.7. Debugging Deadlocks" of the developers handbook can > be found here: > http://deponie.yamagi.org/freebsd/debug/lor_hast/ddb.txt >=20 > Reproducing this problem is easy: > 1. Create a HAST setup in "memsync" mode with both primary and > secondary connected > 2. "newfs -U -j /dev/hast/$device" on the primary > 3. mount "mount /dev/hast/$device /mnt" > 4. Create some load on /mnt. For example copy /usr/ports in > an endless loop. Sooner or later processes accessing /mnt > will deadlock. This may be easer to reproduce while HAST > is performing the initial sync. >=20 > More information can be provided if necessary. >=20 > Thanks, > Yamagi >=20 > --=20 > Homepage: www.yamagi.org > XMPP: yamagi@yamagi.org > GnuPG/GPG: 0xEFBCCBCB --=20 Homepage: www.yamagi.org XMPP: yamagi@yamagi.org GnuPG/GPG: 0xEFBCCBCB --Multipart=_Thu__22_Aug_2013_10_21_21_+0200_nF//5/q+6KpaeERX Content-Type: text/plain; name="ddb2.txt" Content-Disposition: attachment; filename="ddb2.txt" Content-Transfer-Encoding: quoted-printable db> ps pid ppid pgrp uid state wmesg wchan cmd 42798 1267 42798 0 R+ CPU 6 sysctl 42785 42780 42780 2 D flswai 0xffffffff815e7a80 mv 42780 42778 42780 2 Ss wait 0xfffffe010e60f000 sh 42778 1219 1219 0 S piperd 0xfffffe000df65b60 cron 42773 1357 1357 0 D+ getblk 0xffffff87bfe6a5f8 mktemp 42646 1359 1359 0 D+ getblk 0xffffff87bfe4bcf8 mktemp 9787 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 9784 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 9783 1346 1346 80 S lockf 0xfffffe02f1879700 httpd 9779 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 9773 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 9769 1346 1346 80 S lockf 0xfffffe02f1efd580 httpd 9768 1346 1346 80 S lockf 0xfffffe02ad5d7400 httpd 9766 1346 1346 80 S lockf 0xfffffe02f18b2680 httpd 9764 1346 1346 80 S lockf 0xfffffe02ad421600 httpd 9755 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 9750 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 9749 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 9748 1346 1346 80 S kqread 0xfffffe0278c7e700 httpd 8852 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 8851 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 8849 1346 1346 80 D wswbuf0 0xffffffff81388280 httpd 8847 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 8842 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 8840 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 8839 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 8837 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 7978 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 7974 1346 1346 80 S lockf 0xfffffe02ad84f100 httpd 7971 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 7969 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 7166 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 7164 1346 1346 80 S lockf 0xfffffe02ad136d00 httpd 6396 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 3076 1320 3076 0 D+ getblk 0xffffff87bf2f0ff8 cpdup 2087 1315 2087 0 D+ getblk 0xffffff87bf2f0ff8 cpdup 1359 1311 1359 0 S+ piperd 0xfffffe010e668b60 sh 1357 1296 1357 0 S+ piperd 0xfffffe000de7db60 sh 1356 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 1352 1324 1352 0 S+ select 0xfffffe000dc99640 top 1351 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 1350 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 1349 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 1348 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 1347 1346 1346 80 D vofflock 0xfffffe000de479d2 httpd 1346 1 1346 0 Ss select 0xfffffe000df7d540 httpd 1324 1322 1324 0 S+ pause 0xfffffe0028a940a0 tcsh 1322 1292 1322 0 Ss+ pause 0xfffffe000df340a0 csh 1320 1318 1320 0 S+ pause 0xfffffe00289b99f0 tcsh 1318 1292 1318 0 Ss+ pause 0xfffffe0028799548 csh 1315 1313 1315 0 S+ pause 0xfffffe00287990a0 tcsh 1313 1292 1313 0 Ss+ pause 0xfffffe000dee6548 csh 1311 1309 1311 0 S+ pause 0xfffffe000dee69f0 tcsh 1309 1292 1309 0 Ss+ pause 0xfffffe000dec19f0 csh 1296 1293 1296 0 S+ pause 0xfffffe00287a40a0 tcsh 1293 1292 1293 0 Ss+ pause 0xfffffe002879b9f0 csh 1292 1 1292 0 Rs CPU 7 tmux 1290 1276 1290 0 S+ select 0xfffffe0028eff840 tmux 1283 1124 1124 845 DLJ (threaded) hastd 100196 S uwait 0xfffffe000daf1580 hastd 100195 S uwait 0xfffffe000db66180 hastd 100194 S uwait 0xfffffe000db66080 hastd 100193 D physrd 0xffffff87be46aa60 hastd 100192 S ggwait 0xfffffe000de08000 hastd 100191 S sbwait 0xfffffe0028645694 hastd 100190 S sigwait 0xfffffe000dd37000 hastd 100130 S uwait 0xfffffe000df7b080 hastd 1282 1124 1124 845 DLJ (threaded) hastd 100203 S uwait 0xfffffe000dc85900 hastd 100202 S uwait 0xfffffe000dd59480 hastd 100201 S uwait 0xfffffe000dc85a80 hastd 100200 S uwait 0xfffffe000dc85b80 hastd 100199 S uwait 0xfffffe000dc85c80 hastd 100198 S sbwait 0xfffffe000df54694 hastd 100197 S sigwait 0xfffffe000da60000 hastd 100117 D physwr 0xffffff87be479160 hastd 1276 1275 1276 0 S+ pause 0xfffffe000dec10a0 csh 1275 1272 1275 1001 S+ wait 0xfffffe000df214a8 su 1272 1271 1272 1001 Ss+ pause 0xfffffe002879b0a0 csh 1271 1269 1269 1001 S select 0xfffffe000dc42140 sshd 1269 1204 1269 0 Ss select 0xfffffe000db5eac0 sshd 1267 1261 1267 0 S+ pause 0xfffffe0028a950a0 csh 1261 1 1261 0 Ss+ wait 0xfffffe0028a954a8 login 1260 1 1260 0 Ss+ ttyin 0xfffffe000d9e0ca8 getty 1259 1 1259 0 Ss+ ttyin 0xfffffe000d9e20a8 getty 1258 1 1258 0 Ss+ ttyin 0xfffffe000d9e24a8 getty 1257 1 1257 0 Ss+ ttyin 0xfffffe000d9988a8 getty 1256 1 1256 0 Ss+ ttyin 0xfffffe000d9984a8 getty 1255 1 1255 0 Ss+ ttyin 0xfffffe000d997ca8 getty 1254 1 1254 0 Ss+ ttyin 0xfffffe000d9104a8 getty 1253 1 1253 0 Ss+ ttyin 0xfffffe000d9990a8 getty 1219 1 1219 0 Ss nanslp 0xffffffff813fc468 cron 1204 1 1204 0 Ss select 0xfffffe000df7d0c0 sshd 1198 1 1198 26 Ss select 0xfffffe000dc42540 exim-4.80.1-2 1185 1 1185 181 Ss select 0xfffffe000df7c8c0 nrpe2 1124 1 1124 0 Ss select 0xfffffe000dc9c440 hastd 1093 1 1093 0 Ss select 0xfffffe000df7c5c0 syslogd 930 1 930 0 Ss select 0xfffffe000df7c0c0 devd 135 1 135 0 Ss pause 0xfffffe000de660a0 adjkerntz 18 0 0 0 DL m:w1 0xfffffe000dd07800 [g_mirror gmro= otfs] 17 0 0 0 DL sdflush 0xffffffff815f3778 [softdepflush] 16 0 0 0 DL wswbuf0 0xffffffff81388280 [syncer] 9 0 0 0 DL vlruwt 0xfffffe000da52000 [vnlru] 8 0 0 0 DL wswbuf0 0xffffffff81388280 [bufdaemon] 7 0 0 0 DL pgzero 0xffffffff815fd81c [pagezero] 6 0 0 0 DL psleep 0xffffffff815fc9e8 [vmdaemon] 5 0 0 0 DL psleep 0xffffffff815fc9ac [pagedaemon] 4 0 0 0 DL ccb_scan 0xffffffff813bd5e0 [xpt_thrd] 3 0 0 0 DL waiting_ 0xffffffff815edac0 [sctp_iterator] 2 0 0 0 DL - 0xfffffe000d99b248 [fdc0] 15 0 0 0 DL (threaded) [usb] 100059 D - 0xffffff8000e47e18 [usbus3] 100058 D - 0xffffff8000e47dc0 [usbus3] 100057 D - 0xffffff8000e47d68 [usbus3] 100056 D - 0xffffff8000e47d10 [usbus3] 100055 D - 0xffffff8000e3eef0 [usbus2] 100054 D - 0xffffff8000e3ee98 [usbus2] 100053 D - 0xffffff8000e3ee40 [usbus2] 100052 D - 0xffffff8000e3ede8 [usbus2] 100050 D - 0xffffff8000e35ef0 [usbus1] 100049 D - 0xffffff8000e35e98 [usbus1] 100048 D - 0xffffff8000e35e40 [usbus1] 100047 D - 0xffffff8000e35de8 [usbus1] 100045 D - 0xffffff8000e2cef0 [usbus0] 100044 D - 0xffffff8000e2ce98 [usbus0] 100043 D - 0xffffff8000e2ce40 [usbus0] 100042 D - 0xffffff8000e2cde8 [usbus0] 14 0 0 0 DL - 0xffffffff813fb104 [yarrow] 13 0 0 0 RL (threaded) [geom] 100023 D - 0xffffffff813f4350 [g_down] 100022 Run CPU 3 [g_up] 100021 D - 0xffffffff813f4338 [g_event] 12 0 0 0 RL (threaded) [intr] 100064 I [irq7: ppc0] 100062 I [swi0: uart ua= rt] 100061 I [irq12: psm0] 100060 I [irq1: atkbd0] 100051 I [irq18: uhci2] 100046 CanRun [irq19: uhci1 = ahci0] 100041 I [irq17: uhci0 = ehci0] 100040 I [irq16: twa0] 100032 I [swi2: cambio] 100029 I [swi6: task qu= eue] 100028 I [swi6: Giant t= askq] 100026 I [swi5: fast ta= skq] 100020 I [swi1: netisr = 0] 100019 I [swi4: clock] 100018 I [swi4: clock] 100017 I [swi4: clock] 100016 I [swi4: clock] 100015 I [swi4: clock] 100014 I [swi4: clock] 100013 I [swi4: clock] 100012 Run CPU 0 [swi4: clock] 100011 I [swi3: vm] 11 0 0 0 RL (threaded) [idle] 100010 CanRun [idle: cpu7] 100009 CanRun [idle: cpu6] 100008 Run CPU 5 [idle: cpu5] 100007 Run CPU 4 [idle: cpu4] 100006 CanRun [idle: cpu3] 100005 Run CPU 2 [idle: cpu2] 100004 Run CPU 1 [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xfffffe000a2b0950 [init] 10 0 0 0 DL audit_wo 0xffffffff815f26f0 [audit] 0 0 0 0 DLs (threaded) [kernel] 100065 D - 0xfffffe000d9fd380 [mca taskq] 100039 D - 0xfffffe000a514600 [em1 txq] 100038 D - 0xfffffe000a514680 [em1 que] 100037 D - 0xfffffe000a50d480 [em0 txq] 100036 D - 0xfffffe000a50d500 [em0 que] 100035 D - 0xfffffe000a49f500 [acpi_task_2] 100034 D - 0xfffffe000a49f500 [acpi_task_1] 100033 D - 0xfffffe000a49f500 [acpi_task_0] 100031 D - 0xfffffe000a4a0280 [ffs_trim task= q] 100030 D - 0xfffffe000a461400 [kqueue taskq] 100027 D - 0xfffffe000a461100 [thread taskq] 100024 D - 0xfffffe000a2c2300 [firmware task= q] 100000 D sched 0xffffffff813f4620 [swapper] ----------------------------------------------------------------------- # httpd stuck in wswbuf db> trace 8849 Tracing pid 8849 tid 100328 td 0xfffffe02788e8000 sched_switch() at sched_switch+0x194/frame 0xffffff887167d360 mi_switch() at mi_switch+0x208/frame 0xffffff887167d3b0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887167d3e0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887167d410 _sleep() at _sleep+0x3d4/frame 0xffffff887167d4a0 getpbuf() at getpbuf+0x4e/frame 0xffffff887167d4c0 cluster_wbuild_gb() at cluster_wbuild_gb+0x21e/frame 0xffffff887167d570 cluster_write_gb() at cluster_write_gb+0x651/frame 0xffffff887167d670 ffs_write() at ffs_write+0x65e/frame 0xffffff887167d710 VOP_WRITE_APV() at VOP_WRITE_APV+0x16e/frame 0xffffff887167d820 vn_write() at vn_write+0x311/frame 0xffffff887167d8b0 vn_io_fault() at vn_io_fault+0x9e/frame 0xffffff887167da30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887167da80 kern_writev() at kern_writev+0x6c/frame 0xffffff887167dac0 sys_write() at sys_write+0x64/frame 0xffffff887167db10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887167dc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887167dc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- # httpd stuck in vofflo db> trace 8851 Tracing pid 8851 tid 100240 td 0xfffffe010e5f3490 sched_switch() at sched_switch+0x194/frame 0xffffff88714c5720 mi_switch() at mi_switch+0x208/frame 0xffffff88714c5770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714c57a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88714c57d0 _sleep() at _sleep+0x3d4/frame 0xffffff88714c5860 foffset_lock() at foffset_lock+0x86/frame 0xffffff88714c5890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88714c58b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88714c5a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff88714c5a80 kern_writev() at kern_writev+0x6c/frame 0xffffff88714c5ac0 sys_write() at sys_write+0x64/frame 0xffffff88714c5b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714c5c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714c5c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- ----------------------------------------------------------------------- db> show pcpu cpuid =3D 6 dynamic pcpu =3D 0xffffff807eebe900 curthread =3D 0xfffffe002896a490: pid 42798 "sysctl" curpcb =3D 0xffffff8871484d00 fpcurthread =3D 0xfffffe002896a490: pid 42798 "sysctl" idlethread =3D 0xfffffe000a2b3490: tid 100009 "idle: cpu6" curpmap =3D 0xfffffe000df87750 tssp =3D 0xffffffff81601070 commontssp =3D 0xffffffff81601070 rsp0 =3D 0xffffff8871484d00 gs32p =3D 0xffffffff815ff1a8 ldt =3D 0xffffffff815ff1e8 tss =3D 0xffffffff815ff1d8 spin locks held: db> show allpcpu Current CPU: 6 cpuid =3D 0 dynamic pcpu =3D 0x66b900 curthread =3D 0xfffffe000a2c4000: pid 12 "swi4: clock" curpcb =3D 0xffffff80002bed00 fpcurthread =3D none idlethread =3D 0xfffffe000a2b2490: tid 100003 "idle: cpu0" curpmap =3D 0xffffffff813f5070 tssp =3D 0xffffffff81600e00 commontssp =3D 0xffffffff81600e00 rsp0 =3D 0xffffff80002bed00 gs32p =3D 0xffffffff815fef38 ldt =3D 0xffffffff815fef78 tss =3D 0xffffffff815fef68 spin locks held: cpuid =3D 1 dynamic pcpu =3D 0xffffff807ee9b900 curthread =3D 0xfffffe000a2b2000: pid 11 "idle: cpu1" curpcb =3D 0xffffff8000296d00 fpcurthread =3D none idlethread =3D 0xfffffe000a2b2000: tid 100004 "idle: cpu1" curpmap =3D 0xffffffff813f5070 tssp =3D 0xffffffff81600e68 commontssp =3D 0xffffffff81600e68 rsp0 =3D 0xffffff8000296d00 gs32p =3D 0xffffffff815fefa0 ldt =3D 0xffffffff815fefe0 tss =3D 0xffffffff815fefd0 spin locks held: cpuid =3D 2 dynamic pcpu =3D 0xffffff807eea2900 curthread =3D 0xfffffe000a2c0920: pid 11 "idle: cpu2" curpcb =3D 0xffffff800029bd00 fpcurthread =3D none idlethread =3D 0xfffffe000a2c0920: tid 100005 "idle: cpu2" curpmap =3D 0xffffffff813f5070 tssp =3D 0xffffffff81600ed0 commontssp =3D 0xffffffff81600ed0 rsp0 =3D 0xffffff800029bd00 gs32p =3D 0xffffffff815ff008 ldt =3D 0xffffffff815ff048 tss =3D 0xffffffff815ff038 spin locks held: cpuid =3D 3 dynamic pcpu =3D 0xffffff807eea9900 curthread =3D 0xfffffe000a2d3000: pid 13 "g_up" curpcb =3D 0xffffff80002f0d00 fpcurthread =3D none idlethread =3D 0xfffffe000a2c0490: tid 100006 "idle: cpu3" curpmap =3D 0xffffffff813f5070 tssp =3D 0xffffffff81600f38 commontssp =3D 0xffffffff81600f38 rsp0 =3D 0xffffff80002f0d00 gs32p =3D 0xffffffff815ff070 ldt =3D 0xffffffff815ff0b0 tss =3D 0xffffffff815ff0a0 spin locks held: exclusive spin mutex sched lock 3 (sched lock) r =3D 0 (0xffffffff81414cc0)= locked @ /usr/src/sys/kern/kern_intr.c:916 cpuid =3D 4 dynamic pcpu =3D 0xffffff807eeb0900 curthread =3D 0xfffffe000a2c0000: pid 11 "idle: cpu4" curpcb =3D 0xffffff80002a5d00 fpcurthread =3D none idlethread =3D 0xfffffe000a2c0000: tid 100007 "idle: cpu4" curpmap =3D 0xffffffff813f5070 tssp =3D 0xffffffff81600fa0 commontssp =3D 0xffffffff81600fa0 rsp0 =3D 0xffffff80002a5d00 gs32p =3D 0xffffffff815ff0d8 ldt =3D 0xffffffff815ff118 tss =3D 0xffffffff815ff108 spin locks held: cpuid =3D 5 dynamic pcpu =3D 0xffffff807eeb7900 curthread =3D 0xfffffe000a2b3920: pid 11 "idle: cpu5" curpcb =3D 0xffffff80002aad00 fpcurthread =3D none idlethread =3D 0xfffffe000a2b3920: tid 100008 "idle: cpu5" curpmap =3D 0xffffffff813f5070 tssp =3D 0xffffffff81601008 commontssp =3D 0xffffffff81601008 rsp0 =3D 0xffffff80002aad00 gs32p =3D 0xffffffff815ff140 ldt =3D 0xffffffff815ff180 tss =3D 0xffffffff815ff170 spin locks held: cpuid =3D 6 dynamic pcpu =3D 0xffffff807eebe900 curthread =3D 0xfffffe002896a490: pid 42798 "sysctl" curpcb =3D 0xffffff8871484d00 fpcurthread =3D 0xfffffe002896a490: pid 42798 "sysctl" idlethread =3D 0xfffffe000a2b3490: tid 100009 "idle: cpu6" curpmap =3D 0xfffffe000df87750 tssp =3D 0xffffffff81601070 commontssp =3D 0xffffffff81601070 rsp0 =3D 0xffffff8871484d00 gs32p =3D 0xffffffff815ff1a8 ldt =3D 0xffffffff815ff1e8 tss =3D 0xffffffff815ff1d8 spin locks held: cpuid =3D 7 dynamic pcpu =3D 0xffffff807eec5900 curthread =3D 0xfffffe000de70490: pid 1292 "tmux" curpcb =3D 0xffffff8871209d00 fpcurthread =3D 0xfffffe000de70490: pid 1292 "tmux" idlethread =3D 0xfffffe000a2c4920: tid 100010 "idle: cpu7" curpmap =3D 0xfffffe002856ca60 tssp =3D 0xffffffff816010d8 commontssp =3D 0xffffffff816010d8 rsp0 =3D 0xffffff8871209d00 gs32p =3D 0xffffffff815ff210 ldt =3D 0xffffffff815ff250 tss =3D 0xffffffff815ff240 spin locks held: ----------------------------------------------------------------------- db> show locks exclusive sleep mutex Giant (Giant) r =3D 0 (0xffffffff813f7780) locked @ /= usr/src/sys/kern/kern_sysctl.c:1492 ----------------------------------------------------------------------- db> show alllocks Process 42798 (sysctl) thread 0xfffffe002896a490 (100227) exclusive sleep mutex Giant (Giant) r =3D 0 (0xffffffff813f7780) locked @ /= usr/src/sys/kern/kern_sysctl.c:1492 Process 42773 (mktemp) thread 0xfffffe000ddbb920 (100131) exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02ad8c0ad8) locked @ /usr/src/= sys/ufs/ffs/ffs_vfsops.c:1721 exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02ad5f8098) locked @ /usr/src/= sys/kern/vfs_subr.c:2223 Process 42646 (mktemp) thread 0xfffffe010e5df920 (100260) exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02ad91e098) locked @ /usr/src/= sys/ufs/ffs/ffs_vfsops.c:1721 exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02ad5f6ad8) locked @ /usr/src/= sys/kern/vfs_subr.c:2223 Process 8849 (httpd) thread 0xfffffe02788e8000 (100328) exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87bfaf67f8) locked @ /= usr/src/sys/kern/vfs_cluster.c:816 exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe000dfabd68) locked @ /usr/src/= sys/kern/vfs_vnops.c:762 Process 3076 (cpdup) thread 0xfffffe000ddbb000 (100157) exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02480265b8) locked @ /usr/src/= sys/ufs/ffs/ffs_vfsops.c:1721 shared lockmgr ufs (ufs) r =3D 0 (0xfffffe010ebcb848) locked @ /usr/src/sys= /kern/vfs_subr.c:2223 Process 2087 (cpdup) thread 0xfffffe002896a000 (100228) exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe022cfec5b8) locked @ /usr/src/= sys/kern/vfs_vnops.c:319 Process 1283 (hastd) thread 0xfffffe000de97920 (100193) exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87be46aaf8) locked @ /= usr/src/sys/vm/vm_pager.c:311 Process 1283 (hastd) thread 0xfffffe000de76490 (100191) exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xfffffe0028645648) locked @ /u= sr/src/sys/kern/uipc_sockbuf.c:148 Process 1282 (hastd) thread 0xfffffe0028a15490 (100198) exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xfffffe000df54648) locked @ /u= sr/src/sys/kern/uipc_sockbuf.c:148 Process 1282 (hastd) thread 0xfffffe000daee920 (100117) exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87be4791f8) locked @ /= usr/src/sys/vm/vm_pager.c:311 Process 16 (syncer) thread 0xfffffe000daf2000 (100073) exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87bf1a4bf8) locked @ /= usr/src/sys/kern/vfs_cluster.c:816 exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02f180f098) locked @ /usr/src/= sys/kern/vfs_subr.c:1823 Process 8 (bufdaemon) thread 0xfffffe000d98f490 (100071) exclusive lockmgr bufwait (bufwait) r =3D 0 (0xffffff87bf1490f8) locked @ /= usr/src/sys/kern/vfs_cluster.c:816 exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe02f13f9098) locked @ /usr/src/= sys/kern/vfs_bio.c:2753 ----------------------------------------------------------------------- db> show lockedvnods Locked vnodes 0xfffffe000dfabcd0: 0xfffffe000dfabcd0: tag ufs, type VREG tag ufs, type VREG usecount 1, writecount 1, refcount 102 mountedhere 0 usecount 1, writecount 1, refcount 102 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) v_object 0xfffffe00289e5570 ref 0 pages 800 v_object 0xfffffe00289e5570 ref 0 pages 800 lock type ufs: EXCL by thread 0xfffffe02788e8000 (pid 8849) lock type ufs: EXCL by thread 0xfffffe02788e8000 (pid 8849) #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #1 0xffffffff80b520fc at ffs_lock+0x9c #1 0xffffffff80b520fc at ffs_lock+0x9c #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #3 0xffffffff809b62e5 at _vn_lock+0x55 #3 0xffffffff809b62e5 at _vn_lock+0x55 #4 0xffffffff809b74b6 at vn_write+0x136 #4 0xffffffff809b74b6 at vn_write+0x136 #5 0xffffffff809b4f7e at vn_io_fault+0x9e #5 0xffffffff809b4f7e at vn_io_fault+0x9e #6 0xffffffff80954bc5 at dofilewrite+0x85 #6 0xffffffff80954bc5 at dofilewrite+0x85 #7 0xffffffff8095651c at kern_writev+0x6c #7 0xffffffff8095651c at kern_writev+0x6c #8 0xffffffff80956634 at sys_write+0x64 #8 0xffffffff80956634 at sys_write+0x64 #9 0xffffffff80ceb239 at amd64_syscall+0x2f9 #9 0xffffffff80ceb239 at amd64_syscall+0x2f9 #10 0xffffffff80cd57e7 at Xfast_syscall+0xf7 #10 0xffffffff80cd57e7 at Xfast_syscall+0xf7 ino 1446711, on dev mirror/gmrootfs ino 1446711, on dev mirror/gmrootfs 0xfffffe010ebcb7b0: 0xfffffe010ebcb7b0: tag ufs, type VDIR tag ufs, type VDIR usecount 1, writecount 0, refcount 4 mountedhere 0 usecount 1, writecount 0, refcount 4 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) v_object 0xfffffe010eb691d0 ref 0 pages 4 v_object 0xfffffe010eb691d0 ref 0 pages 4 lock type ufs: SHARED (count 1) lock type ufs: SHARED (count 1) #0 0xffffffff808e827c at __lockmgr_args+0xd5c #0 0xffffffff808e827c at __lockmgr_args+0xd5c #1 0xffffffff80b520fc at ffs_lock+0x9c #1 0xffffffff80b520fc at ffs_lock+0x9c #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #3 0xffffffff809b62e5 at _vn_lock+0x55 #3 0xffffffff809b62e5 at _vn_lock+0x55 #4 0xffffffff809a87cb at vget+0x7b #4 0xffffffff809a87cb at vget+0x7b #5 0xffffffff80990df3 at cache_lookup_times+0x553 #5 0xffffffff80990df3 at cache_lookup_times+0x553 #6 0xffffffff80991298 at vfs_cache_lookup+0xc8 #6 0xffffffff80991298 at vfs_cache_lookup+0xc8 #7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102 #7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102 #8 0xffffffff80998bdf at lookup+0x32f #8 0xffffffff80998bdf at lookup+0x32f #9 0xffffffff80999c9f at namei+0x40f #9 0xffffffff80999c9f at namei+0x40f #10 0xffffffff809ac5b6 at kern_statat_vnhook+0xb6 #10 0xffffffff809ac5b6 at kern_statat_vnhook+0xb6 #11 0xffffffff809ac7d5 at kern_statat+0x15 #11 0xffffffff809ac7d5 at kern_statat+0x15 #12 0xffffffff809ac89a at sys_lstat+0x2a #12 0xffffffff809ac89a at sys_lstat+0x2a #13 0xffffffff80ceb239 at amd64_syscall+0x2f9 #13 0xffffffff80ceb239 at amd64_syscall+0x2f9 #14 0xffffffff80cd57e7 at Xfast_syscall+0xf7 #14 0xffffffff80cd57e7 at Xfast_syscall+0xf7 ino 1203840, on dev hast/links ino 1203840, on dev hast/links 0xfffffe02f13f9000: 0xfffffe02f13f9000: tag ufs, type VREG tag ufs, type VREG usecount 0, writecount 0, refcount 5 mountedhere 0 usecount 0, writecount 0, refcount 5 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) v_object 0xfffffe02f16cc2b8 ref 0 pages 29 v_object 0xfffffe02f16cc2b8 ref 0 pages 29 lock type ufs: EXCL by thread 0xfffffe000d98f490 (pid 8) lock type ufs: EXCL by thread 0xfffffe000d98f490 (pid 8) #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #1 0xffffffff80b520fc at ffs_lock+0x9c #1 0xffffffff80b520fc at ffs_lock+0x9c #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #3 0xffffffff809b62e5 at _vn_lock+0x55 #3 0xffffffff809b62e5 at _vn_lock+0x55 #4 0xffffffff8098b590 at flushbufqueues+0x4f0 #4 0xffffffff8098b590 at flushbufqueues+0x4f0 #5 0xffffffff8098b816 at buf_do_flush+0x16 #5 0xffffffff8098b816 at buf_do_flush+0x16 #6 0xffffffff8098b9c4 at buf_daemon+0xe4 #6 0xffffffff8098b9c4 at buf_daemon+0xe4 #7 0xffffffff808d2e55 at fork_exit+0x135 #7 0xffffffff808d2e55 at fork_exit+0x135 #8 0xffffffff80cd5a2e at fork_trampoline+0xe #8 0xffffffff80cd5a2e at fork_trampoline+0xe ino 228168177, on dev hast/links ino 228168177, on dev hast/links 0xfffffe02f180f000: 0xfffffe02f180f000: tag ufs, type VREG tag ufs, type VREG usecount 0, writecount 0, refcount 6 mountedhere 0 usecount 0, writecount 0, refcount 6 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) v_object 0xfffffe02f16902b8 ref 0 pages 30 v_object 0xfffffe02f16902b8 ref 0 pages 30 lock type ufs: EXCL by thread 0xfffffe000daf2000 (pid 16) lock type ufs: EXCL by thread 0xfffffe000daf2000 (pid 16) #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #1 0xffffffff80b520fc at ffs_lock+0x9c #1 0xffffffff80b520fc at ffs_lock+0x9c #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #3 0xffffffff809b62e5 at _vn_lock+0x55 #3 0xffffffff809b62e5 at _vn_lock+0x55 #4 0xffffffff809aac70 at sync_vnode+0x130 #4 0xffffffff809aac70 at sync_vnode+0x130 #5 0xffffffff809aaef7 at sched_sync+0x1c7 #5 0xffffffff809aaef7 at sched_sync+0x1c7 #6 0xffffffff808d2e55 at fork_exit+0x135 #6 0xffffffff808d2e55 at fork_exit+0x135 #7 0xffffffff80cd5a2e at fork_trampoline+0xe #7 0xffffffff80cd5a2e at fork_trampoline+0xe ino 209869753, on dev hast/links ino 209869753, on dev hast/links 0xfffffe02ad5f6a40: 0xfffffe02ad5f6a40: tag ufs, type VDIR tag ufs, type VDIR usecount 1, writecount 0, refcount 3 mountedhere 0 usecount 1, writecount 0, refcount 3 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) v_object 0xfffffe02ad5c7cb0 ref 0 pages 0 v_object 0xfffffe02ad5c7cb0 ref 0 pages 0 lock type ufs: EXCL by thread 0xfffffe010e5df920 (pid 42646) lock type ufs: EXCL by thread 0xfffffe010e5df920 (pid 42646) #0 0xffffffff808e827c at __lockmgr_args+0xd5c #0 0xffffffff808e827c at __lockmgr_args+0xd5c #1 0xffffffff80b520fc at ffs_lock+0x9c #1 0xffffffff80b520fc at ffs_lock+0x9c #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #3 0xffffffff809b62e5 at _vn_lock+0x55 #3 0xffffffff809b62e5 at _vn_lock+0x55 #4 0xffffffff809a87cb at vget+0x7b #4 0xffffffff809a87cb at vget+0x7b #5 0xffffffff80990df3 at cache_lookup_times+0x553 #5 0xffffffff80990df3 at cache_lookup_times+0x553 #6 0xffffffff80991298 at vfs_cache_lookup+0xc8 #6 0xffffffff80991298 at vfs_cache_lookup+0xc8 #7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102 #7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102 #8 0xffffffff80998bdf at lookup+0x32f #8 0xffffffff80998bdf at lookup+0x32f #9 0xffffffff80999c9f at namei+0x40f #9 0xffffffff80999c9f at namei+0x40f #10 0xffffffff809b5991 at vn_open_cred+0xc1 #10 0xffffffff809b5991 at vn_open_cred+0xc1 #11 0xffffffff809b209a at kern_openat+0x20a #11 0xffffffff809b209a at kern_openat+0x20a #12 0xffffffff80ceb239 at amd64_syscall+0x2f9 #12 0xffffffff80ceb239 at amd64_syscall+0x2f9 #13 0xffffffff80cd57e7 at Xfast_syscall+0xf7 #13 0xffffffff80cd57e7 at Xfast_syscall+0xf7 ino 69421440, on dev hast/links ino 69421440, on dev hast/links 0xfffffe02ad5f8000: 0xfffffe02ad5f8000: tag ufs, type VDIR tag ufs, type VDIR usecount 1, writecount 0, refcount 3 mountedhere 0 usecount 1, writecount 0, refcount 3 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) v_object 0xfffffe02ad51a2b8 ref 0 pages 0 v_object 0xfffffe02ad51a2b8 ref 0 pages 0 lock type ufs: EXCL by thread 0xfffffe000ddbb920 (pid 42773) lock type ufs: EXCL by thread 0xfffffe000ddbb920 (pid 42773) #0 0xffffffff808e827c at __lockmgr_args+0xd5c #0 0xffffffff808e827c at __lockmgr_args+0xd5c #1 0xffffffff80b520fc at ffs_lock+0x9c #1 0xffffffff80b520fc at ffs_lock+0x9c #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #3 0xffffffff809b62e5 at _vn_lock+0x55 #3 0xffffffff809b62e5 at _vn_lock+0x55 #4 0xffffffff809a87cb at vget+0x7b #4 0xffffffff809a87cb at vget+0x7b #5 0xffffffff80990df3 at cache_lookup_times+0x553 #5 0xffffffff80990df3 at cache_lookup_times+0x553 #6 0xffffffff80991298 at vfs_cache_lookup+0xc8 #6 0xffffffff80991298 at vfs_cache_lookup+0xc8 #7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102 #7 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102 #8 0xffffffff80998bdf at lookup+0x32f #8 0xffffffff80998bdf at lookup+0x32f #9 0xffffffff80999c9f at namei+0x40f #9 0xffffffff80999c9f at namei+0x40f #10 0xffffffff809b5991 at vn_open_cred+0xc1 #10 0xffffffff809b5991 at vn_open_cred+0xc1 #11 0xffffffff809b209a at kern_openat+0x20a #11 0xffffffff809b209a at kern_openat+0x20a #12 0xffffffff80ceb239 at amd64_syscall+0x2f9 #12 0xffffffff80ceb239 at amd64_syscall+0x2f9 #13 0xffffffff80cd57e7 at Xfast_syscall+0xf7 #13 0xffffffff80cd57e7 at Xfast_syscall+0xf7 ino 53932032, on dev hast/links ino 53932032, on dev hast/links 0xfffffe02ad91e000: 0xfffffe02ad91e000: tag ufs, type VNON tag ufs, type VNON usecount 1, writecount 0, refcount 1 mountedhere 0 usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) lock type ufs: EXCL by thread 0xfffffe010e5df920 (pid 42646) lock type ufs: EXCL by thread 0xfffffe010e5df920 (pid 42646) #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #1 0xffffffff80b4c593 at ffs_vgetf+0x163 #1 0xffffffff80b4c593 at ffs_vgetf+0x163 #2 0xffffffff80b24341 at ffs_valloc+0x4a1 #2 0xffffffff80b24341 at ffs_valloc+0x4a1 #3 0xffffffff80b5f5af at ufs_makeinode+0xaf #3 0xffffffff80b5f5af at ufs_makeinode+0xaf #4 0xffffffff80d95262 at VOP_CREATE_APV+0x102 #4 0xffffffff80d95262 at VOP_CREATE_APV+0x102 #5 0xffffffff809b5d2e at vn_open_cred+0x45e #5 0xffffffff809b5d2e at vn_open_cred+0x45e #6 0xffffffff809b209a at kern_openat+0x20a #6 0xffffffff809b209a at kern_openat+0x20a #7 0xffffffff80ceb239 at amd64_syscall+0x2f9 #7 0xffffffff80ceb239 at amd64_syscall+0x2f9 #8 0xffffffff80cd57e7 at Xfast_syscall+0xf7 #8 0xffffffff80cd57e7 at Xfast_syscall+0xf7 ino 69421952, on dev hast/links ino 69421952, on dev hast/links 0xfffffe02ad8c0a40: 0xfffffe02ad8c0a40: tag ufs, type VNON tag ufs, type VNON usecount 1, writecount 0, refcount 1 mountedhere 0 usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) lock type ufs: EXCL by thread 0xfffffe000ddbb920 (pid 42773) lock type ufs: EXCL by thread 0xfffffe000ddbb920 (pid 42773) #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #1 0xffffffff80b4c593 at ffs_vgetf+0x163 #1 0xffffffff80b4c593 at ffs_vgetf+0x163 #2 0xffffffff80b24341 at ffs_valloc+0x4a1 #2 0xffffffff80b24341 at ffs_valloc+0x4a1 #3 0xffffffff80b5f5af at ufs_makeinode+0xaf #3 0xffffffff80b5f5af at ufs_makeinode+0xaf #4 0xffffffff80d95262 at VOP_CREATE_APV+0x102 #4 0xffffffff80d95262 at VOP_CREATE_APV+0x102 #5 0xffffffff809b5d2e at vn_open_cred+0x45e #5 0xffffffff809b5d2e at vn_open_cred+0x45e #6 0xffffffff809b209a at kern_openat+0x20a #6 0xffffffff809b209a at kern_openat+0x20a #7 0xffffffff80ceb239 at amd64_syscall+0x2f9 #7 0xffffffff80ceb239 at amd64_syscall+0x2f9 #8 0xffffffff80cd57e7 at Xfast_syscall+0xf7 #8 0xffffffff80cd57e7 at Xfast_syscall+0xf7 ino 53932416, on dev hast/links ino 53932416, on dev hast/links 0xfffffe022cfec520: 0xfffffe022cfec520: tag ufs, type VREG tag ufs, type VREG usecount 0, writecount 0, refcount 4 mountedhere 0 usecount 0, writecount 0, refcount 4 mountedhere 0 flags (VI_ACTIVE|VI_DOINGINACT) flags (VI_ACTIVE|VI_DOINGINACT) v_object 0xfffffe024803c570 ref 0 pages 11 v_object 0xfffffe024803c570 ref 0 pages 11 lock type ufs: EXCL by thread 0xfffffe002896a000 (pid 2087) lock type ufs: EXCL by thread 0xfffffe002896a000 (pid 2087) #0 0xffffffff808e827c at __lockmgr_args+0xd5c #0 0xffffffff808e827c at __lockmgr_args+0xd5c #1 0xffffffff80b520fc at ffs_lock+0x9c #1 0xffffffff80b520fc at ffs_lock+0x9c #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #2 0xffffffff80d93df3 at VOP_LOCK1_APV+0xe3 #3 0xffffffff809b62e5 at _vn_lock+0x55 #3 0xffffffff809b62e5 at _vn_lock+0x55 #4 0xffffffff809b7f4a at vn_close+0xca #4 0xffffffff809b7f4a at vn_close+0xca #5 0xffffffff809b80cd at vn_closefile+0x5d #5 0xffffffff809b80cd at vn_closefile+0x5d #6 0xffffffff808c2583 at _fdrop+0x23 #6 0xffffffff808c2583 at _fdrop+0x23 #7 0xffffffff808c4adc at closef+0x5c #7 0xffffffff808c4adc at closef+0x5c #8 0xffffffff808c50e1 at kern_close+0x121 #8 0xffffffff808c50e1 at kern_close+0x121 #9 0xffffffff80ceb239 at amd64_syscall+0x2f9 #9 0xffffffff80ceb239 at amd64_syscall+0x2f9 #10 0xffffffff80cd57e7 at Xfast_syscall+0xf7 #10 0xffffffff80cd57e7 at Xfast_syscall+0xf7 ino 1204245, on dev hast/links ino 1204245, on dev hast/links 0xfffffe0248026520: 0xfffffe0248026520: tag ufs, type VNON tag ufs, type VNON usecount 1, writecount 0, refcount 1 mountedhere 0 usecount 1, writecount 0, refcount 1 mountedhere 0 flags (VI_ACTIVE) flags (VI_ACTIVE) lock type ufs: EXCL by thread 0xfffffe000ddbb000 (pid 3076) lock type ufs: EXCL by thread 0xfffffe000ddbb000 (pid 3076) #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #0 0xffffffff808e7e2e at __lockmgr_args+0x90e #1 0xffffffff80b4c593 at ffs_vgetf+0x163 #1 0xffffffff80b4c593 at ffs_vgetf+0x163 #2 0xffffffff80b58d8c at ufs_lookup_ino+0xcec #2 0xffffffff80b58d8c at ufs_lookup_ino+0xcec #3 0xffffffff80d925b2 at VOP_CACHEDLOOKUP_APV+0x102 #3 0xffffffff80d925b2 at VOP_CACHEDLOOKUP_APV+0x102 #4 0xffffffff809912c8 at vfs_cache_lookup+0xf8 #4 0xffffffff809912c8 at vfs_cache_lookup+0xf8 #5 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102 #5 0xffffffff80d927e2 at VOP_LOOKUP_APV+0x102 #6 0xffffffff80998bdf at lookup+0x32f #6 0xffffffff80998bdf at lookup+0x32f #7 0xffffffff80999c9f at namei+0x40f #7 0xffffffff80999c9f at namei+0x40f #8 0xffffffff809ac5b6 at kern_statat_vnhook+0xb6 #8 0xffffffff809ac5b6 at kern_statat_vnhook+0xb6 #9 0xffffffff809ac7d5 at kern_statat+0x15 #9 0xffffffff809ac7d5 at kern_statat+0x15 #10 0xffffffff809ac89a at sys_lstat+0x2a #10 0xffffffff809ac89a at sys_lstat+0x2a #11 0xffffffff80ceb239 at amd64_syscall+0x2f9 #11 0xffffffff80ceb239 at amd64_syscall+0x2f9 #12 0xffffffff80cd57e7 at Xfast_syscall+0xf7 #12 0xffffffff80cd57e7 at Xfast_syscall+0xf7 ino 1204244, on dev hast/links ino 1204244, on dev hast/links ----------------------------------------------------------------------- db> alltrace Tracing command sysctl pid 42798 tid 100227 td 0xfffffe002896a490 kdb_enter() at kdb_enter+0x3b/frame 0xffffff8871484900 kdb_sysctl_enter() at kdb_sysctl_enter+0x89/frame 0xffffff8871484940 sysctl_root() at sysctl_root+0x147/frame 0xffffff8871484990 userland_sysctl() at userland_sysctl+0x158/frame 0xffffff8871484a40 sys___sysctl() at sys___sysctl+0xaa/frame 0xffffff8871484b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871484c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871484c30 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip =3D 0x80092713c, rsp = =3D 0x7fffffffd278, rbp =3D 0x7fffffffd760 --- Tracing command mv pid 42785 tid 100292 td 0xfffffe010e5db000 sched_switch() at sched_switch+0x194/frame 0xffffff88715c97a0 mi_switch() at mi_switch+0x208/frame 0xffffff88715c97f0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88715c9820 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88715c9850 _sleep() at _sleep+0x3d4/frame 0xffffff88715c98e0 bwillwrite() at bwillwrite+0xc9/frame 0xffffff88715c98f0 kern_renameat() at kern_renameat+0x4c/frame 0xffffff88715c9b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88715c9c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88715c9c30 --- syscall (128, FreeBSD ELF64, sys_rename), rip =3D 0x80088069c, rsp =3D = 0x7fffffffd0d8, rbp =3D 0x7fffffffdf0b --- Tracing command sh pid 42780 tid 100249 td 0xfffffe010e5bc920 sched_switch() at sched_switch+0x194/frame 0xffffff88714f26d0 mi_switch() at mi_switch+0x208/frame 0xffffff88714f2720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714f2750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714f27= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88714f27d0 _sleep() at _sleep+0x368/frame 0xffffff88714f2860 kern_wait6() at kern_wait6+0x5e2/frame 0xffffff88714f28f0 kern_wait() at kern_wait+0x9c/frame 0xffffff88714f2a50 sys_wait4() at sys_wait4+0x35/frame 0xffffff88714f2b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714f2c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714f2c30 --- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x800d2e4ac, rsp =3D 0x7= fffffffd838, rbp =3D 0x7fffffffda08 --- Tracing command cron pid 42778 tid 100332 td 0xfffffe0028efa490 sched_switch() at sched_switch+0x194/frame 0xffffff8871691850 mi_switch() at mi_switch+0x208/frame 0xffffff88716918a0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716918d0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88716919= 30 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871691950 _sleep() at _sleep+0x368/frame 0xffffff88716919e0 pipe_read() at pipe_read+0x4a2/frame 0xffffff8871691a30 dofileread() at dofileread+0xa1/frame 0xffffff8871691a80 kern_readv() at kern_readv+0x6c/frame 0xffffff8871691ac0 sys_read() at sys_read+0x64/frame 0xffffff8871691b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871691c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871691c30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800d4bb1c, rsp =3D 0x7f= ffffffd478, rbp =3D 0x800f9a560 --- Tracing command mktemp pid 42773 tid 100131 td 0xfffffe000ddbb920 sched_switch() at sched_switch+0x194/frame 0xffffff88712a4270 mi_switch() at mi_switch+0x208/frame 0xffffff88712a42c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712a42f0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712a4320 __lockmgr_args() at __lockmgr_args+0x921/frame 0xffffff88712a4400 getblk() at getblk+0x119/frame 0xffffff88712a4490 breadn_flags() at breadn_flags+0x40/frame 0xffffff88712a44e0 bread() at bread+0x26/frame 0xffffff88712a4510 ffs_vgetf() at ffs_vgetf+0x2e9/frame 0xffffff88712a4590 ffs_valloc() at ffs_valloc+0x4a1/frame 0xffffff88712a4630 ufs_makeinode() at ufs_makeinode+0xaf/frame 0xffffff88712a47f0 VOP_CREATE_APV() at VOP_CREATE_APV+0x102/frame 0xffffff88712a4820 vn_open_cred() at vn_open_cred+0x45e/frame 0xffffff88712a4990 kern_openat() at kern_openat+0x20a/frame 0xffffff88712a4b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712a4c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712a4c30 --- syscall (5, FreeBSD ELF64, sys_open), rip =3D 0x80091cd3c, rsp =3D 0x7f= ffffffd538, rbp =3D 0x800c0704f --- Tracing command mktemp pid 42646 tid 100260 td 0xfffffe010e5df920 sched_switch() at sched_switch+0x194/frame 0xffffff8871529270 mi_switch() at mi_switch+0x208/frame 0xffffff88715292c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88715292f0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8871529320 __lockmgr_args() at __lockmgr_args+0x921/frame 0xffffff8871529400 getblk() at getblk+0x119/frame 0xffffff8871529490 breadn_flags() at breadn_flags+0x40/frame 0xffffff88715294e0 bread() at bread+0x26/frame 0xffffff8871529510 ffs_vgetf() at ffs_vgetf+0x2e9/frame 0xffffff8871529590 ffs_valloc() at ffs_valloc+0x4a1/frame 0xffffff8871529630 ufs_makeinode() at ufs_makeinode+0xaf/frame 0xffffff88715297f0 VOP_CREATE_APV() at VOP_CREATE_APV+0x102/frame 0xffffff8871529820 vn_open_cred() at vn_open_cred+0x45e/frame 0xffffff8871529990 kern_openat() at kern_openat+0x20a/frame 0xffffff8871529b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871529c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871529c30 --- syscall (5, FreeBSD ELF64, sys_open), rip =3D 0x80091cd3c, rsp =3D 0x7f= ffffffd538, rbp =3D 0x800c0704f --- Tracing command httpd pid 9787 tid 100085 td 0xfffffe000df3b920 sched_switch() at sched_switch+0x194/frame 0xffffff88711be720 mi_switch() at mi_switch+0x208/frame 0xffffff88711be770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711be7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88711be7d0 _sleep() at _sleep+0x3d4/frame 0xffffff88711be860 foffset_lock() at foffset_lock+0x86/frame 0xffffff88711be890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88711be8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88711bea30 dofilewrite() at dofilewrite+0x85/frame 0xffffff88711bea80 kern_writev() at kern_writev+0x6c/frame 0xffffff88711beac0 sys_write() at sys_write+0x64/frame 0xffffff88711beb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711bec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711bec30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 9784 tid 100245 td 0xfffffe010e5f2490 sched_switch() at sched_switch+0x194/frame 0xffffff88714de720 mi_switch() at mi_switch+0x208/frame 0xffffff88714de770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714de7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88714de7d0 _sleep() at _sleep+0x3d4/frame 0xffffff88714de860 foffset_lock() at foffset_lock+0x86/frame 0xffffff88714de890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88714de8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88714dea30 dofilewrite() at dofilewrite+0x85/frame 0xffffff88714dea80 kern_writev() at kern_writev+0x6c/frame 0xffffff88714deac0 sys_write() at sys_write+0x64/frame 0xffffff88714deb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714dec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714dec30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 9783 tid 100099 td 0xfffffe000df39000 sched_switch() at sched_switch+0x194/frame 0xffffff88712046d0 mi_switch() at mi_switch+0x208/frame 0xffffff8871204720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871204750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712047= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712047d0 _sleep() at _sleep+0x368/frame 0xffffff8871204860 lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88712048f0 lf_advlock() at lf_advlock+0x47/frame 0xffffff8871204940 vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff8871204a40 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff8871204a70 sys_flock() at sys_flock+0x13e/frame 0xffffff8871204b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871204c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871204c30 --- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0= x7fffffffd9d8, rbp =3D 0x7fffffffda5c --- Tracing command httpd pid 9779 tid 100315 td 0xfffffe0028575490 sched_switch() at sched_switch+0x194/frame 0xffffff887163c720 mi_switch() at mi_switch+0x208/frame 0xffffff887163c770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887163c7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887163c7d0 _sleep() at _sleep+0x3d4/frame 0xffffff887163c860 foffset_lock() at foffset_lock+0x86/frame 0xffffff887163c890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887163c8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887163ca30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887163ca80 kern_writev() at kern_writev+0x6c/frame 0xffffff887163cac0 sys_write() at sys_write+0x64/frame 0xffffff887163cb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887163cc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887163cc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 9773 tid 100276 td 0xfffffe010e702920 sched_switch() at sched_switch+0x194/frame 0xffffff8871579720 mi_switch() at mi_switch+0x208/frame 0xffffff8871579770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88715797a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88715797d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871579860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871579890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88715798b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871579a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871579a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871579ac0 sys_write() at sys_write+0x64/frame 0xffffff8871579b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871579c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871579c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 9769 tid 100181 td 0xfffffe0028a7d920 sched_switch() at sched_switch+0x194/frame 0xffffff887139e6d0 mi_switch() at mi_switch+0x208/frame 0xffffff887139e720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887139e750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887139e7= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887139e7d0 _sleep() at _sleep+0x368/frame 0xffffff887139e860 lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff887139e8f0 lf_advlock() at lf_advlock+0x47/frame 0xffffff887139e940 vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff887139ea40 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff887139ea70 sys_flock() at sys_flock+0x13e/frame 0xffffff887139eb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887139ec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887139ec30 --- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0= x7fffffffd9d8, rbp =3D 0x7fffffffda5c --- Tracing command httpd pid 9768 tid 100087 td 0xfffffe000ddbc920 sched_switch() at sched_switch+0x194/frame 0xffffff88711c86d0 mi_switch() at mi_switch+0x208/frame 0xffffff88711c8720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711c8750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711c87= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711c87d0 _sleep() at _sleep+0x368/frame 0xffffff88711c8860 lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88711c88f0 lf_advlock() at lf_advlock+0x47/frame 0xffffff88711c8940 vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff88711c8a40 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff88711c8a70 sys_flock() at sys_flock+0x13e/frame 0xffffff88711c8b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711c8c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711c8c30 --- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0= x7fffffffd9d8, rbp =3D 0x7fffffffda5c --- Tracing command httpd pid 9766 tid 100140 td 0xfffffe000df5e920 sched_switch() at sched_switch+0x194/frame 0xffffff88712d16d0 mi_switch() at mi_switch+0x208/frame 0xffffff88712d1720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712d1750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712d17= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712d17d0 _sleep() at _sleep+0x368/frame 0xffffff88712d1860 lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88712d18f0 lf_advlock() at lf_advlock+0x47/frame 0xffffff88712d1940 vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff88712d1a40 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff88712d1a70 sys_flock() at sys_flock+0x13e/frame 0xffffff88712d1b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712d1c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712d1c30 --- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0= x7fffffffd9d8, rbp =3D 0x7fffffffda5c --- Tracing command httpd pid 9764 tid 100233 td 0xfffffe000dfdb490 sched_switch() at sched_switch+0x194/frame 0xffffff88714a26d0 mi_switch() at mi_switch+0x208/frame 0xffffff88714a2720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714a2750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714a27= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88714a27d0 _sleep() at _sleep+0x368/frame 0xffffff88714a2860 lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88714a28f0 lf_advlock() at lf_advlock+0x47/frame 0xffffff88714a2940 vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff88714a2a40 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff88714a2a70 sys_flock() at sys_flock+0x13e/frame 0xffffff88714a2b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714a2c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714a2c30 --- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0= x7fffffffd9d8, rbp =3D 0x7fffffffda5c --- Tracing command httpd pid 9755 tid 100126 td 0xfffffe000df9f920 sched_switch() at sched_switch+0x194/frame 0xffffff887128b720 mi_switch() at mi_switch+0x208/frame 0xffffff887128b770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887128b7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887128b7d0 _sleep() at _sleep+0x3d4/frame 0xffffff887128b860 foffset_lock() at foffset_lock+0x86/frame 0xffffff887128b890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887128b8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887128ba30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887128ba80 kern_writev() at kern_writev+0x6c/frame 0xffffff887128bac0 sys_write() at sys_write+0x64/frame 0xffffff887128bb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887128bc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887128bc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 9750 tid 100120 td 0xfffffe000df5f920 sched_switch() at sched_switch+0x194/frame 0xffffff887126d720 mi_switch() at mi_switch+0x208/frame 0xffffff887126d770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887126d7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887126d7d0 _sleep() at _sleep+0x3d4/frame 0xffffff887126d860 foffset_lock() at foffset_lock+0x86/frame 0xffffff887126d890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887126d8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887126da30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887126da80 kern_writev() at kern_writev+0x6c/frame 0xffffff887126dac0 sys_write() at sys_write+0x64/frame 0xffffff887126db10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887126dc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887126dc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 9749 tid 100142 td 0xfffffe000df9e920 sched_switch() at sched_switch+0x194/frame 0xffffff88712db720 mi_switch() at mi_switch+0x208/frame 0xffffff88712db770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712db7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712db7d0 _sleep() at _sleep+0x3d4/frame 0xffffff88712db860 foffset_lock() at foffset_lock+0x86/frame 0xffffff88712db890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88712db8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88712dba30 dofilewrite() at dofilewrite+0x85/frame 0xffffff88712dba80 kern_writev() at kern_writev+0x6c/frame 0xffffff88712dbac0 sys_write() at sys_write+0x64/frame 0xffffff88712dbb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712dbc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712dbc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 9748 tid 100127 td 0xfffffe000df5f000 sched_switch() at sched_switch+0x194/frame 0xffffff88712906d0 mi_switch() at mi_switch+0x208/frame 0xffffff8871290720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871290750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712907= b0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88712907e0 _sleep() at _sleep+0x3ae/frame 0xffffff8871290870 kern_kevent() at kern_kevent+0x331/frame 0xffffff8871290a50 sys_kevent() at sys_kevent+0x90/frame 0xffffff8871290b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871290c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871290c30 --- syscall (363, FreeBSD ELF64, sys_kevent), rip =3D 0x80228ed8c, rsp =3D = 0x7fffffffd958, rbp =3D 0x8029112f0 --- Tracing command httpd pid 8852 tid 100307 td 0xfffffe0278833000 sched_switch() at sched_switch+0x194/frame 0xffffff8871614720 mi_switch() at mi_switch+0x208/frame 0xffffff8871614770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716147a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88716147d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871614860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871614890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88716148b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871614a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871614a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871614ac0 sys_write() at sys_write+0x64/frame 0xffffff8871614b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871614c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871614c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 8851 tid 100240 td 0xfffffe010e5f3490 sched_switch() at sched_switch+0x194/frame 0xffffff88714c5720 mi_switch() at mi_switch+0x208/frame 0xffffff88714c5770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714c57a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88714c57d0 _sleep() at _sleep+0x3d4/frame 0xffffff88714c5860 foffset_lock() at foffset_lock+0x86/frame 0xffffff88714c5890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88714c58b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88714c5a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff88714c5a80 kern_writev() at kern_writev+0x6c/frame 0xffffff88714c5ac0 sys_write() at sys_write+0x64/frame 0xffffff88714c5b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88714c5c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88714c5c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 8849 tid 100328 td 0xfffffe02788e8000 sched_switch() at sched_switch+0x194/frame 0xffffff887167d360 mi_switch() at mi_switch+0x208/frame 0xffffff887167d3b0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887167d3e0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887167d410 _sleep() at _sleep+0x3d4/frame 0xffffff887167d4a0 getpbuf() at getpbuf+0x4e/frame 0xffffff887167d4c0 cluster_wbuild_gb() at cluster_wbuild_gb+0x21e/frame 0xffffff887167d570 cluster_write_gb() at cluster_write_gb+0x651/frame 0xffffff887167d670 ffs_write() at ffs_write+0x65e/frame 0xffffff887167d710 VOP_WRITE_APV() at VOP_WRITE_APV+0x16e/frame 0xffffff887167d820 vn_write() at vn_write+0x311/frame 0xffffff887167d8b0 vn_io_fault() at vn_io_fault+0x9e/frame 0xffffff887167da30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887167da80 kern_writev() at kern_writev+0x6c/frame 0xffffff887167dac0 sys_write() at sys_write+0x64/frame 0xffffff887167db10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887167dc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887167dc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 8847 tid 100327 td 0xfffffe02788e8490 sched_switch() at sched_switch+0x194/frame 0xffffff8871678720 mi_switch() at mi_switch+0x208/frame 0xffffff8871678770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716787a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88716787d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871678860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871678890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88716788b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871678a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871678a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871678ac0 sys_write() at sys_write+0x64/frame 0xffffff8871678b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871678c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871678c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 8842 tid 100326 td 0xfffffe02788e8920 sched_switch() at sched_switch+0x194/frame 0xffffff8871673720 mi_switch() at mi_switch+0x208/frame 0xffffff8871673770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716737a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88716737d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871673860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871673890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88716738b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871673a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871673a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871673ac0 sys_write() at sys_write+0x64/frame 0xffffff8871673b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871673c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871673c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 8840 tid 100325 td 0xfffffe02788ee000 sched_switch() at sched_switch+0x194/frame 0xffffff887166e720 mi_switch() at mi_switch+0x208/frame 0xffffff887166e770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887166e7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887166e7d0 _sleep() at _sleep+0x3d4/frame 0xffffff887166e860 foffset_lock() at foffset_lock+0x86/frame 0xffffff887166e890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887166e8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887166ea30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887166ea80 kern_writev() at kern_writev+0x6c/frame 0xffffff887166eac0 sys_write() at sys_write+0x64/frame 0xffffff887166eb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887166ec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887166ec30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 8839 tid 100324 td 0xfffffe0028efd490 sched_switch() at sched_switch+0x194/frame 0xffffff8871669720 mi_switch() at mi_switch+0x208/frame 0xffffff8871669770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88716697a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88716697d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871669860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871669890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88716698b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871669a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871669a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871669ac0 sys_write() at sys_write+0x64/frame 0xffffff8871669b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871669c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871669c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 8837 tid 100132 td 0xfffffe000ddbb490 sched_switch() at sched_switch+0x194/frame 0xffffff88712a9720 mi_switch() at mi_switch+0x208/frame 0xffffff88712a9770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712a97a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712a97d0 _sleep() at _sleep+0x3d4/frame 0xffffff88712a9860 foffset_lock() at foffset_lock+0x86/frame 0xffffff88712a9890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88712a98b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88712a9a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff88712a9a80 kern_writev() at kern_writev+0x6c/frame 0xffffff88712a9ac0 sys_write() at sys_write+0x64/frame 0xffffff88712a9b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712a9c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712a9c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 7978 tid 100141 td 0xfffffe000de77000 sched_switch() at sched_switch+0x194/frame 0xffffff88712d6720 mi_switch() at mi_switch+0x208/frame 0xffffff88712d6770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712d67a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712d67d0 _sleep() at _sleep+0x3d4/frame 0xffffff88712d6860 foffset_lock() at foffset_lock+0x86/frame 0xffffff88712d6890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88712d68b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88712d6a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff88712d6a80 kern_writev() at kern_writev+0x6c/frame 0xffffff88712d6ac0 sys_write() at sys_write+0x64/frame 0xffffff88712d6b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712d6c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712d6c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 7974 tid 100262 td 0xfffffe010e5df000 sched_switch() at sched_switch+0x194/frame 0xffffff88715336d0 mi_switch() at mi_switch+0x208/frame 0xffffff8871533720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871533750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88715337= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88715337d0 _sleep() at _sleep+0x368/frame 0xffffff8871533860 lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88715338f0 lf_advlock() at lf_advlock+0x47/frame 0xffffff8871533940 vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff8871533a40 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff8871533a70 sys_flock() at sys_flock+0x13e/frame 0xffffff8871533b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871533c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871533c30 --- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0= x7fffffffd9d8, rbp =3D 0x7fffffffda5c --- Tracing command httpd pid 7971 tid 100208 td 0xfffffe0028eea920 sched_switch() at sched_switch+0x194/frame 0xffffff8871425720 mi_switch() at mi_switch+0x208/frame 0xffffff8871425770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88714257a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88714257d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871425860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871425890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88714258b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871425a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871425a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871425ac0 sys_write() at sys_write+0x64/frame 0xffffff8871425b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871425c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871425c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 7969 tid 100098 td 0xfffffe000df39490 sched_switch() at sched_switch+0x194/frame 0xffffff88711ff720 mi_switch() at mi_switch+0x208/frame 0xffffff88711ff770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711ff7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88711ff7d0 _sleep() at _sleep+0x3d4/frame 0xffffff88711ff860 foffset_lock() at foffset_lock+0x86/frame 0xffffff88711ff890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88711ff8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff88711ffa30 dofilewrite() at dofilewrite+0x85/frame 0xffffff88711ffa80 kern_writev() at kern_writev+0x6c/frame 0xffffff88711ffac0 sys_write() at sys_write+0x64/frame 0xffffff88711ffb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711ffc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711ffc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 7166 tid 100278 td 0xfffffe010e702000 sched_switch() at sched_switch+0x194/frame 0xffffff8871583720 mi_switch() at mi_switch+0x208/frame 0xffffff8871583770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88715837a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88715837d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871583860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871583890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88715838b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871583a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871583a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871583ac0 sys_write() at sys_write+0x64/frame 0xffffff8871583b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871583c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871583c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command httpd pid 7164 tid 100175 td 0xfffffe0028a7e490 sched_switch() at sched_switch+0x194/frame 0xffffff88713806d0 mi_switch() at mi_switch+0x208/frame 0xffffff8871380720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871380750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713807= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713807d0 _sleep() at _sleep+0x368/frame 0xffffff8871380860 lf_advlockasync() at lf_advlockasync+0xddd/frame 0xffffff88713808f0 lf_advlock() at lf_advlock+0x47/frame 0xffffff8871380940 vop_stdadvlock() at vop_stdadvlock+0xb3/frame 0xffffff8871380a40 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x105/frame 0xffffff8871380a70 sys_flock() at sys_flock+0x13e/frame 0xffffff8871380b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871380c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871380c30 --- syscall (131, FreeBSD ELF64, sys_flock), rip =3D 0x80223447c, rsp =3D 0= x7fffffffd9d8, rbp =3D 0x7fffffffda5c --- Tracing command httpd pid 6396 tid 100115 td 0xfffffe000daef000 sched_switch() at sched_switch+0x194/frame 0xffffff8871254720 mi_switch() at mi_switch+0x208/frame 0xffffff8871254770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712547a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88712547d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871254860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871254890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88712548b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871254a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871254a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871254ac0 sys_write() at sys_write+0x64/frame 0xffffff8871254b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871254c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871254c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command cpdup pid 3076 tid 100157 td 0xfffffe000ddbb000 sched_switch() at sched_switch+0x194/frame 0xffffff8871326280 mi_switch() at mi_switch+0x208/frame 0xffffff88713262d0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871326300 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8871326330 __lockmgr_args() at __lockmgr_args+0x921/frame 0xffffff8871326410 getblk() at getblk+0x119/frame 0xffffff88713264a0 breadn_flags() at breadn_flags+0x40/frame 0xffffff88713264f0 bread() at bread+0x26/frame 0xffffff8871326520 ffs_vgetf() at ffs_vgetf+0x2e9/frame 0xffffff88713265a0 ufs_lookup_ino() at ufs_lookup_ino+0xcec/frame 0xffffff88713266b0 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x102/frame 0xffffff88713266= e0 vfs_cache_lookup() at vfs_cache_lookup+0xf8/frame 0xffffff8871326750 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x102/frame 0xffffff8871326780 lookup() at lookup+0x32f/frame 0xffffff8871326820 namei() at namei+0x40f/frame 0xffffff88713268d0 kern_statat_vnhook() at kern_statat_vnhook+0xb6/frame 0xffffff8871326a50 kern_statat() at kern_statat+0x15/frame 0xffffff8871326a70 sys_lstat() at sys_lstat+0x2a/frame 0xffffff8871326b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871326c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871326c30 --- syscall (190, FreeBSD ELF64, sys_lstat), rip =3D 0x800b314ac, rsp =3D 0= x7fffffffd128, rbp =3D 0x7fffffffda40 --- Tracing command cpdup pid 2087 tid 100228 td 0xfffffe002896a000 sched_switch() at sched_switch+0x194/frame 0xffffff88714894c0 mi_switch() at mi_switch+0x208/frame 0xffffff8871489510 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871489540 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8871489570 __lockmgr_args() at __lockmgr_args+0x921/frame 0xffffff8871489650 getblk() at getblk+0x119/frame 0xffffff88714896e0 breadn_flags() at breadn_flags+0x40/frame 0xffffff8871489730 ffs_update() at ffs_update+0x117/frame 0xffffff88714897c0 ufs_inactive() at ufs_inactive+0x178/frame 0xffffff8871489810 VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0x105/frame 0xffffff8871489840 vinactive() at vinactive+0xa4/frame 0xffffff8871489890 vputx() at vputx+0x3e1/frame 0xffffff88714898f0 vn_close() at vn_close+0x13a/frame 0xffffff8871489970 vn_closefile() at vn_closefile+0x5d/frame 0xffffff88714899f0 _fdrop() at _fdrop+0x23/frame 0xffffff8871489a10 closef() at closef+0x5c/frame 0xffffff8871489ac0 kern_close() at kern_close+0x121/frame 0xffffff8871489b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871489c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871489c30 --- syscall (6, FreeBSD ELF64, sys_close), rip =3D 0x800b40adc, rsp =3D 0x7= fffffffd128, rbp =3D 0 --- Tracing command sh pid 1359 tid 100184 td 0xfffffe0028a16920 sched_switch() at sched_switch+0x194/frame 0xffffff88713ad850 mi_switch() at mi_switch+0x208/frame 0xffffff88713ad8a0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713ad8d0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713ad9= 30 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713ad950 _sleep() at _sleep+0x368/frame 0xffffff88713ad9e0 pipe_read() at pipe_read+0x4a2/frame 0xffffff88713ada30 dofileread() at dofileread+0xa1/frame 0xffffff88713ada80 kern_readv() at kern_readv+0x6c/frame 0xffffff88713adac0 sys_read() at sys_read+0x64/frame 0xffffff88713adb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713adc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713adc30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800dbbb1c, rsp =3D 0x7f= ffffffd3d8, rbp =3D 0x80140a88e --- Tracing command sh pid 1357 tid 100151 td 0xfffffe000df5e490 sched_switch() at sched_switch+0x194/frame 0xffffff8871308850 mi_switch() at mi_switch+0x208/frame 0xffffff88713088a0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713088d0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713089= 30 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871308950 _sleep() at _sleep+0x368/frame 0xffffff88713089e0 pipe_read() at pipe_read+0x4a2/frame 0xffffff8871308a30 dofileread() at dofileread+0xa1/frame 0xffffff8871308a80 kern_readv() at kern_readv+0x6c/frame 0xffffff8871308ac0 sys_read() at sys_read+0x64/frame 0xffffff8871308b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871308c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871308c30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800dbbb1c, rsp =3D 0x7f= ffffffd3d8, rbp =3D 0x80140a88e --- Tracing command httpd pid 1356 tid 100153 td 0xfffffe000df89920 sched_switch() at sched_switch+0x194/frame 0xffffff8871312720 mi_switch() at mi_switch+0x208/frame 0xffffff8871312770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713127a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88713127d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871312860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871312890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88713128b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871312a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871312a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871312ac0 sys_write() at sys_write+0x64/frame 0xffffff8871312b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871312c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871312c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7f8, rbp =3D 0x13 --- Tracing command top pid 1352 tid 100217 td 0xfffffe0028efd920 sched_switch() at sched_switch+0x194/frame 0xffffff8871452700 mi_switch() at mi_switch+0x208/frame 0xffffff8871452750 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871452780 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714527= e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff8871452810 _cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff8871452870 seltdwait() at seltdwait+0x56/frame 0xffffff88714528a0 kern_select() at kern_select+0x6f0/frame 0xffffff8871452ac0 sys_select() at sys_select+0x5d/frame 0xffffff8871452b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871452c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871452c30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x800fa6a9c, rsp =3D 0= x7fffffffd8c8, rbp =3D 0x2f --- Tracing command httpd pid 1351 tid 100160 td 0xfffffe000ddb8000 sched_switch() at sched_switch+0x194/frame 0xffffff8871335720 mi_switch() at mi_switch+0x208/frame 0xffffff8871335770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713357a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88713357d0 _sleep() at _sleep+0x3d4/frame 0xffffff8871335860 foffset_lock() at foffset_lock+0x86/frame 0xffffff8871335890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff88713358b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff8871335a30 dofilewrite() at dofilewrite+0x85/frame 0xffffff8871335a80 kern_writev() at kern_writev+0x6c/frame 0xffffff8871335ac0 sys_write() at sys_write+0x64/frame 0xffffff8871335b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871335c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871335c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7d8, rbp =3D 0x13 --- Tracing command httpd pid 1350 tid 100162 td 0xfffffe000daf2490 sched_switch() at sched_switch+0x194/frame 0xffffff887133f720 mi_switch() at mi_switch+0x208/frame 0xffffff887133f770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887133f7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887133f7d0 _sleep() at _sleep+0x3d4/frame 0xffffff887133f860 foffset_lock() at foffset_lock+0x86/frame 0xffffff887133f890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887133f8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887133fa30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887133fa80 kern_writev() at kern_writev+0x6c/frame 0xffffff887133fac0 sys_write() at sys_write+0x64/frame 0xffffff887133fb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887133fc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887133fc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7d8, rbp =3D 0x13 --- Tracing command httpd pid 1349 tid 100114 td 0xfffffe000dec8490 sched_switch() at sched_switch+0x194/frame 0xffffff887124f720 mi_switch() at mi_switch+0x208/frame 0xffffff887124f770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887124f7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887124f7d0 _sleep() at _sleep+0x3d4/frame 0xffffff887124f860 foffset_lock() at foffset_lock+0x86/frame 0xffffff887124f890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887124f8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887124fa30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887124fa80 kern_writev() at kern_writev+0x6c/frame 0xffffff887124fac0 sys_write() at sys_write+0x64/frame 0xffffff887124fb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887124fc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887124fc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7d8, rbp =3D 0x13 --- Tracing command httpd pid 1348 tid 100165 td 0xfffffe000dfca920 sched_switch() at sched_switch+0x194/frame 0xffffff887134e720 mi_switch() at mi_switch+0x208/frame 0xffffff887134e770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887134e7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887134e7d0 _sleep() at _sleep+0x3d4/frame 0xffffff887134e860 foffset_lock() at foffset_lock+0x86/frame 0xffffff887134e890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887134e8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887134ea30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887134ea80 kern_writev() at kern_writev+0x6c/frame 0xffffff887134eac0 sys_write() at sys_write+0x64/frame 0xffffff887134eb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887134ec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887134ec30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7d8, rbp =3D 0x13 --- Tracing command httpd pid 1347 tid 100104 td 0xfffffe000de80920 sched_switch() at sched_switch+0x194/frame 0xffffff887121d720 mi_switch() at mi_switch+0x208/frame 0xffffff887121d770 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887121d7a0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887121d7d0 _sleep() at _sleep+0x3d4/frame 0xffffff887121d860 foffset_lock() at foffset_lock+0x86/frame 0xffffff887121d890 foffset_lock_uio() at foffset_lock_uio+0x18/frame 0xffffff887121d8b0 vn_io_fault() at vn_io_fault+0x6d/frame 0xffffff887121da30 dofilewrite() at dofilewrite+0x85/frame 0xffffff887121da80 kern_writev() at kern_writev+0x6c/frame 0xffffff887121dac0 sys_write() at sys_write+0x64/frame 0xffffff887121db10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887121dc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887121dc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x8022adafc, rsp =3D 0x7= fffffffd7d8, rbp =3D 0x13 --- Tracing command httpd pid 1346 tid 100095 td 0xfffffe000df3a490 sched_switch() at sched_switch+0x194/frame 0xffffff88711f0700 mi_switch() at mi_switch+0x208/frame 0xffffff88711f0750 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711f0780 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711f07= e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88711f0810 _cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff88711f0870 seltdwait() at seltdwait+0x56/frame 0xffffff88711f08a0 kern_select() at kern_select+0x6f0/frame 0xffffff88711f0ac0 sys_select() at sys_select+0x5d/frame 0xffffff88711f0b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711f0c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711f0c30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x8022ada9c, rsp =3D 0= x7fffffffda28, rbp =3D 0 --- Tracing command tcsh pid 1324 tid 100204 td 0xfffffe0028eeb490 sched_switch() at sched_switch+0x194/frame 0xffffff8871411900 mi_switch() at mi_switch+0x208/frame 0xffffff8871411950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871411980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714119= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871411a00 _sleep() at _sleep+0x368/frame 0xffffff8871411a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871411ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871411b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871411c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871411c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce68, rbp =3D 0x801408500 --- Tracing command csh pid 1322 tid 100089 td 0xfffffe000df3b000 sched_switch() at sched_switch+0x194/frame 0xffffff88711d2900 mi_switch() at mi_switch+0x208/frame 0xffffff88711d2950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711d2980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711d29= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711d2a00 _sleep() at _sleep+0x368/frame 0xffffff88711d2a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff88711d2ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff88711d2b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711d2c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711d2c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce58, rbp =3D 0x801407500 --- Tracing command tcsh pid 1320 tid 100173 td 0xfffffe000de82920 sched_switch() at sched_switch+0x194/frame 0xffffff8871376900 mi_switch() at mi_switch+0x208/frame 0xffffff8871376950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871376980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713769= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871376a00 _sleep() at _sleep+0x368/frame 0xffffff8871376a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871376ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871376b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871376c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871376c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce68, rbp =3D 0x801408500 --- Tracing command csh pid 1318 tid 100212 td 0xfffffe0028938920 sched_switch() at sched_switch+0x194/frame 0xffffff8871439900 mi_switch() at mi_switch+0x208/frame 0xffffff8871439950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871439980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714399= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871439a00 _sleep() at _sleep+0x368/frame 0xffffff8871439a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871439ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871439b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871439c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871439c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce58, rbp =3D 0x801407500 --- Tracing command tcsh pid 1315 tid 100213 td 0xfffffe0028938490 sched_switch() at sched_switch+0x194/frame 0xffffff887143e900 mi_switch() at mi_switch+0x208/frame 0xffffff887143e950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887143e980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887143e9= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887143ea00 _sleep() at _sleep+0x368/frame 0xffffff887143ea90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff887143eae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff887143eb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887143ec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887143ec30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce68, rbp =3D 0x801408500 --- Tracing command csh pid 1313 tid 100124 td 0xfffffe000df68490 sched_switch() at sched_switch+0x194/frame 0xffffff8871281900 mi_switch() at mi_switch+0x208/frame 0xffffff8871281950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871281980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712819= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871281a00 _sleep() at _sleep+0x368/frame 0xffffff8871281a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871281ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871281b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871281c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871281c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce58, rbp =3D 0x801407500 --- Tracing command tcsh pid 1311 tid 100123 td 0xfffffe000df68920 sched_switch() at sched_switch+0x194/frame 0xffffff887127c900 mi_switch() at mi_switch+0x208/frame 0xffffff887127c950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887127c980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887127c9= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887127ca00 _sleep() at _sleep+0x368/frame 0xffffff887127ca90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff887127cae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff887127cb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887127cc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887127cc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce68, rbp =3D 0x801408500 --- Tracing command csh pid 1309 tid 100093 td 0xfffffe000dec9920 sched_switch() at sched_switch+0x194/frame 0xffffff88711e6900 mi_switch() at mi_switch+0x208/frame 0xffffff88711e6950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711e6980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711e69= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711e6a00 _sleep() at _sleep+0x368/frame 0xffffff88711e6a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff88711e6ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff88711e6b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711e6c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711e6c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce58, rbp =3D 0x801407500 --- Tracing command tcsh pid 1296 tid 100206 td 0xfffffe002896b000 sched_switch() at sched_switch+0x194/frame 0xffffff887141b900 mi_switch() at mi_switch+0x208/frame 0xffffff887141b950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887141b980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887141b9= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887141ba00 _sleep() at _sleep+0x368/frame 0xffffff887141ba90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff887141bae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff887141bb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887141bc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887141bc30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce68, rbp =3D 0x801408800 --- Tracing command csh pid 1293 tid 100163 td 0xfffffe000dfdb000 sched_switch() at sched_switch+0x194/frame 0xffffff8871344900 mi_switch() at mi_switch+0x208/frame 0xffffff8871344950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871344980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713449= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871344a00 _sleep() at _sleep+0x368/frame 0xffffff8871344a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871344ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871344b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871344c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871344c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffce58, rbp =3D 0x801407500 --- Tracing command tmux pid 1292 tid 100100 td 0xfffffe000de70490 cpustop_handler() at cpustop_handler+0x28/frame 0xffffff8000280d00 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff8000280d20 trap() at trap+0x355/frame 0xffffff8000280f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff8000280f20 --- trap 0x13, rip =3D 0xffffffff808f01ad, rsp =3D 0xffffff8000280fe0, rbp = =3D 0xffffff8871209910 --- _mtx_lock_sleep() at _mtx_lock_sleep+0x17d/frame 0xffffff8871209910 _mtx_lock_flags() at _mtx_lock_flags+0x106/frame 0xffffff8871209940 sysctl_root() at sysctl_root+0x255/frame 0xffffff8871209990 userland_sysctl() at userland_sysctl+0x158/frame 0xffffff8871209a40 sys___sysctl() at sys___sysctl+0xaa/frame 0xffffff8871209b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871209c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871209c30 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip =3D 0x800ff913c, rsp = =3D 0x7fffffffcae8, rbp =3D 0x7fffffffcb00 --- Tracing command tmux pid 1290 tid 100112 td 0xfffffe000df38920 sched_switch() at sched_switch+0x194/frame 0xffffff88712457c0 mi_switch() at mi_switch+0x208/frame 0xffffff8871245810 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871245840 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712458= a0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712458c0 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff8871245920 seltdwait() at seltdwait+0xac/frame 0xffffff8871245950 sys_poll() at sys_poll+0x421/frame 0xffffff8871245b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871245c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871245c30 --- syscall (209, FreeBSD ELF64, sys_poll), rip =3D 0x800f92b3c, rsp =3D 0x= 7fffffffd0e8, rbp =3D 0x80144f100 --- Tracing command hastd pid 1283 tid 100196 td 0xfffffe0028fda000 sched_switch() at sched_switch+0x194/frame 0xffffff88713e98b0 mi_switch() at mi_switch+0x208/frame 0xffffff88713e9900 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713e9930 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713e99= 90 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713e99b0 _sleep() at _sleep+0x368/frame 0xffffff88713e9a40 do_wait() at do_wait+0x3fd/frame 0xffffff88713e9ad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff88713e9b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713e9c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713e9c30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7ffffeff7ef8, rbp =3D 0 --- Tracing command hastd pid 1283 tid 100195 td 0xfffffe000de97000 sched_switch() at sched_switch+0x194/frame 0xffffff88713e48b0 mi_switch() at mi_switch+0x208/frame 0xffffff88713e4900 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713e4930 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713e49= 90 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713e49b0 _sleep() at _sleep+0x368/frame 0xffffff88713e4a40 do_wait() at do_wait+0x3fd/frame 0xffffff88713e4ad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff88713e4b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713e4c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713e4c30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7fffff1f8ed8, rbp =3D 0 --- Tracing command hastd pid 1283 tid 100194 td 0xfffffe000de97490 sched_switch() at sched_switch+0x194/frame 0xffffff88713df8a0 mi_switch() at mi_switch+0x208/frame 0xffffff88713df8f0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713df920 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713df9= 80 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88713df9b0 _sleep() at _sleep+0x3ae/frame 0xffffff88713dfa40 do_wait() at do_wait+0x53a/frame 0xffffff88713dfad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff88713dfb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713dfc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713dfc30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7fffff3f9eb8, rbp =3D 0 --- Tracing command hastd pid 1283 tid 100193 td 0xfffffe000de97920 sched_switch() at sched_switch+0x194/frame 0xffffff88713da7f0 mi_switch() at mi_switch+0x208/frame 0xffffff88713da840 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713da870 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff88713da8a0 _sleep() at _sleep+0x3d4/frame 0xffffff88713da930 bwait() at bwait+0x64/frame 0xffffff88713da960 physio() at physio+0x210/frame 0xffffff88713da9b0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff88713daa20 dofileread() at dofileread+0xa1/frame 0xffffff88713daa70 kern_preadv() at kern_preadv+0x68/frame 0xffffff88713daac0 sys_pread() at sys_pread+0x68/frame 0xffffff88713dab10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713dac30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713dac30 --- syscall (475, FreeBSD ELF64, sys_pread), rip =3D 0x80175596c, rsp =3D 0= x7fffff5faf98, rbp =3D 0x801cb3700 --- Tracing command hastd pid 1283 tid 100192 td 0xfffffe000de76000 sched_switch() at sched_switch+0x194/frame 0xffffff88713d56d0 mi_switch() at mi_switch+0x208/frame 0xffffff88713d5720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713d5750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713d57= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713d57d0 _sleep() at _sleep+0x368/frame 0xffffff88713d5860 g_gate_ioctl() at g_gate_ioctl+0x5d6/frame 0xffffff88713d5a00 devfs_ioctl_f() at devfs_ioctl_f+0x7a/frame 0xffffff88713d5a60 kern_ioctl() at kern_ioctl+0xcf/frame 0xffffff88713d5ab0 sys_ioctl() at sys_ioctl+0xfd/frame 0xffffff88713d5b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713d5c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713d5c30 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip =3D 0x801756abc, rsp =3D 0x= 7fffff7fbf88, rbp =3D 0x801d00800 --- Tracing command hastd pid 1283 tid 100191 td 0xfffffe000de76490 sched_switch() at sched_switch+0x194/frame 0xffffff88713d0780 mi_switch() at mi_switch+0x208/frame 0xffffff88713d07d0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713d0800 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713d08= 60 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713d0880 _sleep() at _sleep+0x368/frame 0xffffff88713d0910 soreceive_generic() at soreceive_generic+0xe5f/frame 0xffffff88713d09c0 kern_recvit() at kern_recvit+0x1c4/frame 0xffffff88713d0a80 recvit() at recvit+0x21/frame 0xffffff88713d0aa0 sys_recvfrom() at sys_recvfrom+0x82/frame 0xffffff88713d0b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713d0c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713d0c30 --- syscall (29, FreeBSD ELF64, sys_recvfrom), rip =3D 0x8016ffe0c, rsp =3D= 0x7fffff9fcec8, rbp =3D 0x7fffff9fcf60 --- =20 Tracing command hastd pid 1283 tid 100190 td 0xfffffe000de76920 sched_switch() at sched_switch+0x194/frame 0xffffff88713cb7c0 mi_switch() at mi_switch+0x208/frame 0xffffff88713cb810 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713cb840 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713cb8= a0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88713cb8d0 _sleep() at _sleep+0x3ae/frame 0xffffff88713cb960 kern_sigtimedwait() at kern_sigtimedwait+0x280/frame 0xffffff88713cba40 sys_sigtimedwait() at sys_sigtimedwait+0x95/frame 0xffffff88713cbb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713cbc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713cbc30 --- syscall (345, FreeBSD ELF64, sys_sigtimedwait), rip =3D 0x8016b091c, rs= p =3D 0x7fffffbfdf08, rbp =3D 0x5215a7e6 --- Tracing command hastd pid 1283 tid 100130 td 0xfffffe000df9f000 sched_switch() at sched_switch+0x194/frame 0xffffff887129f8b0 mi_switch() at mi_switch+0x208/frame 0xffffff887129f900 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887129f930 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887129f9= 90 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887129f9b0 _sleep() at _sleep+0x368/frame 0xffffff887129fa40 do_wait() at do_wait+0x3fd/frame 0xffffff887129fad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff887129fb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887129fc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887129fc30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7fffffffcb68, rbp =3D 0 --- Tracing command hastd pid 1282 tid 100203 td 0xfffffe002896b920 sched_switch() at sched_switch+0x194/frame 0xffffff887140c8b0 mi_switch() at mi_switch+0x208/frame 0xffffff887140c900 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887140c930 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887140c9= 90 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887140c9b0 _sleep() at _sleep+0x368/frame 0xffffff887140ca40 do_wait() at do_wait+0x3fd/frame 0xffffff887140cad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff887140cb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887140cc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887140cc30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7ffffeff7ef8, rbp =3D 0 --- Tracing command hastd pid 1282 tid 100202 td 0xfffffe0028a14000 sched_switch() at sched_switch+0x194/frame 0xffffff88714078b0 mi_switch() at mi_switch+0x208/frame 0xffffff8871407900 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871407930 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714079= 90 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88714079b0 _sleep() at _sleep+0x368/frame 0xffffff8871407a40 do_wait() at do_wait+0x3fd/frame 0xffffff8871407ad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff8871407b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871407c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871407c30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7fffff1f8ed8, rbp =3D 0 --- Tracing command hastd pid 1282 tid 100201 td 0xfffffe0028a14490 sched_switch() at sched_switch+0x194/frame 0xffffff88714028a0 mi_switch() at mi_switch+0x208/frame 0xffffff88714028f0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871402920 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88714029= 80 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88714029b0 _sleep() at _sleep+0x3ae/frame 0xffffff8871402a40 do_wait() at do_wait+0x53a/frame 0xffffff8871402ad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff8871402b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871402c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871402c30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7fffff3f9eb8, rbp =3D 0 --- Tracing command hastd pid 1282 tid 100200 td 0xfffffe0028a14920 sched_switch() at sched_switch+0x194/frame 0xffffff88713fd8b0 mi_switch() at mi_switch+0x208/frame 0xffffff88713fd900 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713fd930 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713fd9= 90 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713fd9b0 _sleep() at _sleep+0x368/frame 0xffffff88713fda40 do_wait() at do_wait+0x3fd/frame 0xffffff88713fdad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff88713fdb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713fdc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713fdc30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7fffff5faef8, rbp =3D 0 --- Tracing command hastd pid 1282 tid 100199 td 0xfffffe0028a15000 sched_switch() at sched_switch+0x194/frame 0xffffff88713f88b0 mi_switch() at mi_switch+0x208/frame 0xffffff88713f8900 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713f8930 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713f89= 90 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713f89b0 _sleep() at _sleep+0x368/frame 0xffffff88713f8a40 do_wait() at do_wait+0x3fd/frame 0xffffff88713f8ad0 __umtx_op_wait_uint_private() at __umtx_op_wait_uint_private+0x6e/frame 0xf= fffff88713f8b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713f8c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713f8c30 --- syscall (454, FreeBSD ELF64, sys__umtx_op), rip =3D 0x800e7f4dc, rsp = =3D 0x7fffff7fbee8, rbp =3D 0 --- Tracing command hastd pid 1282 tid 100198 td 0xfffffe0028a15490 sched_switch() at sched_switch+0x194/frame 0xffffff88713f3780 mi_switch() at mi_switch+0x208/frame 0xffffff88713f37d0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713f3800 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713f38= 60 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713f3880 _sleep() at _sleep+0x368/frame 0xffffff88713f3910 soreceive_generic() at soreceive_generic+0xe5f/frame 0xffffff88713f39c0 kern_recvit() at kern_recvit+0x1c4/frame 0xffffff88713f3a80 recvit() at recvit+0x21/frame 0xffffff88713f3aa0 sys_recvfrom() at sys_recvfrom+0x82/frame 0xffffff88713f3b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713f3c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713f3c30 --- syscall (29, FreeBSD ELF64, sys_recvfrom), rip =3D 0x8016ffe0c, rsp =3D= 0x7fffff9fcec8, rbp =3D 0x7fffff9fcf60 --- Tracing command hastd pid 1282 tid 100197 td 0xfffffe0028a15920 sched_switch() at sched_switch+0x194/frame 0xffffff88713ee7c0 mi_switch() at mi_switch+0x208/frame 0xffffff88713ee810 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713ee840 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713ee8= a0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88713ee8d0 _sleep() at _sleep+0x3ae/frame 0xffffff88713ee960 kern_sigtimedwait() at kern_sigtimedwait+0x280/frame 0xffffff88713eea40 sys_sigtimedwait() at sys_sigtimedwait+0x95/frame 0xffffff88713eeb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713eec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713eec30 --- syscall (345, FreeBSD ELF64, sys_sigtimedwait), rip =3D 0x8016b091c, rs= p =3D 0x7fffffbfdf08, rbp =3D 0x5215a7e6 --- Tracing command hastd pid 1282 tid 100117 td 0xfffffe000daee920 sched_switch() at sched_switch+0x194/frame 0xffffff887125e7f0 mi_switch() at mi_switch+0x208/frame 0xffffff887125e840 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887125e870 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887125e8a0 _sleep() at _sleep+0x3d4/frame 0xffffff887125e930 bwait() at bwait+0x64/frame 0xffffff887125e960 physio() at physio+0x2b4/frame 0xffffff887125e9b0 devfs_write_f() at devfs_write_f+0xb1/frame 0xffffff887125ea20 dofilewrite() at dofilewrite+0x85/frame 0xffffff887125ea70 kern_pwritev() at kern_pwritev+0x68/frame 0xffffff887125eac0 sys_pwrite() at sys_pwrite+0x68/frame 0xffffff887125eb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887125ec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887125ec30 --- syscall (476, FreeBSD ELF64, sys_pwrite), rip =3D 0x80175594c, rsp =3D = 0x7fffffffcbd8, rbp =3D 0x29f8800000 --- Tracing command csh pid 1276 tid 100111 td 0xfffffe000dec9000 sched_switch() at sched_switch+0x194/frame 0xffffff8871240900 mi_switch() at mi_switch+0x208/frame 0xffffff8871240950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871240980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712409= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871240a00 _sleep() at _sleep+0x368/frame 0xffffff8871240a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871240ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871240b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871240c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871240c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffcea8, rbp =3D 0x80140b600 --- Tracing command su pid 1275 tid 100129 td 0xfffffe000df9f490 sched_switch() at sched_switch+0x194/frame 0xffffff887129a6d0 mi_switch() at mi_switch+0x208/frame 0xffffff887129a720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887129a750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887129a7= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887129a7d0 _sleep() at _sleep+0x368/frame 0xffffff887129a860 kern_wait6() at kern_wait6+0x5e2/frame 0xffffff887129a8f0 kern_wait() at kern_wait+0x9c/frame 0xffffff887129aa50 sys_wait4() at sys_wait4+0x35/frame 0xffffff887129ab10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887129ac30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887129ac30 --- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x800ed34ac, rsp =3D 0x7= fffffffd578, rbp =3D 0x4fc --- Tracing command csh pid 1272 tid 100166 td 0xfffffe000dfca490 sched_switch() at sched_switch+0x194/frame 0xffffff8871353900 mi_switch() at mi_switch+0x208/frame 0xffffff8871353950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871353980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713539= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871353a00 _sleep() at _sleep+0x368/frame 0xffffff8871353a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff8871353ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff8871353b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871353c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871353c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffcfa8, rbp =3D 0x80141c900 --- Tracing command sshd pid 1271 tid 100116 td 0xfffffe000df69490 sched_switch() at sched_switch+0x194/frame 0xffffff8871259710 mi_switch() at mi_switch+0x208/frame 0xffffff8871259760 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871259790 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712597= f0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871259810 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff8871259870 seltdwait() at seltdwait+0xac/frame 0xffffff88712598a0 kern_select() at kern_select+0x6f0/frame 0xffffff8871259ac0 sys_select() at sys_select+0x5d/frame 0xffffff8871259b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871259c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871259c30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x8024f8a9c, rsp =3D 0= x7fffffffcc68, rbp =3D 0x7fffffffccf0 --- Tracing command sshd pid 1269 tid 100187 td 0xfffffe000daec000 sched_switch() at sched_switch+0x194/frame 0xffffff88713bc7c0 mi_switch() at mi_switch+0x208/frame 0xffffff88713bc810 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713bc840 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713bc8= a0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713bc8c0 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88713bc920 seltdwait() at seltdwait+0xac/frame 0xffffff88713bc950 sys_poll() at sys_poll+0x421/frame 0xffffff88713bcb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713bcc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713bcc30 --- syscall (209, FreeBSD ELF64, sys_poll), rip =3D 0x802492b3c, rsp =3D 0x= 7fffffffcd08, rbp =3D 0x803017190 --- Tracing command csh pid 1267 tid 100185 td 0xfffffe000daec490 sched_switch() at sched_switch+0x194/frame 0xffffff88713b2900 mi_switch() at mi_switch+0x208/frame 0xffffff88713b2950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713b2980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713b29= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713b2a00 _sleep() at _sleep+0x368/frame 0xffffff88713b2a90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff88713b2ae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff88713b2b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88713b2c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88713b2c30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x800d4299c, rsp =3D 0x7= fffffffd048, rbp =3D 0x801460400 --- Tracing command login pid 1261 tid 100179 td 0xfffffe000daec920 sched_switch() at sched_switch+0x194/frame 0xffffff88713946d0 mi_switch() at mi_switch+0x208/frame 0xffffff8871394720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871394750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713947= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88713947d0 _sleep() at _sleep+0x368/frame 0xffffff8871394860 kern_wait6() at kern_wait6+0x5e2/frame 0xffffff88713948f0 kern_wait() at kern_wait+0x9c/frame 0xffffff8871394a50 sys_wait4() at sys_wait4+0x35/frame 0xffffff8871394b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871394c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871394c30 --- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x800ed54ac, rsp =3D 0x7= fffffffdc08, rbp =3D 0 --- Tracing command getty pid 1260 tid 100178 td 0xfffffe000daed000 sched_switch() at sched_switch+0x194/frame 0xffffff887138f770 mi_switch() at mi_switch+0x208/frame 0xffffff887138f7c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887138f7f0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887138f8= 50 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887138f870 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff887138f8d0 tty_wait() at tty_wait+0x48/frame 0xffffff887138f900 ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff887138f990 ttydev_read() at ttydev_read+0x8e/frame 0xffffff887138f9c0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff887138fa30 dofileread() at dofileread+0xa1/frame 0xffffff887138fa80 kern_readv() at kern_readv+0x6c/frame 0xffffff887138fac0 sys_read() at sys_read+0x64/frame 0xffffff887138fb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887138fc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887138fc30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f= ffffffdca8, rbp =3D 0 --- Tracing command getty pid 1259 tid 100177 td 0xfffffe000daed490 sched_switch() at sched_switch+0x194/frame 0xffffff887138a770 mi_switch() at mi_switch+0x208/frame 0xffffff887138a7c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887138a7f0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887138a8= 50 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887138a870 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff887138a8d0 tty_wait() at tty_wait+0x48/frame 0xffffff887138a900 ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff887138a990 ttydev_read() at ttydev_read+0x8e/frame 0xffffff887138a9c0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff887138aa30 dofileread() at dofileread+0xa1/frame 0xffffff887138aa80 kern_readv() at kern_readv+0x6c/frame 0xffffff887138aac0 sys_read() at sys_read+0x64/frame 0xffffff887138ab10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887138ac30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887138ac30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f= ffffffdca8, rbp =3D 0 --- Tracing command getty pid 1258 tid 100176 td 0xfffffe000daed920 sched_switch() at sched_switch+0x194/frame 0xffffff8871385770 mi_switch() at mi_switch+0x208/frame 0xffffff88713857c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88713857f0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713858= 50 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871385870 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88713858d0 tty_wait() at tty_wait+0x48/frame 0xffffff8871385900 ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff8871385990 ttydev_read() at ttydev_read+0x8e/frame 0xffffff88713859c0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff8871385a30 dofileread() at dofileread+0xa1/frame 0xffffff8871385a80 kern_readv() at kern_readv+0x6c/frame 0xffffff8871385ac0 sys_read() at sys_read+0x64/frame 0xffffff8871385b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871385c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871385c30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f= ffffffdca8, rbp =3D 0 --- Tracing command getty pid 1257 tid 100134 td 0xfffffe000de80490 sched_switch() at sched_switch+0x194/frame 0xffffff88712b3770 mi_switch() at mi_switch+0x208/frame 0xffffff88712b37c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712b37f0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712b38= 50 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712b3870 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88712b38d0 tty_wait() at tty_wait+0x48/frame 0xffffff88712b3900 ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff88712b3990 ttydev_read() at ttydev_read+0x8e/frame 0xffffff88712b39c0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff88712b3a30 dofileread() at dofileread+0xa1/frame 0xffffff88712b3a80 kern_readv() at kern_readv+0x6c/frame 0xffffff88712b3ac0 sys_read() at sys_read+0x64/frame 0xffffff88712b3b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712b3c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712b3c30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f= ffffffdca8, rbp =3D 0 --- Tracing command getty pid 1256 tid 100107 td 0xfffffe000df1f000 sched_switch() at sched_switch+0x194/frame 0xffffff887122c770 mi_switch() at mi_switch+0x208/frame 0xffffff887122c7c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887122c7f0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887122c8= 50 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff887122c870 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff887122c8d0 tty_wait() at tty_wait+0x48/frame 0xffffff887122c900 ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff887122c990 ttydev_read() at ttydev_read+0x8e/frame 0xffffff887122c9c0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff887122ca30 dofileread() at dofileread+0xa1/frame 0xffffff887122ca80 kern_readv() at kern_readv+0x6c/frame 0xffffff887122cac0 sys_read() at sys_read+0x64/frame 0xffffff887122cb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887122cc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887122cc30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f= ffffffdca8, rbp =3D 0 --- Tracing command getty pid 1255 tid 100128 td 0xfffffe000daee490 sched_switch() at sched_switch+0x194/frame 0xffffff8871295770 mi_switch() at mi_switch+0x208/frame 0xffffff88712957c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712957f0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712958= 50 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871295870 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88712958d0 tty_wait() at tty_wait+0x48/frame 0xffffff8871295900 ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff8871295990 ttydev_read() at ttydev_read+0x8e/frame 0xffffff88712959c0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff8871295a30 dofileread() at dofileread+0xa1/frame 0xffffff8871295a80 kern_readv() at kern_readv+0x6c/frame 0xffffff8871295ac0 sys_read() at sys_read+0x64/frame 0xffffff8871295b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871295c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871295c30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f= ffffffdca8, rbp =3D 0 --- Tracing command getty pid 1254 tid 100076 td 0xfffffe000daef490 sched_switch() at sched_switch+0x194/frame 0xffffff8871191770 mi_switch() at mi_switch+0x208/frame 0xffffff88711917c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711917f0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711918= 50 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff8871191870 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88711918d0 tty_wait() at tty_wait+0x48/frame 0xffffff8871191900 ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff8871191990 ttydev_read() at ttydev_read+0x8e/frame 0xffffff88711919c0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff8871191a30 dofileread() at dofileread+0xa1/frame 0xffffff8871191a80 kern_readv() at kern_readv+0x6c/frame 0xffffff8871191ac0 sys_read() at sys_read+0x64/frame 0xffffff8871191b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871191c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871191c30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f= ffffffdca8, rbp =3D 0 --- Tracing command getty pid 1253 tid 100084 td 0xfffffe000decb000 sched_switch() at sched_switch+0x194/frame 0xffffff88711b9770 mi_switch() at mi_switch+0x208/frame 0xffffff88711b97c0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711b97f0 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711b98= 50 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711b9870 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88711b98d0 tty_wait() at tty_wait+0x48/frame 0xffffff88711b9900 ttydisc_read() at ttydisc_read+0x33c/frame 0xffffff88711b9990 ttydev_read() at ttydev_read+0x8e/frame 0xffffff88711b99c0 devfs_read_f() at devfs_read_f+0x90/frame 0xffffff88711b9a30 dofileread() at dofileread+0xa1/frame 0xffffff88711b9a80 kern_readv() at kern_readv+0x6c/frame 0xffffff88711b9ac0 sys_read() at sys_read+0x64/frame 0xffffff88711b9b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711b9c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711b9c30 --- syscall (3, FreeBSD ELF64, sys_read), rip =3D 0x800b3bb1c, rsp =3D 0x7f= ffffffdca8, rbp =3D 0 --- Tracing command cron pid 1219 tid 100167 td 0xfffffe000dec4920 sched_switch() at sched_switch+0x194/frame 0xffffff88713588a0 mi_switch() at mi_switch+0x208/frame 0xffffff88713588f0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871358920 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88713589= 80 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88713589b0 _sleep() at _sleep+0x3ae/frame 0xffffff8871358a40 kern_nanosleep() at kern_nanosleep+0x118/frame 0xffffff8871358ab0 sys_nanosleep() at sys_nanosleep+0x6e/frame 0xffffff8871358b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871358c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871358c30 --- syscall (240, FreeBSD ELF64, sys_nanosleep), rip =3D 0x800d2be2c, rsp = =3D 0x7fffffffdae8, rbp =3D 0x3c --- Tracing command sshd pid 1204 tid 100090 td 0xfffffe000deca920 sched_switch() at sched_switch+0x194/frame 0xffffff88711d7710 mi_switch() at mi_switch+0x208/frame 0xffffff88711d7760 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711d7790 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711d77= f0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711d7810 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88711d7870 seltdwait() at seltdwait+0xac/frame 0xffffff88711d78a0 kern_select() at kern_select+0x6f0/frame 0xffffff88711d7ac0 sys_select() at sys_select+0x5d/frame 0xffffff88711d7b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711d7c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711d7c30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x8024f8a9c, rsp =3D 0= x7fffffffcd88, rbp =3D 0x2 --- =20 Tracing command exim-4.80.1-2 pid 1198 tid 100135 td 0xfffffe000de6f000 sched_switch() at sched_switch+0x194/frame 0xffffff88712b8710 mi_switch() at mi_switch+0x208/frame 0xffffff88712b8760 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712b8790 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712b87= f0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712b8810 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88712b8870 seltdwait() at seltdwait+0xac/frame 0xffffff88712b88a0 kern_select() at kern_select+0x6f0/frame 0xffffff88712b8ac0 sys_select() at sys_select+0x5d/frame 0xffffff88712b8b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712b8c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712b8c30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x80234ba9c, rsp =3D 0= x7fffffffc6b8, rbp =3D 0x5 --- Tracing command nrpe2 pid 1185 tid 100102 td 0xfffffe000ddbc490 sched_switch() at sched_switch+0x194/frame 0xffffff8871213700 mi_switch() at mi_switch+0x208/frame 0xffffff8871213750 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8871213780 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712137= e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff8871213810 _cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff8871213870 seltdwait() at seltdwait+0x56/frame 0xffffff88712138a0 kern_select() at kern_select+0x6f0/frame 0xffffff8871213ac0 sys_select() at sys_select+0x5d/frame 0xffffff8871213b10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff8871213c30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff8871213c30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x800b34a9c, rsp =3D 0= x7fffffffd038, rbp =3D 0x7fffffffdd10 --- Tracing command hastd pid 1124 tid 100133 td 0xfffffe000de6f490 sched_switch() at sched_switch+0x194/frame 0xffffff88712ae700 mi_switch() at mi_switch+0x208/frame 0xffffff88712ae750 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712ae780 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712ae7= e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff88712ae810 _cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff88712ae870 seltdwait() at seltdwait+0x56/frame 0xffffff88712ae8a0 kern_select() at kern_select+0x6f0/frame 0xffffff88712aeac0 sys_select() at sys_select+0x5d/frame 0xffffff88712aeb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712aec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712aec30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x801756a9c, rsp =3D 0= x7fffffffd898, rbp =3D 0x12 --- Tracing command syslogd pid 1093 tid 100094 td 0xfffffe000df3a920 sched_switch() at sched_switch+0x194/frame 0xffffff88711eb710 mi_switch() at mi_switch+0x208/frame 0xffffff88711eb760 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88711eb790 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88711eb7= f0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88711eb810 _cv_wait_sig() at _cv_wait_sig+0x181/frame 0xffffff88711eb870 seltdwait() at seltdwait+0xac/frame 0xffffff88711eb8a0 kern_select() at kern_select+0x6f0/frame 0xffffff88711ebac0 sys_select() at sys_select+0x5d/frame 0xffffff88711ebb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88711ebc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88711ebc30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x800b3ea9c, rsp =3D 0= x7fffffffd128, rbp =3D 0x8010a00c8 --- Tracing command devd pid 930 tid 100155 td 0xfffffe000df89000 sched_switch() at sched_switch+0x194/frame 0xffffff887131c700 mi_switch() at mi_switch+0x208/frame 0xffffff887131c750 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887131c780 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff887131c7= e0 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19/frame 0xffffff887131c810 _cv_timedwait_sig() at _cv_timedwait_sig+0x18f/frame 0xffffff887131c870 seltdwait() at seltdwait+0x56/frame 0xffffff887131c8a0 kern_select() at kern_select+0x6f0/frame 0xffffff887131cac0 sys_select() at sys_select+0x5d/frame 0xffffff887131cb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff887131cc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff887131cc30 --- syscall (93, FreeBSD ELF64, sys_select), rip =3D 0x447f1c, rsp =3D 0x7f= ffffffbc38, rbp =3D 0x10 --- Tracing command adjkerntz pid 135 tid 100149 td 0xfffffe000de6e000 sched_switch() at sched_switch+0x194/frame 0xffffff88712fe900 mi_switch() at mi_switch+0x208/frame 0xffffff88712fe950 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff88712fe980 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff88712fe9= e0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff88712fea00 _sleep() at _sleep+0x368/frame 0xffffff88712fea90 kern_sigsuspend() at kern_sigsuspend+0x9b/frame 0xffffff88712feae0 sys_sigsuspend() at sys_sigsuspend+0x34/frame 0xffffff88712feb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff88712fec30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff88712fec30 --- syscall (4, FreeBSD ELF64, sys_write), rip =3D 0x80087e99c, rsp =3D 0x7= fffffffdc28, rbp =3D 0x1 --- Tracing command g_mirror gmrootfs pid 18 tid 100075 td 0xfffffe000ddbd920 sched_switch() at sched_switch+0x194/frame 0xffffff887118c9d0 mi_switch() at mi_switch+0x208/frame 0xffffff887118ca20 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff887118ca50 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff887118ca80 _sleep() at _sleep+0x3d4/frame 0xffffff887118cb10 g_mirror_worker() at g_mirror_worker+0xb4d/frame 0xffffff887118cbe0 fork_exit() at fork_exit+0x135/frame 0xffffff887118cc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff887118cc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff887118ccf0, rbp =3D 0 --- Tracing command softdepflush pid 17 tid 100074 td 0xfffffe000daef920 sched_switch() at sched_switch+0x194/frame 0xffffff884afbba60 mi_switch() at mi_switch+0x208/frame 0xffffff884afbbab0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afbbae0 sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884afbbb10 _sleep() at _sleep+0x26d/frame 0xffffff884afbbba0 softdep_flush() at softdep_flush+0x26a/frame 0xffffff884afbbbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884afbbc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afbbc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884afbbcf0, rbp =3D 0 --- Tracing command syncer pid 16 tid 100073 td 0xfffffe000daf2000 sched_switch() at sched_switch+0x194/frame 0xffffff884afb6800 mi_switch() at mi_switch+0x208/frame 0xffffff884afb6850 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afb6880 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884afb68b0 _sleep() at _sleep+0x3d4/frame 0xffffff884afb6940 getpbuf() at getpbuf+0x4e/frame 0xffffff884afb6960 cluster_wbuild_gb() at cluster_wbuild_gb+0x21e/frame 0xffffff884afb6a10 ffs_syncvnode() at ffs_syncvnode+0x2fa/frame 0xffffff884afb6a80 ffs_fsync() at ffs_fsync+0x45/frame 0xffffff884afb6ac0 VOP_FSYNC_APV() at VOP_FSYNC_APV+0x105/frame 0xffffff884afb6af0 sync_vnode() at sync_vnode+0x157/frame 0xffffff884afb6b70 sched_sync() at sched_sync+0x1c7/frame 0xffffff884afb6be0 fork_exit() at fork_exit+0x135/frame 0xffffff884afb6c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afb6c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884afb6cf0, rbp =3D 0 --- Tracing command vnlru pid 9 tid 100072 td 0xfffffe000d98f000 sched_switch() at sched_switch+0x194/frame 0xffffff884afb1a10 mi_switch() at mi_switch+0x208/frame 0xffffff884afb1a60 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afb1a90 sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884afb1ac0 _sleep() at _sleep+0x26d/frame 0xffffff884afb1b50 vnlru_proc() at vnlru_proc+0x58a/frame 0xffffff884afb1be0 fork_exit() at fork_exit+0x135/frame 0xffffff884afb1c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afb1c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884afb1cf0, rbp =3D 0 --- Tracing command bufdaemon pid 8 tid 100071 td 0xfffffe000d98f490 sched_switch() at sched_switch+0x194/frame 0xffffff884afac8f0 mi_switch() at mi_switch+0x208/frame 0xffffff884afac940 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afac970 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884afac9a0 _sleep() at _sleep+0x3d4/frame 0xffffff884afaca30 getpbuf() at getpbuf+0x4e/frame 0xffffff884afaca50 cluster_wbuild_gb() at cluster_wbuild_gb+0x21e/frame 0xffffff884afacb00 flushbufqueues() at flushbufqueues+0x72e/frame 0xffffff884afacba0 buf_do_flush() at buf_do_flush+0x16/frame 0xffffff884afacbc0 buf_daemon() at buf_daemon+0xe4/frame 0xffffff884afacbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884afacc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afacc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884afaccf0, rbp =3D 0 --- Tracing command pagezero pid 7 tid 100070 td 0xfffffe000d98f920 sched_switch() at sched_switch+0x194/frame 0xffffff884afa7a80 mi_switch() at mi_switch+0x208/frame 0xffffff884afa7ad0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afa7b00 sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884afa7b30 _sleep() at _sleep+0x26d/frame 0xffffff884afa7bc0 vm_pagezero() at vm_pagezero+0x73/frame 0xffffff884afa7be0 fork_exit() at fork_exit+0x135/frame 0xffffff884afa7c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afa7c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884afa7cf0, rbp =3D 0 --- Tracing command vmdaemon pid 6 tid 100069 td 0xfffffe000d991000 sched_switch() at sched_switch+0x194/frame 0xffffff884afa2a40 mi_switch() at mi_switch+0x208/frame 0xffffff884afa2a90 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884afa2ac0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884afa2af0 _sleep() at _sleep+0x3d4/frame 0xffffff884afa2b80 vm_daemon() at vm_daemon+0x4d/frame 0xffffff884afa2be0 fork_exit() at fork_exit+0x135/frame 0xffffff884afa2c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884afa2c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884afa2cf0, rbp =3D 0 --- Tracing command pagedaemon pid 5 tid 100068 td 0xfffffe000d991490 sched_switch() at sched_switch+0x194/frame 0xffffff884af9d980 mi_switch() at mi_switch+0x208/frame 0xffffff884af9d9d0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af9da00 sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884af9da30 _sleep() at _sleep+0x26d/frame 0xffffff884af9dac0 vm_pageout() at vm_pageout+0x269/frame 0xffffff884af9dbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884af9dc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af9dc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884af9dcf0, rbp =3D 0 --- Tracing command xpt_thrd pid 4 tid 100067 td 0xfffffe000d991920 sched_switch() at sched_switch+0x194/frame 0xffffff884af98a80 mi_switch() at mi_switch+0x208/frame 0xffffff884af98ad0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af98b00 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884af98b30 _sleep() at _sleep+0x3d4/frame 0xffffff884af98bc0 xpt_scanner_thread() at xpt_scanner_thread+0xd5/frame 0xffffff884af98be0 fork_exit() at fork_exit+0x135/frame 0xffffff884af98c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af98c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884af98cf0, rbp =3D 0 --- Tracing command sctp_iterator pid 3 tid 100066 td 0xfffffe000d993000 sched_switch() at sched_switch+0x194/frame 0xffffff884af93a90 mi_switch() at mi_switch+0x208/frame 0xffffff884af93ae0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af93b10 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884af93b40 _sleep() at _sleep+0x3d4/frame 0xffffff884af93bd0 sctp_iterator_thread() at sctp_iterator_thread+0x3f/frame 0xffffff884af93be0 fork_exit() at fork_exit+0x135/frame 0xffffff884af93c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af93c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884af93cf0, rbp =3D 0 --- Tracing command fdc0 pid 2 tid 100063 td 0xfffffe000d994000 sched_switch() at sched_switch+0x194/frame 0xffffff884af7e9f0 mi_switch() at mi_switch+0x208/frame 0xffffff884af7ea40 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af7ea70 sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff884af7eaa0 _sleep() at _sleep+0x26d/frame 0xffffff884af7eb30 fdc_thread() at fdc_thread+0x74a/frame 0xffffff884af7ebe0 fork_exit() at fork_exit+0x135/frame 0xffffff884af7ec30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af7ec30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884af7ecf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100059 td 0xfffffe000a532920 sched_switch() at sched_switch+0x194/frame 0xffffff884aee2aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aee2af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aee2b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aee2b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aee2bb0 usb_process() at usb_process+0x16b/frame 0xffffff884aee2be0 fork_exit() at fork_exit+0x135/frame 0xffffff884aee2c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aee2c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aee2cf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100058 td 0xfffffe000d700000 sched_switch() at sched_switch+0x194/frame 0xffffff884aeddaa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aeddaf0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aeddb20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aeddb50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aeddbb0 usb_process() at usb_process+0x16b/frame 0xffffff884aeddbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884aeddc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aeddc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aeddcf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100057 td 0xfffffe000d700490 sched_switch() at sched_switch+0x194/frame 0xffffff884aed8aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aed8af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aed8b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aed8b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aed8bb0 usb_process() at usb_process+0x16b/frame 0xffffff884aed8be0 fork_exit() at fork_exit+0x135/frame 0xffffff884aed8c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aed8c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aed8cf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100056 td 0xfffffe000d700920 sched_switch() at sched_switch+0x194/frame 0xffffff884aed3aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aed3af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aed3b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aed3b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aed3bb0 usb_process() at usb_process+0x16b/frame 0xffffff884aed3be0 fork_exit() at fork_exit+0x135/frame 0xffffff884aed3c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aed3c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aed3cf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100055 td 0xfffffe000d701000 sched_switch() at sched_switch+0x194/frame 0xffffff884ad4baa0 mi_switch() at mi_switch+0x208/frame 0xffffff884ad4baf0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ad4bb20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ad4bb50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884ad4bbb0 usb_process() at usb_process+0x16b/frame 0xffffff884ad4bbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884ad4bc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ad4bc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884ad4bcf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100054 td 0xfffffe000d701490 sched_switch() at sched_switch+0x194/frame 0xffffff884ad46aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884ad46af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ad46b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ad46b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884ad46bb0 usb_process() at usb_process+0x16b/frame 0xffffff884ad46be0 fork_exit() at fork_exit+0x135/frame 0xffffff884ad46c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ad46c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884ad46cf0, rbp =3D 0 --- =20 Tracing command usb pid 15 tid 100053 td 0xfffffe000d701920 sched_switch() at sched_switch+0x194/frame 0xffffff884ad41aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884ad41af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ad41b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ad41b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884ad41bb0 usb_process() at usb_process+0x16b/frame 0xffffff884ad41be0 fork_exit() at fork_exit+0x135/frame 0xffffff884ad41c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ad41c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884ad41cf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100052 td 0xfffffe000d702000 sched_switch() at sched_switch+0x194/frame 0xffffff884ad3caa0 mi_switch() at mi_switch+0x208/frame 0xffffff884ad3caf0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ad3cb20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ad3cb50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884ad3cbb0 usb_process() at usb_process+0x16b/frame 0xffffff884ad3cbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884ad3cc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ad3cc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884ad3ccf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100050 td 0xfffffe000d702920 sched_switch() at sched_switch+0x194/frame 0xffffff884abacaa0 mi_switch() at mi_switch+0x208/frame 0xffffff884abacaf0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884abacb20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884abacb50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884abacbb0 usb_process() at usb_process+0x16b/frame 0xffffff884abacbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884abacc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884abacc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884abaccf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100049 td 0xfffffe000a52e000 sched_switch() at sched_switch+0x194/frame 0xffffff884aba7aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aba7af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aba7b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aba7b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aba7bb0 usb_process() at usb_process+0x16b/frame 0xffffff884aba7be0 fork_exit() at fork_exit+0x135/frame 0xffffff884aba7c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aba7c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aba7cf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100048 td 0xfffffe000a52e490 sched_switch() at sched_switch+0x194/frame 0xffffff884aba2aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aba2af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aba2b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aba2b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aba2bb0 usb_process() at usb_process+0x16b/frame 0xffffff884aba2be0 fork_exit() at fork_exit+0x135/frame 0xffffff884aba2c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aba2c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aba2cf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100047 td 0xfffffe000a52e920 sched_switch() at sched_switch+0x194/frame 0xffffff884ab9daa0 mi_switch() at mi_switch+0x208/frame 0xffffff884ab9daf0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884ab9db20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884ab9db50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884ab9dbb0 usb_process() at usb_process+0x16b/frame 0xffffff884ab9dbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884ab9dc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ab9dc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884ab9dcf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100045 td 0xfffffe000a52f490 sched_switch() at sched_switch+0x194/frame 0xffffff884aa0daa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aa0daf0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aa0db20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aa0db50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aa0dbb0 usb_process() at usb_process+0x16b/frame 0xffffff884aa0dbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884aa0dc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aa0dc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aa0dcf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100044 td 0xfffffe000a52f920 sched_switch() at sched_switch+0x194/frame 0xffffff884aa08aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aa08af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aa08b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aa08b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aa08bb0 usb_process() at usb_process+0x16b/frame 0xffffff884aa08be0 fork_exit() at fork_exit+0x135/frame 0xffffff884aa08c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aa08c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aa08cf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100043 td 0xfffffe000a531000 sched_switch() at sched_switch+0x194/frame 0xffffff884aa03aa0 mi_switch() at mi_switch+0x208/frame 0xffffff884aa03af0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884aa03b20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884aa03b50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884aa03bb0 usb_process() at usb_process+0x16b/frame 0xffffff884aa03be0 fork_exit() at fork_exit+0x135/frame 0xffffff884aa03c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884aa03c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884aa03cf0, rbp =3D 0 --- Tracing command usb pid 15 tid 100042 td 0xfffffe000a531490 sched_switch() at sched_switch+0x194/frame 0xffffff884a9feaa0 mi_switch() at mi_switch+0x208/frame 0xffffff884a9feaf0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884a9feb20 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884a9feb50 _cv_wait() at _cv_wait+0x17d/frame 0xffffff884a9febb0 usb_process() at usb_process+0x16b/frame 0xffffff884a9febe0 fork_exit() at fork_exit+0x135/frame 0xffffff884a9fec30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884a9fec30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884a9fecf0, rbp =3D 0 --- Tracing command yarrow pid 14 tid 100025 td 0xfffffe000a2dd000 sched_switch() at sched_switch+0x194/frame 0xffffff8000300a60 mi_switch() at mi_switch+0x208/frame 0xffffff8000300ab0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000300ae0 sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffff8000300b10 _sleep() at _sleep+0x26d/frame 0xffffff8000300ba0 random_kthread() at random_kthread+0x1ad/frame 0xffffff8000300be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000300c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000300c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000300cf0, rbp =3D 0 --- Tracing command geom pid 13 tid 100023 td 0xfffffe000a2dd920 sched_switch() at sched_switch+0x194/frame 0xffffff80002f5a30 mi_switch() at mi_switch+0x208/frame 0xffffff80002f5a80 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff80002f5ab0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff80002f5ae0 _sleep() at _sleep+0x3d4/frame 0xffffff80002f5b70 g_io_schedule_down() at g_io_schedule_down+0x2bc/frame 0xffffff80002f5bd0 g_down_procbody() at g_down_procbody+0x72/frame 0xffffff80002f5be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002f5c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002f5c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002f5cf0, rbp =3D 0 --- Tracing command geom pid 13 tid 100022 td 0xfffffe000a2d3000 cpustop_handler() at cpustop_handler+0x28/frame 0xffffff8000264d00 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff8000264d20 trap() at trap+0x355/frame 0xffffff8000264f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff8000264f20 --- trap 0x13, rip =3D 0xffffffff8092c60f, rsp =3D 0xffffff8000264fe0, rbp = =3D 0xffffff80002f0850 --- cpu_search_lowest() at cpu_search_lowest+0x8f/frame 0xffffff80002f0850 cpu_search_lowest() at cpu_search_lowest+0x18e/frame 0xffffff80002f08e0 sched_pickcpu() at sched_pickcpu+0x2b4/frame 0xffffff80002f0950 sched_add() at sched_add+0x65/frame 0xffffff80002f0990 intr_event_schedule_thread() at intr_event_schedule_thread+0xac/frame 0xfff= fff80002f09d0 intr_event_handle() at intr_event_handle+0xe8/frame 0xffffff80002f0a10 intr_execute_handlers() at intr_execute_handlers+0x5f/frame 0xffffff80002f0= a40 lapic_handle_intr() at lapic_handle_intr+0x32/frame 0xffffff80002f0a70 Xapic_isr1() at Xapic_isr1+0xa0/frame 0xffffff80002f0a70 --- interrupt, rip =3D 0xffffffff80cda633, rsp =3D 0xffffff80002f0b30, rbp = =3D 0xffffff80002f0b50 --- spinlock_exit() at spinlock_exit+0x33/frame 0xffffff80002f0b50 pmap_invalidate_range() at pmap_invalidate_range+0xc9/frame 0xffffff80002f0= b70 biodone() at biodone+0xaa/frame 0xffffff80002f0bb0 g_io_schedule_up() at g_io_schedule_up+0xcd/frame 0xffffff80002f0bd0 g_up_procbody() at g_up_procbody+0x72/frame 0xffffff80002f0be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002f0c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002f0c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002f0cf0, rbp =3D 0 --- Tracing command geom pid 13 tid 100021 td 0xfffffe000a2d3490 sched_switch() at sched_switch+0x194/frame 0xffffff80002eba70 mi_switch() at mi_switch+0x208/frame 0xffffff80002ebac0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff80002ebaf0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff80002ebb20 _sleep() at _sleep+0x3d4/frame 0xffffff80002ebbb0 g_run_events() at g_run_events+0x4c1/frame 0xffffff80002ebbe0 fork_exit() at fork_exit+0x135/frame 0xffffff80002ebc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002ebc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002ebcf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100064 td 0xfffffe000d993920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100062 td 0xfffffe000d994490 sched_switch() at sched_switch+0x194/frame 0xffffff884af6fb40 mi_switch() at mi_switch+0x208/frame 0xffffff884af6fb90 ithread_loop() at ithread_loop+0x243/frame 0xffffff884af6fbe0 fork_exit() at fork_exit+0x135/frame 0xffffff884af6fc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af6fc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884af6fcf0, rbp =3D 0 --- =20 Tracing command intr pid 12 tid 100061 td 0xfffffe000d994920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100060 td 0xfffffe000a532490 sched_switch() at sched_switch+0x194/frame 0xffffff884af65b40 mi_switch() at mi_switch+0x208/frame 0xffffff884af65b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff884af65be0 fork_exit() at fork_exit+0x135/frame 0xffffff884af65c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af65c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884af65cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100051 td 0xfffffe000d702490 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100046 td 0xfffffe000a52f000 sched_switch() at sched_switch+0x194/frame 0xffffff884ab98b40 mi_switch() at mi_switch+0x208/frame 0xffffff884ab98b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff884ab98be0 fork_exit() at fork_exit+0x135/frame 0xffffff884ab98c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884ab98c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884ab98cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100041 td 0xfffffe000a531920 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100040 td 0xfffffe000a532000 sched_switch() at sched_switch+0x194/frame 0xffffff800035bb40 mi_switch() at mi_switch+0x208/frame 0xffffff800035bb90 ithread_loop() at ithread_loop+0x243/frame 0xffffff800035bbe0 fork_exit() at fork_exit+0x135/frame 0xffffff800035bc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800035bc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff800035bcf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100032 td 0xfffffe000a4a3490 sched_switch() at sched_switch+0x194/frame 0xffffff8000323b40 mi_switch() at mi_switch+0x208/frame 0xffffff8000323b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff8000323be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000323c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000323c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000323cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100029 td 0xfffffe000a2d7920 sched_switch() at sched_switch+0x194/frame 0xffffff8000314b40 mi_switch() at mi_switch+0x208/frame 0xffffff8000314b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff8000314be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000314c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000314c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000314cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100028 td 0xfffffe000a2dc000 sched_switch() at sched_switch+0x194/frame 0xffffff800030fb40 mi_switch() at mi_switch+0x208/frame 0xffffff800030fb90 ithread_loop() at ithread_loop+0x243/frame 0xffffff800030fbe0 fork_exit() at fork_exit+0x135/frame 0xffffff800030fc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800030fc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff800030fcf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100026 td 0xfffffe000a2dc920 sched_switch() at sched_switch+0x194/frame 0xffffff8000305b40 mi_switch() at mi_switch+0x208/frame 0xffffff8000305b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff8000305be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000305c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000305c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000305cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100020 td 0xfffffe000a2d3920 sched_switch() at sched_switch+0x194/frame 0xffffff80002e6b40 mi_switch() at mi_switch+0x208/frame 0xffffff80002e6b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff80002e6be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002e6c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002e6c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002e6cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100019 td 0xfffffe000a2d4000 sched_switch() at sched_switch+0x194/frame 0xffffff80002e1b40 mi_switch() at mi_switch+0x208/frame 0xffffff80002e1b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff80002e1be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002e1c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002e1c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002e1cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100018 td 0xfffffe000a2d4490 sched_switch() at sched_switch+0x194/frame 0xffffff80002dcb40 mi_switch() at mi_switch+0x208/frame 0xffffff80002dcb90 ithread_loop() at ithread_loop+0x243/frame 0xffffff80002dcbe0 fork_exit() at fork_exit+0x135/frame 0xffffff80002dcc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002dcc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002dccf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100017 td 0xfffffe000a2d4920 sched_switch() at sched_switch+0x194/frame 0xffffff80002d7b40 mi_switch() at mi_switch+0x208/frame 0xffffff80002d7b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff80002d7be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002d7c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002d7c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002d7cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100016 td 0xfffffe000a2d7000 sched_switch() at sched_switch+0x194/frame 0xffffff80002d2b40 mi_switch() at mi_switch+0x208/frame 0xffffff80002d2b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff80002d2be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002d2c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002d2c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002d2cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100015 td 0xfffffe000a2c3000 sched_switch() at sched_switch+0x194/frame 0xffffff80002cdb40 mi_switch() at mi_switch+0x208/frame 0xffffff80002cdb90 ithread_loop() at ithread_loop+0x243/frame 0xffffff80002cdbe0 fork_exit() at fork_exit+0x135/frame 0xffffff80002cdc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002cdc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002cdcf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100014 td 0xfffffe000a2c3490 sched_switch() at sched_switch+0x194/frame 0xffffff80002c8b40 mi_switch() at mi_switch+0x208/frame 0xffffff80002c8b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff80002c8be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002c8c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002c8c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002c8cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100013 td 0xfffffe000a2c3920 sched_switch() at sched_switch+0x194/frame 0xffffff80002c3b40 mi_switch() at mi_switch+0x208/frame 0xffffff80002c3b90 ithread_loop() at ithread_loop+0x243/frame 0xffffff80002c3be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002c3c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002c3c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002c3cf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100012 td 0xfffffe000a2c4000 cpustop_handler() at cpustop_handler+0x28/frame 0xffffffff816105e0 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffffff81610600 trap() at trap+0x355/frame 0xffffffff81610800 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffffff81610800 --- trap 0x13, rip =3D 0xffffffff808f019c, rsp =3D 0xffffffff816108c0, rbp = =3D 0xffffff80002bea70 --- _mtx_lock_sleep() at _mtx_lock_sleep+0x16c/frame 0xffffff80002bea70 _mtx_lock_flags() at _mtx_lock_flags+0x106/frame 0xffffff80002beaa0 softclock() at softclock+0x28f/frame 0xffffff80002beb60 intr_event_execute_handlers() at intr_event_execute_handlers+0x6a/frame 0xf= fffff80002beb90 ithread_loop() at ithread_loop+0xac/frame 0xffffff80002bebe0 fork_exit() at fork_exit+0x135/frame 0xffffff80002bec30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002bec30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002becf0, rbp =3D 0 --- Tracing command intr pid 12 tid 100011 td 0xfffffe000a2c4490 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100010 td 0xfffffe000a2c4920 sched_switch() at sched_switch+0x194/frame 0xffffff80002b4ae0 mi_switch() at mi_switch+0x208/frame 0xffffff80002b4b30 critical_exit() at critical_exit+0xb0/frame 0xffffff80002b4b50 sched_idletd() at sched_idletd+0x11e/frame 0xffffff80002b4be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002b4c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002b4c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002b4cf0, rbp =3D 0 --- Tracing command idle pid 11 tid 100009 td 0xfffffe000a2b3490 sched_switch() at sched_switch+0x194/frame 0xffffff80002afb00 mi_switch() at mi_switch+0x208/frame 0xffffff80002afb50 sched_idletd() at sched_idletd+0x17a/frame 0xffffff80002afbe0 fork_exit() at fork_exit+0x135/frame 0xffffff80002afc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002afc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002afcf0, rbp =3D 0 --- Tracing command idle pid 11 tid 100008 td 0xfffffe000a2b3920 cpustop_handler() at cpustop_handler+0x28/frame 0xffffff8000272d00 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff8000272d20 trap() at trap+0x355/frame 0xffffff8000272f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff8000272f20 --- trap 0x13, rip =3D 0xffffffff80ccff06, rsp =3D 0xffffff8000272fe0, rbp = =3D 0xffffff80002aaad0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xffffff80002aaad0 acpi_cpu_idle() at acpi_cpu_idle+0x2ac/frame 0xffffff80002aab10 cpu_idle_acpi() at cpu_idle_acpi+0x45/frame 0xffffff80002aab30 cpu_idle() at cpu_idle+0x78/frame 0xffffff80002aab50 sched_idletd() at sched_idletd+0x11e/frame 0xffffff80002aabe0 fork_exit() at fork_exit+0x135/frame 0xffffff80002aac30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002aac30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002aacf0, rbp =3D 0 --- Tracing command idle pid 11 tid 100007 td 0xfffffe000a2c0000 cpustop_handler() at cpustop_handler+0x28/frame 0xffffff800026bd00 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff800026bd20 trap() at trap+0x355/frame 0xffffff800026bf20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff800026bf20 --- trap 0x13, rip =3D 0xffffffff80ccff06, rsp =3D 0xffffff800026bfe0, rbp = =3D 0xffffff80002a5ad0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xffffff80002a5ad0 acpi_cpu_idle() at acpi_cpu_idle+0x2ac/frame 0xffffff80002a5b10 cpu_idle_acpi() at cpu_idle_acpi+0x45/frame 0xffffff80002a5b30 cpu_idle() at cpu_idle+0x78/frame 0xffffff80002a5b50 sched_idletd() at sched_idletd+0x11e/frame 0xffffff80002a5be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002a5c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002a5c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002a5cf0, rbp =3D 0 --- Tracing command idle pid 11 tid 100006 td 0xfffffe000a2c0490 sched_switch() at sched_switch+0x194/frame 0xffffff80002a0ae0 mi_switch() at mi_switch+0x208/frame 0xffffff80002a0b30 critical_exit() at critical_exit+0xb0/frame 0xffffff80002a0b50 sched_idletd() at sched_idletd+0x11e/frame 0xffffff80002a0be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002a0c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002a0c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002a0cf0, rbp =3D 0 --- Tracing command idle pid 11 tid 100005 td 0xfffffe000a2c0920 cpustop_handler() at cpustop_handler+0x28/frame 0xffffff800025dd00 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff800025dd20 trap() at trap+0x355/frame 0xffffff800025df20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff800025df20 --- trap 0x13, rip =3D 0xffffffff80ccff06, rsp =3D 0xffffff800025dfe0, rbp = =3D 0xffffff800029bad0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xffffff800029bad0 acpi_cpu_idle() at acpi_cpu_idle+0x2ac/frame 0xffffff800029bb10 cpu_idle_acpi() at cpu_idle_acpi+0x45/frame 0xffffff800029bb30 cpu_idle() at cpu_idle+0x78/frame 0xffffff800029bb50 sched_idletd() at sched_idletd+0x11e/frame 0xffffff800029bbe0 fork_exit() at fork_exit+0x135/frame 0xffffff800029bc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800029bc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff800029bcf0, rbp =3D 0 --- Tracing command idle pid 11 tid 100004 td 0xfffffe000a2b2000 cpustop_handler() at cpustop_handler+0x28/frame 0xffffff8000256d00 ipi_nmi_handler() at ipi_nmi_handler+0x3d/frame 0xffffff8000256d20 trap() at trap+0x355/frame 0xffffff8000256f20 nmi_calltrap() at nmi_calltrap+0x8/frame 0xffffff8000256f20 --- trap 0x13, rip =3D 0xffffffff80ccff06, rsp =3D 0xffffff8000256fe0, rbp = =3D 0xffffff8000296ad0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6/frame 0xffffff8000296ad0 acpi_cpu_idle() at acpi_cpu_idle+0x2ac/frame 0xffffff8000296b10 cpu_idle_acpi() at cpu_idle_acpi+0x45/frame 0xffffff8000296b30 cpu_idle() at cpu_idle+0x78/frame 0xffffff8000296b50 sched_idletd() at sched_idletd+0x11e/frame 0xffffff8000296be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000296c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000296c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000296cf0, rbp =3D 0 --- Tracing command idle pid 11 tid 100003 td 0xfffffe000a2b2490 sched_switch() at sched_switch+0x194/frame 0xffffff80002919c0 mi_switch() at mi_switch+0x208/frame 0xffffff8000291a10 critical_exit() at critical_exit+0xb0/frame 0xffffff8000291a30 lapic_handle_timer() at lapic_handle_timer+0x79/frame 0xffffff8000291a70 Xtimerint() at Xtimerint+0x88/frame 0xffffff8000291a70 --- interrupt, rip =3D 0xffffffff80cd9cc9, rsp =3D 0xffffff8000291b30, rbp = =3D 0xffffff8000291b50 --- cpu_idle_mwait() at cpu_idle_mwait+0x49/frame 0xffffff8000291b50 sched_idletd() at sched_idletd+0x11e/frame 0xffffff8000291be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000291c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000291c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000291cf0, rbp =3D 0 --- Tracing command init pid 1 tid 100002 td 0xfffffe000a2b2920 sched_switch() at sched_switch+0x194/frame 0xffffff800028c6d0 mi_switch() at mi_switch+0x208/frame 0xffffff800028c720 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800028c750 sleepq_catch_signals() at sleepq_catch_signals+0x2a2/frame 0xffffff800028c7= b0 sleepq_wait_sig() at sleepq_wait_sig+0x16/frame 0xffffff800028c7d0 _sleep() at _sleep+0x368/frame 0xffffff800028c860 kern_wait6() at kern_wait6+0x5e2/frame 0xffffff800028c8f0 kern_wait() at kern_wait+0x9c/frame 0xffffff800028ca50 sys_wait4() at sys_wait4+0x35/frame 0xffffff800028cb10 amd64_syscall() at amd64_syscall+0x2f9/frame 0xffffff800028cc30 Xfast_syscall() at Xfast_syscall+0xf7/frame 0xffffff800028cc30 --- syscall (7, FreeBSD ELF64, sys_wait4), rip =3D 0x412c8c, rsp =3D 0x7fff= ffffd7a8, rbp =3D 0x7d --- =20 Tracing command audit pid 10 tid 100001 td 0xfffffe000a2b3000 sched_switch() at sched_switch+0x194/frame 0xffffff8000287a50 mi_switch() at mi_switch+0x208/frame 0xffffff8000287aa0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000287ad0 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000287b00 _cv_wait() at _cv_wait+0x17d/frame 0xffffff8000287b60 audit_worker() at audit_worker+0x77/frame 0xffffff8000287be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000287c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000287c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000287cf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100065 td 0xfffffe000d993490 sched_switch() at sched_switch+0x194/frame 0xffffff884af8dab0 mi_switch() at mi_switch+0x208/frame 0xffffff884af8db00 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff884af8db30 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff884af8db60 msleep_spin() at msleep_spin+0x21c/frame 0xffffff884af8dbc0 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff884af8d= be0 fork_exit() at fork_exit+0x135/frame 0xffffff884af8dc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff884af8dc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff884af8dcf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100039 td 0xfffffe000a4a1000 sched_switch() at sched_switch+0x194/frame 0xffffff8000356ab0 mi_switch() at mi_switch+0x208/frame 0xffffff8000356b00 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000356b30 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000356b60 msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000356bc0 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000356= be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000356c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000356c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000356cf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100038 td 0xfffffe000a4a1490 sched_switch() at sched_switch+0x194/frame 0xffffff8000351ab0 mi_switch() at mi_switch+0x208/frame 0xffffff8000351b00 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000351b30 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000351b60 msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000351bc0 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000351= be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000351c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000351c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000351cf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100037 td 0xfffffe000a4a1920 sched_switch() at sched_switch+0x194/frame 0xffffff8000344ab0 mi_switch() at mi_switch+0x208/frame 0xffffff8000344b00 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000344b30 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000344b60 msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000344bc0 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000344= be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000344c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000344c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000344cf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100036 td 0xfffffe000a4a2000 sched_switch() at sched_switch+0x194/frame 0xffffff800033fab0 mi_switch() at mi_switch+0x208/frame 0xffffff800033fb00 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800033fb30 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff800033fb60 msleep_spin() at msleep_spin+0x21c/frame 0xffffff800033fbc0 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff800033f= be0 fork_exit() at fork_exit+0x135/frame 0xffffff800033fc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800033fc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff800033fcf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100035 td 0xfffffe000a4a2490 sched_switch() at sched_switch+0x194/frame 0xffffff8000332ab0 mi_switch() at mi_switch+0x208/frame 0xffffff8000332b00 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000332b30 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000332b60 msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000332bc0 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000332= be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000332c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000332c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000332cf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100034 td 0xfffffe000a4a2920 sched_switch() at sched_switch+0x194/frame 0xffffff800032dab0 mi_switch() at mi_switch+0x208/frame 0xffffff800032db00 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800032db30 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff800032db60 msleep_spin() at msleep_spin+0x21c/frame 0xffffff800032dbc0 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff800032d= be0 fork_exit() at fork_exit+0x135/frame 0xffffff800032dc30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800032dc30 --- trap 0, rip =3D 0, rsp =3D 0xffffff800032dcf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100033 td 0xfffffe000a4a3000 sched_switch() at sched_switch+0x194/frame 0xffffff8000328ab0 mi_switch() at mi_switch+0x208/frame 0xffffff8000328b00 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000328b30 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000328b60 msleep_spin() at msleep_spin+0x21c/frame 0xffffff8000328bc0 taskqueue_thread_loop() at taskqueue_thread_loop+0x5f/frame 0xffffff8000328= be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000328c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000328c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000328cf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100031 td 0xfffffe000a4a3920 sched_switch() at sched_switch+0x194/frame 0xffffff800031ea80 mi_switch() at mi_switch+0x208/frame 0xffffff800031ead0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800031eb00 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff800031eb30 _sleep() at _sleep+0x3d4/frame 0xffffff800031ebc0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7/frame 0xffffff800031e= be0 fork_exit() at fork_exit+0x135/frame 0xffffff800031ec30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800031ec30 --- trap 0, rip =3D 0, rsp =3D 0xffffff800031ecf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100030 td 0xfffffe000a2d7490 sched_switch() at sched_switch+0x194/frame 0xffffff8000319a80 mi_switch() at mi_switch+0x208/frame 0xffffff8000319ad0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff8000319b00 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff8000319b30 _sleep() at _sleep+0x3d4/frame 0xffffff8000319bc0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7/frame 0xffffff8000319= be0 fork_exit() at fork_exit+0x135/frame 0xffffff8000319c30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000319c30 --- trap 0, rip =3D 0, rsp =3D 0xffffff8000319cf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100027 td 0xfffffe000a2dc490 sched_switch() at sched_switch+0x194/frame 0xffffff800030aa80 mi_switch() at mi_switch+0x208/frame 0xffffff800030aad0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff800030ab00 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff800030ab30 _sleep() at _sleep+0x3d4/frame 0xffffff800030abc0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7/frame 0xffffff800030a= be0 fork_exit() at fork_exit+0x135/frame 0xffffff800030ac30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff800030ac30 --- trap 0, rip =3D 0, rsp =3D 0xffffff800030acf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100024 td 0xfffffe000a2dd490 sched_switch() at sched_switch+0x194/frame 0xffffff80002faa80 mi_switch() at mi_switch+0x208/frame 0xffffff80002faad0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffff80002fab00 sleepq_wait() at sleepq_wait+0x4d/frame 0xffffff80002fab30 _sleep() at _sleep+0x3d4/frame 0xffffff80002fabc0 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7/frame 0xffffff80002fa= be0 fork_exit() at fork_exit+0x135/frame 0xffffff80002fac30 fork_trampoline() at fork_trampoline+0xe/frame 0xffffff80002fac30 --- trap 0, rip =3D 0, rsp =3D 0xffffff80002facf0, rbp =3D 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff813f4ad0 sched_switch() at sched_switch+0x194/frame 0xffffffff81a26b50 mi_switch() at mi_switch+0x208/frame 0xffffffff81a26ba0 sleepq_switch() at sleepq_switch+0xfc/frame 0xffffffff81a26bd0 sleepq_timedwait() at sleepq_timedwait+0x4d/frame 0xffffffff81a26c00 _sleep() at _sleep+0x26d/frame 0xffffffff81a26c90 scheduler() at scheduler+0x2a1/frame 0xffffffff81a26cd0 mi_startup() at mi_startup+0x77/frame 0xffffffff81a26cf0 btext() at btext+0x2c --Multipart=_Thu__22_Aug_2013_10_21_21_+0200_nF//5/q+6KpaeERX-- --Signature=_Thu__22_Aug_2013_10_21_21_+0200_AqUicpZAxs_SMgFu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iEYEARECAAYFAlIVygoACgkQWTjlg++8y8uuiQCgy5zRNmWDac8q0CsZNZyhis7U QsoAn2CicaTnt81kdlRhuOCiyLFV+ECX =C514 -----END PGP SIGNATURE----- --Signature=_Thu__22_Aug_2013_10_21_21_+0200_AqUicpZAxs_SMgFu--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130822102121.a799b116b0f6b0f7c90ba578>