Date: Tue, 10 Nov 2015 02:25:09 +0700 From: Eugene Grosbein <eugen@grosbein.net> To: Adrian Chadd <adrian@freebsd.org> Cc: Jeff Roberson <jeff@freebsd.org>, Konstantin Belousov <kib@freebsd.org>, benno <benno@freebsd.org>, "freebsd-mips@freebsd.org" <freebsd-mips@freebsd.org> Subject: Re: USB-related problem Message-ID: <5640F315.5020303@grosbein.net> In-Reply-To: <CAJ-Vmo=6mztfvvBd91LPO5H418K8vW=%2BLk=6V5Z_y5DHu7v7HA@mail.gmail.com> References: <56348063.3090508@grosbein.net> <56351AA6.80903@grosbein.net> <CAJ-VmomLen5p7ELejU7QtC5CO2j8hydb_CvdOqQsfaaMAtmdkA@mail.gmail.com> <563523CA.3040207@grosbein.net> <CAJ-Vmonpd8R5C%2BNvhOGFD1okkkit990DV8c4Dm%2BLmpZ2uFfVkA@mail.gmail.com> <56367686.4090801@grosbein.net> <CAJ-VmomtJ9eKfAJYfn4e2S1xWN8-YHo0M0KH-9V=VjNBf6vVVA@mail.gmail.com> <563707A0.3040700@grosbein.net> <CANCZdfrB0hkjmTU-9NzimBv=X_h-=bF0D2azBxg=9B=kmitv7Q@mail.gmail.com> <56370E1D.3040801@grosbein.net> <CAJ-Vmo=0vOAq8db_GeLWmdXr7xJdzUh44ZZJrQ9vVdpvzT9hiQ@mail.gmail.com> <563F5630.2000407@grosbein.net> <563F6F6F.1010807@grosbein.net> <CAJ-Vmo=fPSi7yZO5Xjodg8HPtTLd44Y9Y_8qg4EgTGwEpHO10A@mail.gmail.com> <563F91A8.9080702@grosbein.net> <CAJ-VmomUvoUerMS20qQsQujcjULVA=_jaLp9Mh3fU1fEpdwzZA@mail.gmail.com> <CAJ-Vmo=eUUZ928KgQbyOi8EdDFSmxhjvDOyAMvfXsqwDbO96ng@mail.gmail.com> <5640C0FD.2040803@grosbein.net> <CAJ-Vmo=6mztfvvBd91LPO5H418K8vW=%2BLk=6V5Z_y5DHu7v7HA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 09.11.2015 22:53, Adrian Chadd wrote: > ok, let's get it to hang, then break into ddb and do 'show > allthreads', 'show alllocks', 'bufvars' with my vfs_bio.c patch, etc. I've re-read https://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-deadlocks.html and rebuilt kernel with additional debugging and reproduced the problem again to supply more vfs lock info: KDB: enter: Break to debugger [ thread pid 14 tid 100036 ] Stopped at kdb_enter+0x4c: lui at,0x8042 db> bufvars vmiodirenable=1 runningbufspace=262144 bufspace=1032192 bufkvaspace=1966080 maxbufspace=2097152 bufmallocspace=0 maxbufmallocspace=78643 lobufspace=1494217 hibufspace=1572864 bufspacethresh=1533540 buffreekvacnt=1851 bufdefragcnt=4684045 lorunningspace=720896 hirunningspace=1048576 dirtybufferflushes=0 bdwriteskip=0 altbufferflushes=0 recursiveflushes=0 numdirtybuffers=31 lodirtybuffers=26 hidirtybuffers=52 dirtybufthresh=46 numfreebuffers=92 lofreebuffers=25 hifreebuffers=37 getnewbufcalls=2343049 getnewbufrestarts=1532 mappingrestarts=0 numbufallocfails=0 flushbufqtarget=100 notbufdflushes=0 barrierwrites=0 unmapped_buf_allowed=0 clean_queues=1 db> ps pid ppid pgrp uid state wmesg wchan cmd 667 664 667 0 S+ ttyin 0x80fe1a70 sh 664 663 663 100 S+ wait 0x82167378 su 663 662 663 0 Ss+ wait 0x821676f0 login 662 629 662 0 Ss select 0x809935a4 telnetd 661 660 660 0 R+ pkg-static 660 652 660 0 S+ wait 0x80946a68 pkg-static 652 651 652 0 S+ wait 0x82167a68 sh 651 1 651 0 Ss+ wait 0x80a4aa68 login 629 1 629 0 Ss select 0x80a18024 inetd 609 1 609 0 Rs cron 606 1 606 0 Ss select 0x80a181a4 sshd 447 1 447 0 Ss select 0x80a182e4 syslogd 39 0 0 0 DL mdwait 0x80fc2000 [md1] 29 0 0 0 DL mdwait 0x80fc2800 [md0] 15 0 0 0 RL [schedcpu] 14 0 0 0 RL CPU 0 [syncer] 9 0 0 0 RL [vnlru] 8 0 0 0 RL [bufspacedaemon] 7 0 0 0 RL [bufdaemon] 6 0 0 0 DL pgzero 0x8047d868 [pagezero] 5 0 0 0 RL (threaded) [pagedaemon] 100031 RunQ [pagedaemon] 100038 D umarcl 0x8047cbc8 [uma] 4 0 0 0 DL jobqueue 0x809d2900 [task: mx25l flash] 13 0 0 0 DL (threaded) [usb] 100021 D - 0xc0888ca4 [usbus0] 100022 D - 0xc0888cd4 [usbus0] 100023 D - 0xc0888d04 [usbus0] 100024 D - 0xc0888d34 [usbus0] 100025 D - 0xc0888d64 [usbus0] 3 0 0 0 DL (threaded) [cam] 100011 D - 0x80401680 [doneq0] 100030 D - 0x804015b4 [scanner] 12 0 0 0 DL (threaded) [geom] 100007 D - 0x804098f4 [g_event] 100008 D - 0x804098fc [g_up] 100009 D - 0x80409900 [g_down] 11 0 0 0 RL (threaded) [intr] 100004 RunQ [swi4: clock (0)] 100005 I [swi3: vm] 100006 I [swi1: netisr 0] 100012 I [swi5: fast taskq] 100014 I [swi6: Giant taskq] 100015 I [swi6: task queue] 100018 I [swi0: uart] 100019 I [apb intr2: gpio0] 100020 RunQ [int1 ehci0] 100026 RunQ [int2 arge0] 100027 I [int3 arge1] 10 0 0 0 RL [idle] 2 0 0 0 RL [rand_harvestq] 1 0 1 0 SLs wait 0x80946000 [init] 0 0 0 0 RLs (threaded) [kernel] 100000 RunQ [swapper] 100010 D - 0x80961000 [firmware taskq] 100013 D - 0x80960780 [thread taskq] 100016 D - 0x80960000 [kqueue taskq] 100017 D - 0x8095ff00 [ffs_trim taskq] 100029 D - 0x80960a80 [CAM taskq] db> show pcpu cpuid = 0 dynamic pcpu = 0x118080 curthread = 0x809d4360: pid 14 "syncer" curpcb = 0xc3adbe98 fpcurthread = none idlethread = 0x809496c0: tid 100003 "idle" ipis = 0x0 next ASID = 152 GENID = 3 spin locks held: db> show allpcpu Current CPU: 0 cpuid = 0 dynamic pcpu = 0x118080 curthread = 0x809d4360: pid 14 "syncer" curpcb = 0xc3adbe98 fpcurthread = none idlethread = 0x809496c0: tid 100003 "idle" ipis = 0x0 next ASID = 152 GENID = 3 spin locks held: db> show locks exclusive sleep mutex buffer arena (buffer arena) r = 0 (0x8041dd00) locked @ /home/src/sys/kern/subr_vmem.c:1202 exclusive lockmgr bufwait (bufwait) r = 0 (0xc3033610) locked @ /home/src/sys/kern/vfs_bio.c:1454 exclusive lockmgr bufwait (bufwait) r = 0 (0xc3036778) locked @ /home/src/sys/kern/vfs_default.c:671 exclusive lockmgr devfs (devfs) r = 0 (0x8430e8a4) locked @ /home/src/sys/kern/vfs_subr.c:1779 db> show alllocks Process 661 (pkg-static) thread 0x80fcda20 (100048) exclusive lockmgr ufs (ufs) r = 0 (0x843eecdc) locked @ /home/src/sys/kern/vfs_vnops.c:873 Process 14 (syncer) thread 0x809d4360 (100036) exclusive sleep mutex buffer arena (buffer arena) r = 0 (0x8041dd00) locked @ /home/src/sys/kern/subr_vmem.c:1202 exclusive lockmgr bufwait (bufwait) r = 0 (0xc3033610) locked @ /home/src/sys/kern/vfs_bio.c:1454 exclusive lockmgr bufwait (bufwait) r = 0 (0xc3036778) locked @ /home/src/sys/kern/vfs_default.c:671 exclusive lockmgr devfs (devfs) r = 0 (0x8430e8a4) locked @ /home/src/sys/kern/vfs_subr.c:1779 db> show lockedvnods Locked vnodes 0x8430e870: tag devfs, type VCHR usecount 1, writecount 0, refcount 9 mountedhere 0x80fe0e00 flags (VI_ACTIVE) v_object 0x843691e0 ref 0 pages 114 cleanbuf 0 dirtybuf 6 lock type devfs: EXCL by thread 0x809d4360 (pid 14, syncer, tid 100036) dev ufs/64k 0x843eeca8: tag ufs, type VREG usecount 1, writecount 1, refcount 7 mountedhere 0 flags (VI_ACTIVE) v_object 0x843d1e60 ref 0 pages 144 cleanbuf 4 dirtybuf 1 lock type ufs: EXCL by thread 0x80fcda20 (pid 661, pkg-static, tid 100048) ino 394022, on dev ufs/64k db> thread 100036 [ thread pid 14 tid 100036 ] kdb_enter+0x4c: lui at,0x8042 db> where Tracing pid 14 tid 100036 td 0x809d4360 db_trace_thread+30 (?,?,?,?) ra c3adb4f800000018 sp 0 sz 0 8008c460+114 (0,?,ffffffff,?) ra c3adb51000000020 sp 100000000 sz 1 8008b7a4+388 (?,?,?,?) ra c3adb530000000a8 sp 0 sz 0 db_command_loop+70 (?,?,?,?) ra c3adb5d800000018 sp 0 sz 0 8008e378+f4 (?,?,?,?) ra c3adb5f0000001a8 sp 0 sz 0 kdb_trap+100 (?,?,?,?) ra c3adb79800000030 sp 0 sz 0 trap+d58 (?,?,?,?) ra c3adb7c8000000c0 sp 0 sz 0 MipsKernGenException+ec (0,4,8035df6c,12f) ra c3adb888000000c8 sp 100000001 sz 1 kdb_enter+4c (?,?,?,?) ra c3adb95000000018 sp 0 sz 0 801b0cc0+ac (?,?,?,?) ra c3adb96800000018 sp 0 sz 0 kdb_alt_break+10 (?,?,?,?) ra c3adb98000000018 sp 0 sz 0 800c5c40+254 (?,?,?,?) ra c3adb99800000058 sp 0 sz 0 intr_event_handle+9c (?,?,?,?) ra c3adb9f000000030 sp 0 sz 0 8031fbfc+160 (?,?,?,?) ra c3adba2000000028 sp 0 sz 0 intr_event_handle+9c (?,?,?,?) ra c3adba4800000030 sp 0 sz 0 cpu_intr+198 (c3adbac8,?,?,?) ra c3adba7800000038 sp 100000000 sz 0 MipsKernIntr+110 (?,?,?,?) ra c3adbab0000000c8 sp 0 sz 0 pid 14 [ thread pid 661 tid 100048 ] 0x7e22a4: db> where Tracing pid 661 tid 100048 td 0x80fcda20 cpu_switch+6c (?,?,?,?) ra c3b1776000000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b1776000000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b1778800000020 sp 0 sz 0 kern_yield+11c (?,?,?,?) ra c3b177a800000030 sp 0 sz 0 getblk+77c (843eeca8,?,?,?) ra c3b177d800000070 sp 100000000 sz 0 ffs_balloc_ufs2+11d8 (843eeca8,?,?,?) ra c3b17848000001a0 sp 100000000 sz 0 802bb954+384 (c3b17b60,?,?,?) ra c3b179e800000090 sp 100000000 sz 0 VOP_WRITE_APV+18c (?,?,?,?) ra c3b17a78000000d0 sp 0 sz 0 80232d1c+1fc (?,?,?,?) ra c3b17b4800000060 sp 0 sz 0 802311d4+4c (?,?,?,?) ra c3b17ba800000038 sp 0 sz 0 802312cc+14c (?,?,c3b17cc0,?) ra c3b17be0000000c0 sp 0 sz 1 802316dc+174 (?,?,?,?) ra c3b17ca000000058 sp 0 sz 0 801d0eec+cc (?,?,?,?) ra c3b17cf800000038 sp 0 sz 0 kern_writev+8c (?,?,?,?) ra c3b17d3000000050 sp 0 sz 0 sys_write+48 (?,?,?,?) ra c3b17d8000000040 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b17dc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,7e22a0) ra c3b17e8000000000 sp 0 sz 0 pid 14 db> alltrace Tracing command sh pid 667 tid 100058 td 0x8436ba20 cpu_switch+6c (?,?,?,?) ra c3b3fb2000000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b3fb2000000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b3fb4800000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b3fb6800000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3b3fb9000000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3b3fbc000000020 sp 0 sz 0 _cv_wait_sig+2ac (?,?,?,?) ra c3b3fbe000000048 sp 0 sz 0 tty_wait+78 (?,?,?,?) ra c3b3fc2800000020 sp 0 sz 0 ttydisc_read+278 (?,?,?,?) ra c3b3fc4800000048 sp 0 sz 0 801dd318+84 (?,?,?,?) ra c3b3fc9000000020 sp 0 sz 0 800f3040+c0 (?,?,?,?) ra c3b3fcb000000048 sp 0 sz 0 801d1094+b0 (?,?,?,?) ra c3b3fcf800000038 sp 0 sz 0 kern_readv+8c (?,?,?,?) ra c3b3fd3000000050 sp 0 sz 0 sys_read+48 (?,?,?,?) ra c3b3fd8000000040 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b3fdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,406067b0) ra c3b3fe8000000000 sp 0 sz 0 pid 14 --More-- Tracing command su pid 664 tid 100055 td 0x821f96c0 cpu_switch+6c (?,?,?,?) ra c3b33ab800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b33ab800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b33ae000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b33b0000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3b33b2800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3b33b5800000020 sp 0 sz 0 _sleep+41c (?,?,?,803528f8) ra c3b33b7800000068 sp 0 sz 0 kern_wait6+6d4 (821f96c0,?,?,?) ra c3b33be000000078 sp 100000000 sz 0 kern_wait+d8 (?,?,?,?) ra c3b33c58000000e0 sp 0 sz 0 sys_wait4+40 (?,?,?,?) ra c3b33d3800000088 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b33dc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,40562810) ra c3b33e8000000000 sp 0 sz 0 pid 14 Tracing command login pid 663 tid 100054 td 0x821f9a20 cpu_switch+6c (?,?,?,?) ra c3b2fab800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b2fab800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b2fae000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b2fb0000000028 sp 0 sz 0 --More-- 801bf160+118 (?,?,?,?) ra c3b2fb2800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3b2fb5800000020 sp 0 sz 0 _sleep+41c (?,?,?,803528f8) ra c3b2fb7800000068 sp 0 sz 0 kern_wait6+6d4 (821f9a20,?,?,?) ra c3b2fbe000000078 sp 100000000 sz 0 kern_wait+d8 (?,?,?,?) ra c3b2fc58000000e0 sp 0 sz 0 sys_wait4+40 (?,?,?,?) ra c3b2fd3800000088 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b2fdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,40564810) ra c3b2fe8000000000 sp 0 sz 0 pid 14 Tracing command telnetd pid 662 tid 100049 td 0x80fcd6c0 cpu_switch+6c (?,?,?,?) ra c3b1ba9800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b1ba9800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b1bac000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b1bae000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3b1bb0800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3b1bb3800000020 sp 0 sz 0 _cv_wait_sig+2ac (?,?,?,?) ra c3b1bb5800000048 sp 0 sz 0 801d0cb4+d0 (?,?,?,?) ra c3b1bba000000038 sp 0 sz 0 kern_select+6ec (?,?,7ffee84c,7ffee8cc) ra c3b1bbd8000001a8 sp 0 sz 1 --More-- sys_select+6c (?,?,?,?) ra c3b1bd8000000040 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b1bdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,40853780) ra c3b1be8000000000 sp 0 sz 0 pid 14 Tracing command pkg-static pid 661 tid 100048 td 0x80fcda20 cpu_switch+6c (?,?,?,?) ra c3b1776000000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b1776000000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b1778800000020 sp 0 sz 0 kern_yield+11c (?,?,?,?) ra c3b177a800000030 sp 0 sz 0 getblk+77c (843eeca8,?,?,?) ra c3b177d800000070 sp 100000000 sz 0 ffs_balloc_ufs2+11d8 (843eeca8,?,?,?) ra c3b17848000001a0 sp 100000000 sz 0 802bb954+384 (c3b17b60,?,?,?) ra c3b179e800000090 sp 100000000 sz 0 VOP_WRITE_APV+18c (?,?,?,?) ra c3b17a78000000d0 sp 0 sz 0 80232d1c+1fc (?,?,?,?) ra c3b17b4800000060 sp 0 sz 0 802311d4+4c (?,?,?,?) ra c3b17ba800000038 sp 0 sz 0 802312cc+14c (?,?,c3b17cc0,?) ra c3b17be0000000c0 sp 0 sz 1 802316dc+174 (?,?,?,?) ra c3b17ca000000058 sp 0 sz 0 801d0eec+cc (?,?,?,?) ra c3b17cf800000038 sp 0 sz 0 kern_writev+8c (?,?,?,?) ra c3b17d3000000050 sp 0 sz 0 --More-- sys_write+48 (?,?,?,?) ra c3b17d8000000040 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b17dc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,7e22a0) ra c3b17e8000000000 sp 0 sz 0 pid 14 Tracing command pkg-static pid 660 tid 100047 td 0x80fce000 cpu_switch+6c (?,?,?,?) ra c3b13ab800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b13ab800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b13ae000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b13b0000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3b13b2800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3b13b5800000020 sp 0 sz 0 _sleep+41c (?,?,?,803528f8) ra c3b13b7800000068 sp 0 sz 0 kern_wait6+6d4 (80fce000,?,?,?) ra c3b13be000000078 sp 100000000 sz 0 kern_wait+d8 (?,?,?,?) ra c3b13c58000000e0 sp 0 sz 0 sys_wait4+40 (?,?,?,?) ra c3b13d3800000088 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b13dc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,780ea0) ra c3b13e8000000000 sp 0 sz 0 pid 14 --More-- Tracing command sh pid 652 tid 100053 td 0x84339000 cpu_switch+6c (?,?,?,?) ra c3b2bab800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b2bab800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b2bae000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b2bb0000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3b2bb2800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3b2bb5800000020 sp 0 sz 0 _sleep+41c (?,?,?,803528f8) ra c3b2bb7800000068 sp 0 sz 0 kern_wait6+6d4 (84339000,?,?,?) ra c3b2bbe000000078 sp 100000000 sz 0 kern_wait+d8 (?,?,?,?) ra c3b2bc58000000e0 sp 0 sz 0 sys_wait4+40 (?,?,?,?) ra c3b2bd3800000088 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b2bdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,4058b810) ra c3b2be8000000000 sp 0 sz 0 pid 14 Tracing command login pid 651 tid 100039 td 0x809796c0 cpu_switch+6c (?,?,?,?) ra c3aebab800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3aebab800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3aebae000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3aebb0000000028 sp 0 sz 0 --More-- 801bf160+118 (?,?,?,?) ra c3aebb2800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3aebb5800000020 sp 0 sz 0 _sleep+41c (?,?,?,803528f8) ra c3aebb7800000068 sp 0 sz 0 kern_wait6+6d4 (809796c0,?,?,?) ra c3aebbe000000078 sp 100000000 sz 0 kern_wait+d8 (?,?,?,?) ra c3aebc58000000e0 sp 0 sz 0 sys_wait4+40 (?,?,?,?) ra c3aebd3800000088 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3aebdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,40564810) ra c3aebe8000000000 sp 0 sz 0 pid 14 Tracing command inetd pid 629 tid 100046 td 0x80fce360 cpu_switch+6c (?,?,?,?) ra c3b0fa9800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b0fa9800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b0fac000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b0fae000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3b0fb0800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3b0fb3800000020 sp 0 sz 0 _cv_wait_sig+2ac (?,?,?,?) ra c3b0fb5800000048 sp 0 sz 0 801d0cb4+d0 (?,?,?,?) ra c3b0fba000000038 sp 0 sz 0 kern_select+6ec (?,?,7ffee150,0) ra c3b0fbd8000001a8 sp 0 sz 1 --More-- sys_select+6c (?,?,?,?) ra c3b0fd8000000040 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b0fdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,405f7780) ra c3b0fe8000000000 sp 0 sz 0 pid 14 Tracing command cron pid 609 tid 100041 td 0x80fcf6c0 cpu_switch+6c (?,?,?,?) ra c3afbbe800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3afbbe800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3afbc1000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3afbc3000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3afbc5800000030 sp 0 sz 0 sleepq_timedwait_sig+18 (?,?,?,?) ra c3afbc8800000020 sp 0 sz 0 _sleep+3f4 (?,?,?,80361504) ra c3afbca800000068 sp 0 sz 0 kern_nanosleep+244 (?,?,?,?) ra c3afbd1000000070 sp 0 sz 0 sys_nanosleep+6c (?,?,?,?) ra c3afbd8000000040 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3afbdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,4056aa20) ra c3afbe8000000000 sp 0 sz 0 pid 14 Tracing command sshd pid 606 tid 100045 td 0x80fce6c0 --More-- cpu_switch+6c (?,?,?,?) ra c3b0ba9800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b0ba9800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b0bac000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b0bae000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3b0bb0800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3b0bb3800000020 sp 0 sz 0 _cv_wait_sig+2ac (?,?,?,?) ra c3b0bb5800000048 sp 0 sz 0 801d0cb4+d0 (?,?,?,?) ra c3b0bba000000038 sp 0 sz 0 kern_select+6ec (?,?,40c100a8,0) ra c3b0bbd8000001a8 sp 0 sz 1 sys_select+6c (?,?,?,?) ra c3b0bd8000000040 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3b0bdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,408d6780) ra c3b0be8000000000 sp 0 sz 0 pid 14 Tracing command syslogd pid 447 tid 100042 td 0x80fcf360 cpu_switch+6c (?,?,?,?) ra c3affa9800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3affa9800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3affac000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3affae000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3affb0800000030 sp 0 sz 0 --More-- sleepq_wait_sig+14 (?,?,?,?) ra c3affb3800000020 sp 0 sz 0 _cv_wait_sig+2ac (?,?,?,?) ra c3affb5800000048 sp 0 sz 0 801d0cb4+d0 (?,?,?,?) ra c3affba000000038 sp 0 sz 0 kern_select+6ec (?,?,40a1f0a0,0) ra c3affbd8000001a8 sp 0 sz 1 sys_select+6c (?,?,?,?) ra c3affd8000000040 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3affdc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,405c7780) ra c3affe8000000000 sp 0 sz 0 pid 14 Tracing command md1 pid 39 tid 100044 td 0x80fcea20 cpu_switch+6c (?,?,?,?) ra c3b07d1800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b07d1800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b07d4000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b07d6000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3b07d8800000020 sp 0 sz 0 _sleep+42c (?,?,?,8034f200) ra c3b07da800000068 sp 0 sz 0 800a7e68+150 (?,?,?,?) ra c3b07e1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3b07e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3b07e8000000000 sp 0 sz 0 pid 14 --More-- Tracing command md0 pid 29 tid 100043 td 0x80fcf000 cpu_switch+6c (?,?,?,?) ra c3b03d1800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3b03d1800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3b03d4000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3b03d6000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3b03d8800000020 sp 0 sz 0 _sleep+42c (?,?,?,8034f200) ra c3b03da800000068 sp 0 sz 0 800a7e68+150 (?,?,?,?) ra c3b03e1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3b03e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3b03e8000000000 sp 0 sz 0 pid 14 Tracing command schedcpu pid 15 tid 100037 td 0x809d4000 cpu_switch+6c (?,?,?,?) ra c3adfce800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3adfce800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3adfd1000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3adfd3000000028 sp 0 sz 0 sleepq_timedwait+78 (?,?,?,?) ra c3adfd5800000020 sp 0 sz 0 _sleep+404 (?,?,?,80346528) ra c3adfd7800000068 sp 0 sz 0 --More-- pause_sbt+110 (?,?,?,?) ra c3adfde000000038 sp 0 sz 0 8019fa50+2dc (?,?,?,?) ra c3adfe1800000040 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3adfe5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3adfe8000000000 sp 0 sz 0 pid 14 Tracing command syncer pid 14 tid 100036 td 0x809d4360 db_trace_thread+30 (?,?,?,?) ra c3adb36800000018 sp 0 sz 0 8008c390+78 (?,?,?,?) ra c3adb380000001b0 sp 0 sz 0 8008b7a4+388 (?,?,?,?) ra c3adb530000000a8 sp 0 sz 0 db_command_loop+70 (?,?,?,?) ra c3adb5d800000018 sp 0 sz 0 8008e378+f4 (?,?,?,?) ra c3adb5f0000001a8 sp 0 sz 0 kdb_trap+100 (?,?,?,?) ra c3adb79800000030 sp 0 sz 0 trap+d58 (?,?,?,?) ra c3adb7c8000000c0 sp 0 sz 0 MipsKernGenException+ec (0,4,8035df6c,12f) ra c3adb888000000c8 sp 100000001 sz 1 kdb_enter+4c (?,?,?,?) ra c3adb95000000018 sp 0 sz 0 801b0cc0+ac (?,?,?,?) ra c3adb96800000018 sp 0 sz 0 kdb_alt_break+10 (?,?,?,?) ra c3adb98000000018 sp 0 sz 0 800c5c40+254 (?,?,?,?) ra c3adb99800000058 sp 0 sz 0 intr_event_handle+9c (?,?,?,?) ra c3adb9f000000030 sp 0 sz 0 --More-- 8031fbfc+160 (?,?,?,?) ra c3adba2000000028 sp 0 sz 0 intr_event_handle+9c (?,?,?,?) ra c3adba4800000030 sp 0 sz 0 cpu_intr+198 (c3adbac8,?,?,?) ra c3adba7800000038 sp 100000000 sz 0 MipsKernIntr+110 (?,?,?,?) ra c3adbab0000000c8 sp 0 sz 0 pid 14 Tracing command vnlru pid 9 tid 100035 td 0x809d46c0 cpu_switch+6c (?,?,?,?) ra c3ad7cf800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3ad7cf800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3ad7d2000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3ad7d4000000028 sp 0 sz 0 sleepq_timedwait+78 (?,?,?,?) ra c3ad7d6800000020 sp 0 sz 0 _sleep+404 (?,?,?,80370010) ra c3ad7d8800000068 sp 0 sz 0 80227154+174 (?,?,?,?) ra c3ad7df000000068 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3ad7e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3ad7e8000000000 sp 0 sz 0 pid 14 Tracing command bufspacedaemon pid 8 tid 100034 td 0x809d4a20 cpu_switch+6c (?,?,?,?) ra c3ad3d2800000000 sp 0 sz 0 --More-- sched_switch+224 (?,?,?,?) ra c3ad3d2800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3ad3d5000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3ad3d7000000028 sp 0 sz 0 sleepq_timedwait+78 (?,?,?,?) ra c3ad3d9800000020 sp 0 sz 0 _sleep+404 (?,?,?,80346528) ra c3ad3db800000068 sp 0 sz 0 8020985c+250 (?,?,?,?) ra c3ad3e2000000038 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3ad3e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3ad3e8000000000 sp 0 sz 0 pid 14 Tracing command bufdaemon pid 7 tid 100033 td 0x809d5000 cpu_switch+6c (?,?,?,?) ra c3acfd1000000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3acfd1000000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3acfd3800000020 sp 0 sz 0 kern_yield+11c (?,?,?,?) ra c3acfd5800000030 sp 0 sz 0 maybe_yield+20 (?,?,?,?) ra c3acfd8800000018 sp 0 sz 0 80206cfc+110 (?,5,0,?) ra c3acfda000000060 sp 1 sz 1 802074d0+24 (?,?,?,?) ra c3acfe0000000020 sp 0 sz 0 80207544+fc (?,?,?,?) ra c3acfe2000000038 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3acfe5800000028 sp 0 sz 0 --More-- fork_trampoline+10 (?,?,?,?) ra c3acfe8000000000 sp 0 sz 0 pid 14 Tracing command pagezero pid 6 tid 100032 td 0x809d5360 cpu_switch+6c (?,?,?,?) ra c3acbd2800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3acbd2800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3acbd5000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3acbd7000000028 sp 0 sz 0 sleepq_timedwait+78 (?,?,?,?) ra c3acbd9800000020 sp 0 sz 0 _sleep+404 (?,?,?,8037e4e4) ra c3acbdb800000068 sp 0 sz 0 802f9e74+1c0 (?,?,?,?) ra c3acbe2000000038 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3acbe5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3acbe8000000000 sp 0 sz 0 pid 14 Tracing command pagedaemon pid 5 tid 100031 td 0x809d56c0 cpu_switch+6c (?,?,?,?) ra c3ac7ce800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3ac7ce800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3ac7d1000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3ac7d3000000028 sp 0 sz 0 --More-- sleepq_timedwait+78 (?,?,?,?) ra c3ac7d5800000020 sp 0 sz 0 _sleep+404 (?,?,?,8036cf04) ra c3ac7d7800000068 sp 0 sz 0 802f2c30+2e4 (?,?,?,?) ra c3ac7de000000078 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3ac7e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3ac7e8000000000 sp 0 sz 0 pid 14 Tracing command pagedaemon pid 5 tid 100038 td 0x80979a20 cpu_switch+6c (?,?,?,?) ra c3ae3d3000000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3ae3d3000000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3ae3d5800000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3ae3d7800000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3ae3da000000020 sp 0 sz 0 _sleep+42c (?,?,?,8037a3f0) ra c3ae3dc000000068 sp 0 sz 0 uma_reclaim_worker+64 (?,?,?,?) ra c3ae3e2800000030 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3ae3e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3ae3e8000000000 sp 0 sz 0 pid 14 Tracing command task: mx25l flash pid 4 tid 100028 td 0x809d6360 --More-- cpu_switch+6c (?,?,?,?) ra c3ab1c6800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3ab1c6800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3ab1c9000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3ab1cb000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3ab1cd800000020 sp 0 sz 0 _sleep+42c (?,?,?,8034d82c) ra c3ab1cf800000068 sp 0 sz 0 8009956c+ac (809d290c,?,?,?) ra c3ab1d60000000f8 sp 100000000 sz 0 fork_exit+b0 (?,?,?,?) ra c3ab1e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3ab1e8000000000 sp 0 sz 0 pid 14 Tracing command usb pid 13 tid 100021 td 0x809776c0 cpu_switch+6c (?,?,?,?) ra c3a93d5800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a93d5800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a93d8000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a93da000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a93dc800000020 sp 0 sz 0 _cv_wait+2a8 (?,?,?,?) ra c3a93de800000048 sp 0 sz 0 800e72bc+158 (?,?,?,?) ra c3a93e3000000028 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a93e5800000028 sp 0 sz 0 --More-- fork_trampoline+10 (?,?,?,?) ra c3a93e8000000000 sp 0 sz 0 pid 14 Tracing command usb pid 13 tid 100022 td 0x80977360 cpu_switch+6c (?,?,?,?) ra c3a97d5800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a97d5800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a97d8000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a97da000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a97dc800000020 sp 0 sz 0 _cv_wait+2a8 (?,?,?,?) ra c3a97de800000048 sp 0 sz 0 800e72bc+158 (?,?,?,?) ra c3a97e3000000028 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a97e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a97e8000000000 sp 0 sz 0 pid 14 Tracing command usb pid 13 tid 100023 td 0x80977000 cpu_switch+6c (?,?,?,?) ra c3a9bd5800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a9bd5800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a9bd8000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a9bda000000028 sp 0 sz 0 --More-- sleepq_wait+78 (?,?,?,?) ra c3a9bdc800000020 sp 0 sz 0 _cv_wait+2a8 (?,?,?,?) ra c3a9bde800000048 sp 0 sz 0 800e72bc+158 (?,?,?,?) ra c3a9be3000000028 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a9be5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a9be8000000000 sp 0 sz 0 pid 14 Tracing command usb pid 13 tid 100024 td 0x8094aa20 cpu_switch+6c (?,?,?,?) ra c3a9fd5800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a9fd5800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a9fd8000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a9fda000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a9fdc800000020 sp 0 sz 0 _cv_wait+2a8 (?,?,?,?) ra c3a9fde800000048 sp 0 sz 0 800e72bc+158 (?,?,?,?) ra c3a9fe3000000028 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a9fe5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a9fe8000000000 sp 0 sz 0 pid 14 Tracing command usb pid 13 tid 100025 td 0x8094a6c0 --More-- cpu_switch+6c (?,?,?,?) ra c3aa3d5800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3aa3d5800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3aa3d8000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3aa3da000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3aa3dc800000020 sp 0 sz 0 _cv_wait+2a8 (?,?,?,?) ra c3aa3de800000048 sp 0 sz 0 800e72bc+158 (?,?,?,?) ra c3aa3e3000000028 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3aa3e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3aa3e8000000000 sp 0 sz 0 pid 14 Tracing command cam pid 3 tid 100011 td 0x809476c0 cpu_switch+6c (?,?,?,?) ra c3a4bd1800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a4bd1800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a4bd4000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a4bd6000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a4bd8800000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3a4bda800000068 sp 0 sz 0 8005c480+90 (?,?,?,?) ra c3a4be1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a4be5800000028 sp 0 sz 0 --More-- fork_trampoline+10 (?,?,?,?) ra c3a4be8000000000 sp 0 sz 0 pid 14 Tracing command cam pid 3 tid 100030 td 0x809d5a20 cpu_switch+6c (?,?,?,?) ra c3abbd1000000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3abbd1000000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3abbd3800000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3abbd5800000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3abbd8000000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3abbda000000068 sp 0 sz 0 80057d20+74 (?,?,?,?) ra c3abbe0800000050 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3abbe5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3abbe8000000000 sp 0 sz 0 pid 14 Tracing command geom pid 12 tid 100007 td 0x809486c0 cpu_switch+6c (?,?,?,?) ra c3a3bcf800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a3bcf800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a3bd2000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a3bd4000000028 sp 0 sz 0 --More-- sleepq_wait+78 (?,?,?,?) ra c3a3bd6800000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3a3bd8800000068 sp 0 sz 0 g_run_events+614 (?,?,?,?) ra c3a3bdf000000050 sp 0 sz 0 800ff868+7c (?,?,?,?) ra c3a3be4000000018 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a3be5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a3be8000000000 sp 0 sz 0 pid 14 Tracing command geom pid 12 tid 100008 td 0x80948360 cpu_switch+6c (?,?,?,?) ra c3a3fd1800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a3fd1800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a3fd4000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a3fd6000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a3fd8800000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3a3fda800000068 sp 0 sz 0 g_io_schedule_up+118 (?,?,?,?) ra c3a3fe1000000030 sp 0 sz 0 800ff7e0+80 (?,?,?,?) ra c3a3fe4000000018 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a3fe5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a3fe8000000000 sp 0 sz 0 pid 14 --More-- Tracing command geom pid 12 tid 100009 td 0x80948000 cpu_switch+6c (?,?,?,?) ra c3a43d1000000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a43d1000000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a43d3800000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a43d5800000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a43d8000000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3a43da000000068 sp 0 sz 0 g_io_schedule_down+7c (?,?,?,?) ra c3a43e0800000038 sp 0 sz 0 800ff758+80 (?,?,?,?) ra c3a43e4000000018 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a43e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a43e8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100004 td 0x80949360 cpu_switch+6c (?,?,?,?) ra c3a2fdc800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a2fdc800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a2fdf000000020 sp 0 sz 0 80141af0+258 (?,?,?,?) ra c3a2fe1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a2fe5800000028 sp 0 sz 0 --More-- fork_trampoline+10 (?,?,?,?) ra c3a2fe8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100005 td 0x80949000 PC 0x80313b408008e8c0: not in kernel 0+0 (?,?,?,?) ra c3a33e8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100006 td 0x80948a20 cpu_switch+6c (?,?,?,?) ra c3a37dc800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a37dc800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a37df000000020 sp 0 sz 0 80141af0+258 (?,?,?,?) ra c3a37e1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a37e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a37e8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100012 td 0x80947360 PC 0x80313b408008e8c0: not in kernel 0+0 (?,?,?,?) ra c3a4fe8000000000 sp 0 sz 0 --More-- pid 14 Tracing command intr pid 11 tid 100014 td 0x80979360 cpu_switch+6c (?,?,?,?) ra c3a57dc800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a57dc800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a57df000000020 sp 0 sz 0 80141af0+258 (?,?,?,?) ra c3a57e1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a57e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a57e8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100015 td 0x80979000 cpu_switch+6c (?,?,?,?) ra c3a5bdc800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a5bdc800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a5bdf000000020 sp 0 sz 0 80141af0+258 (?,?,?,?) ra c3a5be1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a5be5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a5be8000000000 sp 0 sz 0 pid 14 --More-- Tracing command intr pid 11 tid 100018 td 0x80978360 cpu_switch+6c (?,?,?,?) ra c3a67dc800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a67dc800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a67df000000020 sp 0 sz 0 80141af0+258 (?,?,?,?) ra c3a67e1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a67e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a67e8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100019 td 0x80978000 PC 0x80313b408008e8c0: not in kernel 0+0 (?,?,?,?) ra c3a6be8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100020 td 0x80977a20 cpu_switch+6c (?,?,?,?) ra c3a8fdc800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a8fdc800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a8fdf000000020 sp 0 sz 0 80141af0+258 (?,?,?,?) ra c3a8fe1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a8fe5800000028 sp 0 sz 0 --More-- fork_trampoline+10 (?,?,?,?) ra c3a8fe8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100026 td 0x8094a360 cpu_switch+6c (?,?,?,?) ra c3aa9dc800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3aa9dc800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3aa9df000000020 sp 0 sz 0 80141af0+258 (?,?,?,?) ra c3aa9e1000000048 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3aa9e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3aa9e8000000000 sp 0 sz 0 pid 14 Tracing command intr pid 11 tid 100027 td 0x809d66c0 PC 0x80313b408008e8c0: not in kernel 0+0 (?,?,?,?) ra c3aade8000000000 sp 0 sz 0 pid 14 Tracing command idle pid 10 tid 100003 td 0x809496c0 cpu_switch+6c (?,?,?,?) ra c3a2bdf800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a2bdf800000028 sp 0 sz 0 --More-- mi_switch+27c (?,?,?,?) ra c3a2be2000000020 sp 0 sz 0 sched_idletd+b8 (?,?,?,?) ra c3a2be4000000018 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a2be5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a2be8000000000 sp 0 sz 0 pid 14 Tracing command rand_harvestq pid 2 tid 100002 td 0x80949a20 cpu_switch+6c (?,?,?,?) ra c3a27d0800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a27d0800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a27d3000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a27d5000000028 sp 0 sz 0 sleepq_timedwait+78 (?,?,?,?) ra c3a27d7800000020 sp 0 sz 0 _sleep+404 (?,?,?,80346528) ra c3a27d9800000068 sp 0 sz 0 800c2c6c+248 (?,?,?,?) ra c3a27e0000000058 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a27e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a27e8000000000 sp 0 sz 0 pid 14 Tracing command init pid 1 tid 100001 td 0x8094a000 cpu_switch+6c (?,?,?,?) ra c3a23ab800000000 sp 0 sz 0 --More-- sched_switch+224 (?,?,?,?) ra c3a23ab800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a23ae000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a23b0000000028 sp 0 sz 0 801bf160+118 (?,?,?,?) ra c3a23b2800000030 sp 0 sz 0 sleepq_wait_sig+14 (?,?,?,?) ra c3a23b5800000020 sp 0 sz 0 _sleep+41c (?,?,?,803528f8) ra c3a23b7800000068 sp 0 sz 0 kern_wait6+6d4 (8094a000,?,?,?) ra c3a23be000000078 sp 100000000 sz 0 kern_wait+d8 (?,?,?,?) ra c3a23c58000000e0 sp 0 sz 0 sys_wait4+40 (?,?,?,?) ra c3a23d3800000088 sp 0 sz 0 trap+7f0 (?,?,?,?) ra c3a23dc0000000c0 sp 0 sz 0 MipsUserGenException+e8 (?,?,?,42e5e0) ra c3a23e8000000000 sp 0 sz 0 pid 14 Tracing command kernel pid 0 tid 100000 td 0x8040a0e0 cpu_switch+6c (?,?,?,?) ra 8049dd1800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra 8049dd1800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra 8049dd4000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra 8049dd6000000028 sp 0 sz 0 sleepq_timedwait+78 (?,?,?,?) ra 8049dd8800000020 sp 0 sz 0 _sleep+404 (?,?,?,8037ad64) ra 8049dda800000068 sp 0 sz 0 --More-- swapper+28c (?,?,?,?) ra 8049de1000000050 sp 0 sz 0 mi_startup+1f8 (?,?,?,?) ra 8049de6000000020 sp 0 sz 0 _start+70 (?,?,?,?) ra 8049de8000000000 sp 0 sz 0 pid 14 Tracing command kernel pid 0 tid 100010 td 0x80947a20 cpu_switch+6c (?,?,?,?) ra c3a47d2800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a47d2800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a47d5000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a47d7000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a47d9800000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3a47db800000068 sp 0 sz 0 taskqueue_thread_loop+114 (?,?,?,?) ra c3a47e2000000038 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a47e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a47e8000000000 sp 0 sz 0 pid 14 Tracing command kernel pid 0 tid 100013 td 0x80947000 cpu_switch+6c (?,?,?,?) ra c3a53d2800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a53d2800000028 sp 0 sz 0 --More-- mi_switch+27c (?,?,?,?) ra c3a53d5000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a53d7000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a53d9800000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3a53db800000068 sp 0 sz 0 taskqueue_thread_loop+114 (?,?,?,?) ra c3a53e2000000038 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a53e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a53e8000000000 sp 0 sz 0 pid 14 Tracing command kernel pid 0 tid 100016 td 0x80978a20 cpu_switch+6c (?,?,?,?) ra c3a5fd2800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a5fd2800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a5fd5000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a5fd7000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a5fd9800000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3a5fdb800000068 sp 0 sz 0 taskqueue_thread_loop+114 (?,?,?,?) ra c3a5fe2000000038 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a5fe5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a5fe8000000000 sp 0 sz 0 pid 14 --More-- Tracing command kernel pid 0 tid 100017 td 0x809786c0 cpu_switch+6c (?,?,?,?) ra c3a63d2800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3a63d2800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3a63d5000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3a63d7000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3a63d9800000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3a63db800000068 sp 0 sz 0 taskqueue_thread_loop+114 (?,?,?,?) ra c3a63e2000000038 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3a63e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3a63e8000000000 sp 0 sz 0 pid 14 Tracing command kernel pid 0 tid 100029 td 0x809d6000 cpu_switch+6c (?,?,?,?) ra c3ab7d2800000000 sp 0 sz 0 sched_switch+224 (?,?,?,?) ra c3ab7d2800000028 sp 0 sz 0 mi_switch+27c (?,?,?,?) ra c3ab7d5000000020 sp 0 sz 0 801beedc+1a0 (?,?,?,?) ra c3ab7d7000000028 sp 0 sz 0 sleepq_wait+78 (?,?,?,?) ra c3ab7d9800000020 sp 0 sz 0 _sleep+42c (?,?,?,80346528) ra c3ab7db800000068 sp 0 sz 0 --More-- taskqueue_thread_loop+114 (?,?,?,?) ra c3ab7e2000000038 sp 0 sz 0 fork_exit+b0 (?,?,?,?) ra c3ab7e5800000028 sp 0 sz 0 fork_trampoline+10 (?,?,?,?) ra c3ab7e8000000000 sp 0 sz 0 pid 14
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5640F315.5020303>