From owner-freebsd-stable@FreeBSD.ORG Sat Jan 3 20:51:15 2015 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D879A2BB for ; Sat, 3 Jan 2015 20:51:15 +0000 (UTC) Received: from bizet.nethelp.no (bizet.nethelp.no [195.1.209.33]) by mx1.freebsd.org (Postfix) with SMTP id E0F022BEC for ; Sat, 3 Jan 2015 20:51:12 +0000 (UTC) Received: (qmail 49315 invoked from network); 3 Jan 2015 20:51:08 -0000 Received: from bizet.nethelp.no (HELO localhost) (195.1.209.33) by bizet.nethelp.no with SMTP; 3 Jan 2015 20:51:08 -0000 Date: Sat, 03 Jan 2015 21:51:08 +0100 (CET) Message-Id: <20150103.215108.41660163.sthaug@nethelp.no> To: rodrigc@FreeBSD.org Subject: Re: 10.1-STABLE/i386 GENERIC + VIMAGE: Panic when going multiuser From: sthaug@nethelp.no In-Reply-To: References: <20150103.180210.74719874.sthaug@nethelp.no> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Jan 2015 20:51:15 -0000 > > I have a reproducible crash in 10.1-STABLE/i386, svn updated to r276174. > > The kernel is based on GENERIC with *one* line added: options VIMAGE. > > > > I can boot to single user with no problem. Going multiuser results in > > immediate crash with backtrace > > > > #0 0xc0b59be2 at kdb_backtrace+0x52 > > #1 0xc0b1b65f at panic+0x11f > > #2 0xc0dd1666 at vm_fault_hold+0x2076 > > #3 0xc0dcf5b1 at vm_fault+0x81 > > #4 0xc103008b at trap_pfault+0x1bb > > #5 0xc102f8e4 at trap+0x674 > > #6 0xc101ad0c at calltrap+0x6 > > #7 0xc0d0ca7a at sysctl_ip6_temppltime+0x3a > > #8 0xc0b28031 at sysctl_root+0x2b1 > > #9 0xc0b2862a at userland_sysctl+0x1fa > > #10 0xc0b283e8 at sys___sysctl+0x98 > > #11 0xc1030986 at syscall+0x4a6 > > #12 0xc101ada1 at Xint0x80_syscall+0x21 > > > > > That stack trace doesn't have enough info. Can you: > (1) boot into single mode on your system > (2) Add this to /etc/rc.conf: > ddb_enable="YES" > (3) Reboot the system and let it go to multi-user I found I had to also add "options DDB" to the kernel. So this is GENERIC + options VIMAGE + options DDB. I didn't get the panic when going to multiuser this time - but as soon as I did "sysctl -a" the expected panic came. Below is the result of the generated files in /var/crash (I unpacked the textdump.tar.4 file). Steinar Haug, Nethelp consulting, sthaug@nethelp.no ---------------------------------------------------------------------- info.4 Dump header from device /dev/ada0s1b Architecture: i386 Architecture Version: 1 Dump Length: 71168B (0 MB) Blocksize: 512 Dumptime: Sat Jan 3 21:26:03 2015 Hostname: verdi.nethelp.no Magic: FreeBSD Text Dump Version String: FreeBSD 10.1-STABLE #1 r276174: Sat Jan 3 20:50:10 CET 2015 root@verdi.nethelp.no:/usr/src/sys/i386/compile/GENERICV Panic String: vm_fault: fault on nofault entry, addr: cc0e5000 Dump Parity: 1182659946 Bounds: 4 Dump Status: good ---------------------------------------------------------------------- ddb.txt db:0:kdb.enter.panic> run lockinfo db:1:lockinfo> show locks No such command db:1:locks> show alllocks No such command db:1:alllocks> show lockedvnods Locked vnodes db:0:kdb.enter.panic> show pcpu cpuid = 0 dynamic pcpu = 0x3d3a00 curthread = 0xc769f310: pid 984 "sysctl" curpcb = 0xf0923d40 fpcurthread = none idlethread = 0xc6ac3000: tid 100003 "idle: cpu0" APIC ID = 0 currentldt = 0x50 curvnet = 0xc69713c0 db:0:kdb.enter.panic> bt Tracing pid 984 tid 100105 td 0xc769f310 kdb_enter(c11b2609,c11b2609,c11e064d,f09236ec,0,...) at kdb_enter+0x3d/frame 0xf09236a0 panic(c11e064d,cc0e5000,1,f0923804,f09237f4,...) at panic+0x13f/frame 0xf09236e0 vm_fault_hold(c1a6d000,cc0e5000,1,0,0,...) at vm_fault_hold+0x2076/frame 0xf0923840 vm_fault(c1a6d000,cc0e5000,1,0,c0b72331,...) at vm_fault+0x81/frame 0xf0923868 trap_pfault(cc0e5fb8,c1472000,f0923914,c6baf200,f09239a4,...) at trap_pfault+0x1bb/frame 0xf09238e8 trap(f0923a34) at trap+0x674/frame 0xf0923a28 calltrap() at calltrap+0x6/frame 0xf0923a28 --- trap 0xc, eip = 0xc1049023, esp = 0xf0923a74, ebp = 0xf0923a9c --- copyout(f0923b28,cc0e5fb8,4,f0923acc,c142c7a8,...) at copyout+0x37/frame 0xf0923a9c sysctl_ip6_temppltime(c142c7a8,c6a9e3a8,0,f0923b28,f0923b28,...) at sysctl_ip6_temppltime+0x3a/frame 0xf0923ac0 sysctl_root(f0923b28,8,2,c0a058a7,c7017850,...) at sysctl_root+0x2b1/frame 0xf0923b00 userland_sysctl(c769f310,f0923bac,4,28803030,bfbfdb70,...) at userland_sysctl+0x1fa/frame 0xf0923b70 sys___sysctl(c769f310,f0923ca8,2e,f0923c64,c0b329f2,...) at sys___sysctl+0x98/frame 0xf0923c18 syscall(f0923ce8) at syscall+0x4a6/frame 0xf0923cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf0923cdc --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x281a28b7, esp = 0xbfbfdb00, ebp = 0xbfbfdb28 --- db:0:kdb.enter.panic> ps pid ppid pgrp uid state wmesg wchan cmd 984 980 984 0 R+ CPU 0 sysctl 980 972 980 0 S+ pause 0xc6f66934 csh 979 1 979 0 Ss+ ttyin 0xc6d4a870 getty 978 1 978 0 Ss+ ttyin 0xc6d4aa70 getty 977 1 977 0 Ss+ ttyin 0xc6d4ac70 getty 976 1 976 0 Ss+ ttyin 0xc6d4ae70 getty 975 1 975 0 Ss+ ttyin 0xc6d4c070 getty 974 1 974 0 Ss+ ttyin 0xc6d4c270 getty 973 1 973 0 Ss+ ttyin 0xc6baf070 getty 972 1 972 0 Ss+ wait 0xc6f655e0 login 967 965 17 0 S+ nanslp 0xc1515641 sleep 966 1 17 0 S+ piperd 0xc70377f8 logger 965 1 17 0 S+ wait 0xc729c8d0 sh 928 1 928 0 Ss nanslp 0xc1515640 cron 924 1 924 25 Ss pause 0xc729bc24 sendmail 921 1 921 0 Ss select 0xc6fa0b24 sendmail 918 1 918 0 Ss select 0xc6fd23e4 sshd 725 1 725 0 Ss select 0xc6d57da4 syslogd 649 1 649 0 Ss select 0xc6fa1524 devd 16 0 0 0 DL vlruwt 0xc6f658d0 [vnlru] 9 0 0 0 DL syncer 0xc15349c4 [syncer] 8 0 0 0 DL (threaded) [bufdaemon] 100080 D sdflush 0xc6fb6a84 [/var worker] 100079 D sdflush 0xc6fb6884 [/usr worker] 100066 D psleep 0xc1534104 [bufdaemon] 7 0 0 0 DL pgzero 0xc1539320 [pagezero] 6 0 0 0 DL psleep 0xc1539044 [vmdaemon] 5 0 0 0 DL psleep 0xc1594604 [pagedaemon] 4 0 0 0 DL waiting_ 0xc1592874 [sctp_iterator] 3 0 0 0 DL - 0xc6baf83c [fdc0] 15 0 0 0 DL (threaded) [usb] 100053 D - 0xc6d09d34 [usbus4] 100052 D - 0xc6d09d04 [usbus4] 100051 D - 0xc6d09cd4 [usbus4] 100050 D - 0xc6d09ca4 [usbus4] 100049 D - 0xc6cf2dac [usbus3] 100048 D - 0xc6cf2d7c [usbus3] 100047 D - 0xc6cf2d4c [usbus3] 100046 D - 0xc6cf2d1c [usbus3] 100044 D - 0xc6cdbdac [usbus2] 100043 D - 0xc6cdbd7c [usbus2] 100042 D - 0xc6cdbd4c [usbus2] 100041 D - 0xc6cdbd1c [usbus2] 100039 D - 0xc6cc6dac [usbus1] 100038 D - 0xc6cc6d7c [usbus1] 100037 D - 0xc6cc6d4c [usbus1] 100036 D - 0xc6cc6d1c [usbus1] 100034 D - 0xc6ca9dac [usbus0] 100033 D - 0xc6ca9d7c [usbus0] 100032 D - 0xc6ca9d4c [usbus0] 100031 D - 0xc6ca9d1c [usbus0] 2 0 0 0 DL (threaded) [cam] 100062 D - 0xc1456928 [scanner] 100014 D - 0xc1456a80 [doneq0] 14 0 0 0 DL - 0xc14717e0 [rand_harvestq] 13 0 0 0 DL (threaded) [geom] 100011 D - 0xc158ec60 [g_down] 100010 D - 0xc158ec5c [g_up] 100009 D - 0xc158ec58 [g_event] 12 0 0 0 WL (threaded) [intr] 100058 I [irq1: atkbd0] 100057 I [swi0: uart] 100055 I [irq20: hpet0+] 100054 I [irq14: ata0] 100045 I [irq23: uhci3] 100040 I [irq18: uhci2] 100035 I [irq22: uhci1] 100030 I [irq21: uhci0 ehci0] 100029 I [irq17: bge1] 100028 I [irq16: bge0] 100018 I [swi5: fast taskq] 100016 I [swi6: Giant taskq] 100015 I [swi6: task queue] 100008 I [swi4: clock] 100007 I [swi4: clock] 100006 I [swi3: vm] 100005 I [swi1: netisr 0] 11 0 0 0 RL (threaded) [idle] 100004 Run CPU 1 [idle: cpu1] 100003 CanRun [idle: cpu0] 1 0 1 0 SLs wait 0xc6ac02f0 [init] 10 0 0 0 DL audit_wo 0xc1592f14 [audit] 0 0 0 0 DLs (threaded) [kernel] 100061 D - 0xc6ac5380 [CAM taskq] 100059 D - 0xc6d46200 [mca taskq] 100027 D - 0xc6c28e00 [em1 txq] 100026 D - 0xc6c28e80 [em1 que] 100025 D - 0xc6c29200 [em0 txq] 100024 D - 0xc6c29280 [em0 que] 100023 D - 0xc6ac4480 [acpi_task_2] 100022 D - 0xc6ac4480 [acpi_task_1] 100021 D - 0xc6ac4480 [acpi_task_0] 100020 D - 0xc6ac4500 [ffs_trim taskq] 100019 D - 0xc6ac4580 [kqueue taskq] 100017 D - 0xc6ac4700 [thread taskq] 100012 D - 0xc6ac5a80 [firmware taskq] 100000 D swapin 0xc158ece4 [swapper] db:0:kdb.enter.panic> alltrace Tracing command sysctl pid 984 tid 100105 td 0xc769f310 kdb_enter(c11b2609,c11b2609,c11e064d,f09236ec,0,...) at kdb_enter+0x3d/frame 0xf09236a0 panic(c11e064d,cc0e5000,1,f0923804,f09237f4,...) at panic+0x13f/frame 0xf09236e0 vm_fault_hold(c1a6d000,cc0e5000,1,0,0,...) at vm_fault_hold+0x2076/frame 0xf0923840 vm_fault(c1a6d000,cc0e5000,1,0,c0b72331,...) at vm_fault+0x81/frame 0xf0923868 trap_pfault(cc0e5fb8,c1472000,f0923914,c6baf200,f09239a4,...) at trap_pfault+0x1bb/frame 0xf09238e8 trap(f0923a34) at trap+0x674/frame 0xf0923a28 calltrap() at calltrap+0x6/frame 0xf0923a28 --- trap 0xc, eip = 0xc1049023, esp = 0xf0923a74, ebp = 0xf0923a9c --- copyout(f0923b28,cc0e5fb8,4,f0923acc,c142c7a8,...) at copyout+0x37/frame 0xf0923a9c sysctl_ip6_temppltime(c142c7a8,c6a9e3a8,0,f0923b28,f0923b28,...) at sysctl_ip6_temppltime+0x3a/frame 0xf0923ac0 sysctl_root(f0923b28,8,2,c0a058a7,c7017850,...) at sysctl_root+0x2b1/frame 0xf0923b00 userland_sysctl(c769f310,f0923bac,4,28803030,bfbfdb70,...) at userland_sysctl+0x1fa/frame 0xf0923b70 sys___sysctl(c769f310,f0923ca8,2e,f0923c64,c0b329f2,...) at sys___sysctl+0x98/frame 0xf0923c18 syscall(f0923ce8) at syscall+0x4a6/frame 0xf0923cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf0923cdc --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x281a28b7, esp = 0xbfbfdb00, ebp = 0xbfbfdb28 --- Tracing command csh pid 980 tid 100104 td 0xc6f69930 sched_switch(c6f69930,0,104,c0b0edad,0,...) at sched_switch+0x2d2/frame 0xf091fa7c mi_switch(104,0,c0b134a4,c0b0edad,c1a69000,c6f69930) at mi_switch+0x122/frame 0xf091fab4 sleepq_switch(c6f69930,0,c11b698a,1a8,0,...) at sleepq_switch+0x15b/frame 0xf091fadc sleepq_catch_signals(0,c11b698a,154,c6f69930,f091fba4,...) at sleepq_catch_signals+0x5be/frame 0xf091fb30 sleepq_wait_sig(c6f66934,74,c10a57b4,100,0,...) at sleepq_wait_sig+0x14/frame 0xf091fb5c _sleep(c6f66934,c6f66968,174,c10a57b4,0,...) at _sleep+0x29b/frame 0xf091fba4 kern_sigsuspend(c6f69930,0,0,0,0,...) at kern_sigsuspend+0x137/frame 0xf091fbe8 sys_sigsuspend(c6f69930,f091fca8,fffffed8,0,0,...) at sys_sigsuspend+0x58/frame 0xf091fc18 syscall(f091fce8) at syscall+0x4a6/frame 0xf091fcdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf091fcdc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x28176fbb, esp = 0xbfbfe374, ebp = 0xbfbfe3dc --- Tracing command getty pid 979 tid 100103 td 0xc6f69c40 sched_switch(c6f69c40,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xf091b984 mi_switch(104,0,c0b134a4,0,f091ba0c,c6f69c40) at mi_switch+0x122/frame 0xf091b9bc sleepq_switch(c6f69c40,0,c11b698a,1a8,c6d48200,...) at sleepq_switch+0x15b/frame 0xf091b9e4 sleepq_catch_signals(0,c11b698a,154,c6d48200,c154e760,...) at sleepq_catch_signals+0x5be/frame 0xf091ba38 sleepq_wait_sig(c6d4a870,0,c11b907b,101,0,...) at sleepq_wait_sig+0x14/frame 0xf091ba64 _cv_wait_sig(c6d4a870,c158f610,1,1,f091baec,...) at _cv_wait_sig+0x17c/frame 0xf091ba94 tty_wait(c6d4a800,c6d4a870,f091bbe8,1,0,...) at tty_wait+0x1f/frame 0xf091baac ttydisc_read(c6d4a800,f091bbe8,0,c0be7ba0,c7017578,...) at ttydisc_read+0x31c/frame 0xf091baec ttydev_read(c6d19700,f091bbe8,0,f091bb40,c6f69c40,...) at ttydev_read+0x8f/frame 0xf091bb10 devfs_read_f(c7017578,f091bbe8,c6fa9580,0,c6f69c40,...) at devfs_read_f+0xb4/frame 0xf091bb48 dofileread(c7017578,f091bbe8,ffffffff,ffffffff,0,...) at dofileread+0x9e/frame 0xf091bb80 kern_readv(c6f69c40,0,f091bbe8,0,bfbfedf7,...) at kern_readv+0x96/frame 0xf091bbc8 sys_read(c6f69c40,f091bca8,c6f66bc0,f091bc64,c0b329f2,...) at sys_read+0x5c/frame 0xf091bc18 syscall(f091bce8) at syscall+0x4a6/frame 0xf091bcdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf091bcdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281b52b7, esp = 0xbfbfeddc, ebp = 0xbfbfee08 --- Tracing command getty pid 978 tid 100102 td 0xc71b8000 sched_switch(c71b8000,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xf0917984 mi_switch(104,0,c0b134a4,0,f0917a0c,c71b8000) at mi_switch+0x122/frame 0xf09179bc sleepq_switch(c71b8000,0,c11b698a,1a8,c6d48400,...) at sleepq_switch+0x15b/frame 0xf09179e4 sleepq_catch_signals(0,c11b698a,154,c6d48400,c154e760,...) at sleepq_catch_signals+0x5be/frame 0xf0917a38 sleepq_wait_sig(c6d4aa70,0,c11b907b,101,0,...) at sleepq_wait_sig+0x14/frame 0xf0917a64 _cv_wait_sig(c6d4aa70,c158f610,1,1,f0917aec,...) at _cv_wait_sig+0x17c/frame 0xf0917a94 tty_wait(c6d4aa00,c6d4aa70,f0917be8,1,0,...) at tty_wait+0x1f/frame 0xf0917aac ttydisc_read(c6d4aa00,f0917be8,0,c0be7ba0,c7017508,...) at ttydisc_read+0x31c/frame 0xf0917aec ttydev_read(c6d19800,f0917be8,0,f0917b40,c71b8000,...) at ttydev_read+0x8f/frame 0xf0917b10 devfs_read_f(c7017508,f0917be8,c74b4380,0,c71b8000,...) at devfs_read_f+0xb4/frame 0xf0917b48 dofileread(c7017508,f0917be8,ffffffff,ffffffff,0,...) at dofileread+0x9e/frame 0xf0917b80 kern_readv(c71b8000,0,f0917be8,0,bfbfedf7,...) at kern_readv+0x96/frame 0xf0917bc8 sys_read(c71b8000,f0917ca8,c729b000,c71b8000,c71bac40,...) at sys_read+0x5c/frame 0xf0917c18 syscall(f0917ce8) at syscall+0x4a6/frame 0xf0917cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf0917cdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281b52b7, esp = 0xbfbfeddc, ebp = 0xbfbfee08 --- Tracing command getty pid 977 tid 100101 td 0xc71b8310 sched_switch(c71b8310,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xf0913984 mi_switch(104,0,c0b134a4,0,f0913a0c,c71b8310) at mi_switch+0x122/frame 0xf09139bc sleepq_switch(c71b8310,0,c11b698a,1a8,c6d48600,...) at sleepq_switch+0x15b/frame 0xf09139e4 sleepq_catch_signals(0,c11b698a,154,c6d48600,c154e760,...) at sleepq_catch_signals+0x5be/frame 0xf0913a38 sleepq_wait_sig(c6d4ac70,0,c11b907b,101,0,...) at sleepq_wait_sig+0x14/frame 0xf0913a64 _cv_wait_sig(c6d4ac70,c158f610,1,1,f0913aec,...) at _cv_wait_sig+0x17c/frame 0xf0913a94 tty_wait(c6d4ac00,c6d4ac70,f0913be8,1,0,...) at tty_wait+0x1f/frame 0xf0913aac ttydisc_read(c6d4ac00,f0913be8,0,0,0,...) at ttydisc_read+0x31c/frame 0xf0913aec ttydev_read(c6d19900,f0913be8,0,f0913b40,c71b8310,...) at ttydev_read+0x8f/frame 0xf0913b10 devfs_read_f(c7017690,f0913be8,c74b4400,0,c71b8310,...) at devfs_read_f+0xb4/frame 0xf0913b48 dofileread(c7017690,f0913be8,ffffffff,ffffffff,0,...) at dofileread+0x9e/frame 0xf0913b80 kern_readv(c71b8310,0,f0913be8,0,bfbfedf7,...) at kern_readv+0x96/frame 0xf0913bc8 sys_read(c71b8310,f0913ca8,c729b2f0,f0913c64,c0b329f2,...) at sys_read+0x5c/frame 0xf0913c18 syscall(f0913ce8) at syscall+0x4a6/frame 0xf0913cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf0913cdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281b52b7, esp = 0xbfbfeddc, ebp = 0xbfbfee08 --- Tracing command getty pid 976 tid 100085 td 0xc72cc620 sched_switch(c72cc620,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xf0895984 mi_switch(104,0,c0b134a4,0,f0895a0c,c72cc620) at mi_switch+0x122/frame 0xf08959bc sleepq_switch(c72cc620,0,c11b698a,1a8,c6d48800,...) at sleepq_switch+0x15b/frame 0xf08959e4 sleepq_catch_signals(0,c11b698a,154,c6d48800,c154e760,...) at sleepq_catch_signals+0x5be/frame 0xf0895a38 sleepq_wait_sig(c6d4ae70,0,c11b907b,101,0,...) at sleepq_wait_sig+0x14/frame 0xf0895a64 _cv_wait_sig(c6d4ae70,c158f610,1,1,f0895aec,...) at _cv_wait_sig+0x17c/frame 0xf0895a94 tty_wait(c6d4ae00,c6d4ae70,f0895be8,1,0,...) at tty_wait+0x1f/frame 0xf0895aac ttydisc_read(c6d4ae00,f0895be8,0,c0be7ba0,c702e038,...) at ttydisc_read+0x31c/frame 0xf0895aec ttydev_read(c6d19a00,f0895be8,0,f0895b40,c72cc620,...) at ttydev_read+0x8f/frame 0xf0895b10 devfs_read_f(c702e038,f0895be8,c74b4480,0,c72cc620,...) at devfs_read_f+0xb4/frame 0xf0895b48 dofileread(c702e038,f0895be8,ffffffff,ffffffff,0,...) at dofileread+0x9e/frame 0xf0895b80 kern_readv(c72cc620,0,f0895be8,0,bfbfedf7,...) at kern_readv+0x96/frame 0xf0895bc8 sys_read(c72cc620,f0895ca8,c72c2000,f0895c64,c0b329f2,...) at sys_read+0x5c/frame 0xf0895c18 syscall(f0895ce8) at syscall+0x4a6/frame 0xf0895cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf0895cdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281b52b7, esp = 0xbfbfeddc, ebp = 0xbfbfee08 --- Tracing command getty pid 975 tid 100094 td 0xc71bac40 sched_switch(c71bac40,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xf08c8984 mi_switch(104,0,c0b134a4,0,f08c8a0c,c71bac40) at mi_switch+0x122/frame 0xf08c89bc sleepq_switch(c71bac40,0,c11b698a,1a8,c6d48a00,...) at sleepq_switch+0x15b/frame 0xf08c89e4 sleepq_catch_signals(0,c11b698a,154,c6d48a00,c154e760,...) at sleepq_catch_signals+0x5be/frame 0xf08c8a38 sleepq_wait_sig(c6d4c070,0,c11b907b,101,0,...) at sleepq_wait_sig+0x14/frame 0xf08c8a64 _cv_wait_sig(c6d4c070,c158f610,1,1,f08c8aec,...) at _cv_wait_sig+0x17c/frame 0xf08c8a94 tty_wait(c6d4c000,c6d4c070,f08c8be8,1,0,...) at tty_wait+0x1f/frame 0xf08c8aac ttydisc_read(c6d4c000,f08c8be8,0,c0be7ba0,c70175b0,...) at ttydisc_read+0x31c/frame 0xf08c8aec ttydev_read(c6d19b00,f08c8be8,0,f08c8b40,c71bac40,...) at ttydev_read+0x8f/frame 0xf08c8b10 devfs_read_f(c70175b0,f08c8be8,c6fa9500,0,c71bac40,...) at devfs_read_f+0xb4/frame 0xf08c8b48 dofileread(c70175b0,f08c8be8,ffffffff,ffffffff,0,...) at dofileread+0x9e/frame 0xf08c8b80 kern_readv(c71bac40,0,f08c8be8,0,bfbfedf7,...) at kern_readv+0x96/frame 0xf08c8bc8 sys_read(c71bac40,f08c8ca8,c72c05e0,1,c6f67930,...) at sys_read+0x5c/frame 0xf08c8c18 syscall(f08c8ce8) at syscall+0x4a6/frame 0xf08c8cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf08c8cdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281b52b7, esp = 0xbfbfeddc, ebp = 0xbfbfee08 --- Tracing command getty pid 974 tid 100075 td 0xc6f67930 sched_switch(c6f67930,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xee3bf984 mi_switch(104,0,c0b134a4,0,ee3bfa0c,c6f67930) at mi_switch+0x122/frame 0xee3bf9bc sleepq_switch(c6f67930,0,c11b698a,1a8,c6d48c00,...) at sleepq_switch+0x15b/frame 0xee3bf9e4 sleepq_catch_signals(0,c11b698a,154,c6d48c00,c154e760,...) at sleepq_catch_signals+0x5be/frame 0xee3bfa38 sleepq_wait_sig(c6d4c270,0,c11b907b,101,0,...) at sleepq_wait_sig+0x14/frame 0xee3bfa64 _cv_wait_sig(c6d4c270,c158f610,1,1,ee3bfaec,...) at _cv_wait_sig+0x17c/frame 0xee3bfa94 tty_wait(c6d4c200,c6d4c270,ee3bfbe8,1,0,...) at tty_wait+0x1f/frame 0xee3bfaac ttydisc_read(c6d4c200,ee3bfbe8,0,c0be7ba0,c7017540,...) at ttydisc_read+0x31c/frame 0xee3bfaec ttydev_read(c6d19c00,ee3bfbe8,0,ee3bfb40,c6f67930,...) at ttydev_read+0x8f/frame 0xee3bfb10 devfs_read_f(c7017540,ee3bfbe8,c6fa9480,0,c6f67930,...) at devfs_read_f+0xb4/frame 0xee3bfb48 dofileread(c7017540,ee3bfbe8,ffffffff,ffffffff,0,...) at dofileread+0x9e/frame 0xee3bfb80 kern_readv(c6f67930,0,ee3bfbe8,0,bfbfedf7,...) at kern_readv+0x96/frame 0xee3bfbc8 sys_read(c6f67930,ee3bfca8,c6f642f0,1,c72cc620,...) at sys_read+0x5c/frame 0xee3bfc18 syscall(ee3bfce8) at syscall+0x4a6/frame 0xee3bfcdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3bfcdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281b52b7, esp = 0xbfbfeddc, ebp = 0xbfbfee08 --- Tracing command getty pid 973 tid 100076 td 0xc6f67620 sched_switch(c6f67620,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xee3c2984 mi_switch(104,0,c0b134a4,0,ee3c2a0c,c6f67620) at mi_switch+0x122/frame 0xee3c29bc sleepq_switch(c6f67620,0,c11b698a,1a8,c6bafe00,...) at sleepq_switch+0x15b/frame 0xee3c29e4 sleepq_catch_signals(0,c11b698a,154,c6bafe00,c154e760,...) at sleepq_catch_signals+0x5be/frame 0xee3c2a38 sleepq_wait_sig(c6baf070,0,c11b907b,101,0,...) at sleepq_wait_sig+0x14/frame 0xee3c2a64 _cv_wait_sig(c6baf070,c158f610,1,1,ee3c2aec,...) at _cv_wait_sig+0x17c/frame 0xee3c2a94 tty_wait(c6baf000,c6baf070,ee3c2be8,1,0,...) at tty_wait+0x1f/frame 0xee3c2aac ttydisc_read(c6baf000,ee3c2be8,0,c0be7ba0,c70174d0,...) at ttydisc_read+0x31c/frame 0xee3c2aec ttydev_read(c6d19d00,ee3c2be8,0,ee3c2b40,c6f67620,...) at ttydev_read+0x8f/frame 0xee3c2b10 devfs_read_f(c70174d0,ee3c2be8,c6fa9400,0,c6f67620,...) at devfs_read_f+0xb4/frame 0xee3c2b48 dofileread(c70174d0,ee3c2be8,ffffffff,ffffffff,0,...) at dofileread+0x9e/frame 0xee3c2b80 kern_readv(c6f67620,0,ee3c2be8,0,bfbfedf7,...) at kern_readv+0x96/frame 0xee3c2bc8 sys_read(c6f67620,ee3c2ca8,c6f64000,1,c0b329f2,...) at sys_read+0x5c/frame 0xee3c2c18 syscall(ee3c2ce8) at syscall+0x4a6/frame 0xee3c2cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3c2cdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281b52b7, esp = 0xbfbfeddc, ebp = 0xbfbfee08 --- Tracing command login pid 972 tid 100069 td 0xc6f68c40 sched_switch(c6f68c40,0,104,c0dea0cc,0,...) at sched_switch+0x2d2/frame 0xee3aa960 mi_switch(104,0,c315c7c4,3,2,c6f68c40) at mi_switch+0x122/frame 0xee3aa998 sleepq_switch(c6f68c40,0,c11b698a,1a8,0,...) at sleepq_switch+0x15b/frame 0xee3aa9c0 sleepq_catch_signals(0,c11b698a,154,c6f68c40,ee3aaa44,...) at sleepq_catch_signals+0x5be/frame 0xee3aaa14 sleepq_wait_sig(c6f655e0,6c,c11b77f6,100,0,...) at sleepq_wait_sig+0x14/frame 0xee3aaa40 _sleep(c6f655e0,c6f65678,16c,c11b77f6,0,...) at _sleep+0x29b/frame 0xee3aaa88 kern_wait6(c6f68c40,0,3d4,0,ee3aab24,...) at kern_wait6+0x725/frame 0xee3aaaf8 sys_wait4(c6f68c40,ee3aaca8,c6f655e0,4,c088707e,...) at sys_wait4+0x94/frame 0xee3aac18 syscall(ee3aace8) at syscall+0x4a6/frame 0xee3aacdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3aacdc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2811c143, esp = 0xbfbfed20, ebp = 0xbfbfed38 --- Tracing command sleep pid 967 tid 100077 td 0xc6f67310 sched_switch(c6f67310,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xee3caa4c mi_switch(104,0,c0b40637,1,c7293444,c6f67310) at mi_switch+0x122/frame 0xee3caa84 sleepq_switch(c6f67310,0,c11b698a,1a8,4a,...) at sleepq_switch+0x15b/frame 0xee3caaac sleepq_catch_signals(c0000000,3,c0b72670,c6f67310,1,...) at sleepq_catch_signals+0x5be/frame 0xee3cab00 sleepq_timedwait_sig(c1515641,6c,4a,c0000000,3,...) at sleepq_timedwait_sig+0x14/frame 0xee3cab30 _sleep(c1515641,0,16c,c11b32c4,a1dea2f,...) at _sleep+0x26b/frame 0xee3cab78 kern_nanosleep(bfbfee40,ee3cabf0,ee3cabf8,ee3cace8,3c,...) at kern_nanosleep+0x14b/frame 0xee3cabd8 sys_nanosleep(c6f67310,ee3caca8,c6ac0bc0,c6f67310,c71b9930,...) at sys_nanosleep+0x69/frame 0xee3cac18 syscall(ee3cace8) at syscall+0x4a6/frame 0xee3cacdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3cacdc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2817b203, esp = 0xbfbfee0c, ebp = 0xbfbfee54 --- Tracing command logger pid 966 tid 100074 td 0xc6f67c40 sched_switch(c6f67c40,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xee3bb9dc mi_switch(104,0,c0b134a4,0,0,c6f67c40) at mi_switch+0x122/frame 0xee3bba14 sleepq_switch(c6f67c40,0,c11b698a,1a8,ee3bbaa0,...) at sleepq_switch+0x15b/frame 0xee3bba3c sleepq_catch_signals(0,c11b698a,154,0,0,...) at sleepq_catch_signals+0x5be/frame 0xee3bba90 sleepq_wait_sig(c70377f8,5c,c11b7397,100,0,...) at sleepq_wait_sig+0x14/frame 0xee3bbabc _sleep(c70377f8,c7037978,15c,c11b7397,0,...) at _sleep+0x29b/frame 0xee3bbb04 pipe_read(c70175e8,ee3bbbe8,c6ff5c80,0,c6f67c40,...) at pipe_read+0x33a/frame 0xee3bbb48 dofileread(c70175e8,ee3bbbe8,ffffffff,ffffffff,0,...) at dofileread+0x9e/frame 0xee3bbb80 kern_readv(c6f67c40,0,ee3bbbe8,0,28808000,...) at kern_readv+0x96/frame 0xee3bbbc8 sys_read(c6f67c40,ee3bbca8,c6f645e0,1,c6f67310,...) at sys_read+0x5c/frame 0xee3bbc18 syscall(ee3bbce8) at syscall+0x4a6/frame 0xee3bbcdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3bbcdc --- syscall (3, FreeBSD ELF32, sys_read), eip = 0x281a02b7, esp = 0xbfbfe958, ebp = 0xbfbfe96c --- Tracing command sh pid 965 tid 100081 td 0xc71b9930 sched_switch(c71b9930,0,104,ee3e19d0,0,...) at sched_switch+0x2d2/frame 0xee3e1960 mi_switch(104,0,c7294818,0,0,c71b9930) at mi_switch+0x122/frame 0xee3e1998 sleepq_switch(c71b9930,0,c11b698a,1a8,c71b9930,...) at sleepq_switch+0x15b/frame 0xee3e19c0 sleepq_catch_signals(0,c11b698a,154,c71b9930,ee3e1a44,...) at sleepq_catch_signals+0x5be/frame 0xee3e1a14 sleepq_wait_sig(c729c8d0,6c,c11b77f6,100,0,...) at sleepq_wait_sig+0x14/frame 0xee3e1a40 _sleep(c729c8d0,c729c968,16c,c11b77f6,0,...) at _sleep+0x29b/frame 0xee3e1a88 kern_wait6(c71b9930,7,0,0,ee3e1b24,...) at kern_wait6+0x725/frame 0xee3e1af8 sys_wait4(c71b9930,ee3e1ca8,fffffed8,0,0,...) at sys_wait4+0x94/frame 0xee3e1c18 syscall(ee3e1ce8) at syscall+0x4a6/frame 0xee3e1cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3e1cdc --- syscall (7, FreeBSD ELF32, sys_wait4), eip = 0x2815d143, esp = 0xbfbfdd48, ebp = 0xbfbfdd60 --- Tracing command cron pid 928 tid 100100 td 0xc71b8620 sched_switch(c71b8620,0,104,c0b4103a,f0908a70,...) at sched_switch+0x2d2/frame 0xf0908a4c mi_switch(104,0,c0b40637,0,0,c71b8620) at mi_switch+0x122/frame 0xf0908a84 sleepq_switch(c71b8620,0,c11b698a,1a8,6b,...) at sleepq_switch+0x15b/frame 0xf0908aac sleepq_catch_signals(bfd19a33,3,c0b72670,c71b8620,0,...) at sleepq_catch_signals+0x5be/frame 0xf0908b00 sleepq_timedwait_sig(c1515640,6c,6b,bfd19a33,3,...) at sleepq_timedwait_sig+0x14/frame 0xf0908b30 _sleep(c1515640,0,16c,c11b32c4,3380bb39,...) at _sleep+0x26b/frame 0xf0908b78 kern_nanosleep(bfbfee28,f0908bf0,f0908bf8,f0908ce8,3b,...) at kern_nanosleep+0x14b/frame 0xf0908bd8 sys_nanosleep(c71b8620,f0908ca8,c729b5e0,1,0,...) at sys_nanosleep+0x69/frame 0xf0908c18 syscall(f0908ce8) at syscall+0x4a6/frame 0xf0908cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf0908cdc --- syscall (240, FreeBSD ELF32, sys_nanosleep), eip = 0x2819f203, esp = 0xbfbfecb4, ebp = 0xbfbfee44 --- Tracing command sendmail pid 924 tid 100098 td 0xc71b8c40 sched_switch(c71b8c40,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xf08e5a7c mi_switch(104,0,c0b134a4,0,0,c71b8c40) at mi_switch+0x122/frame 0xf08e5ab4 sleepq_switch(c71b8c40,0,c11b698a,1a8,0,...) at sleepq_switch+0x15b/frame 0xf08e5adc sleepq_catch_signals(0,c11b698a,154,c71b8c40,f08e5ba4,...) at sleepq_catch_signals+0x5be/frame 0xf08e5b30 sleepq_wait_sig(c729bc24,74,c10a57b4,100,0,...) at sleepq_wait_sig+0x14/frame 0xf08e5b5c _sleep(c729bc24,c729bc58,174,c10a57b4,0,...) at _sleep+0x29b/frame 0xf08e5ba4 kern_sigsuspend(c71b8c40,0,0,0,0,...) at kern_sigsuspend+0x137/frame 0xf08e5be8 sys_sigsuspend(c71b8c40,f08e5ca8,c729bbc0,c71b8c40,c72cc930,...) at sys_sigsuspend+0x58/frame 0xf08e5c18 syscall(f08e5ce8) at syscall+0x4a6/frame 0xf08e5cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf08e5cdc --- syscall (4, FreeBSD ELF32, sys_write), eip = 0x2836bfbb, esp = 0xbfbfd9cc, ebp = 0xbfbfd9f8 --- Tracing command sendmail pid 921 tid 100099 td 0xc71b8930 sched_switch(c71b8930,0,104,ac,0,...) at sched_switch+0x2d2/frame 0xf08e88d8 mi_switch(104,0,f08e898c,c0b134a4,2845a000,c71b8930) at mi_switch+0x122/frame 0xf08e8910 sleepq_switch(c71b8930,0,c11b698a,1a8,f11734f8,...) at sleepq_switch+0x15b/frame 0xf08e8938 sleepq_catch_signals(50000000,0,c0b72670,c71b8930,1,...) at sleepq_catch_signals+0x5be/frame 0xf08e898c sleepq_timedwait_sig(c6fa0b24,0,30,50000000,0,...) at sleepq_timedwait_sig+0x14/frame 0xf08e89bc _cv_timedwait_sig_sbt(c6fa0b24,c6fa0b10,f11734f8,30,50000000,...) at _cv_timedwait_sig_sbt+0x1a7/frame 0xf08e89f0 seltdwait(30,50000000,0,c71b8930,f08e8ba0,...) at seltdwait+0xc1/frame 0xf08e8a28 kern_select(c71b8930,4,bfbfcfb8,0,0,...) at kern_select+0x8c2/frame 0xf08e8bd0 sys_select(c71b8930,f08e8ca8,c729b8d0,4,c088707e,...) at sys_select+0x69/frame 0xf08e8c18 syscall(f08e8ce8) at syscall+0x4a6/frame 0xf08e8cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf08e8cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28444277, esp = 0xbfbfcf1c, ebp = 0xbfbfd9d8 --- Tracing command sshd pid 918 tid 100072 td 0xc6f68310 sched_switch(c6f68310,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xee3b48e0 mi_switch(104,0,0,0,40a,c6f68310) at mi_switch+0x122/frame 0xee3b4918 sleepq_switch(c6f68310,0,c11b698a,1a8,0,...) at sleepq_switch+0x15b/frame 0xee3b4940 sleepq_catch_signals(0,c11b698a,154,ee3b4ba8,ee3b49d4,...) at sleepq_catch_signals+0x5be/frame 0xee3b4994 sleepq_wait_sig(c6fd23e4,0,c11ecb42,101,0,...) at sleepq_wait_sig+0x14/frame 0xee3b49c0 _cv_wait_sig(c6fd23e4,c6fd23d0,c702dcb0,c6f68310,ee3b4a28,...) at _cv_wait_sig+0x17c/frame 0xee3b49f0 seltdwait(ffffffff,0,0,c6f68310,ee3b4ba0,...) at seltdwait+0xcf/frame 0xee3b4a28 kern_select(c6f68310,5,28c0c158,0,0,...) at kern_select+0x8c2/frame 0xee3b4bd0 sys_select(c6f68310,ee3b4ca8,c6f64bc0,c6f68310,c72cc930,...) at sys_select+0x69/frame 0xee3b4c18 syscall(ee3b4ce8) at syscall+0x4a6/frame 0xee3b4cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3b4cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x28636277, esp = 0xbfbfe43c, ebp = 0xbfbfe460 --- Tracing command syslogd pid 725 tid 100084 td 0xc72cc930 sched_switch(c72cc930,0,104,c152191c,0,...) at sched_switch+0x2d2/frame 0xee3fb8e0 mi_switch(104,0,ee3fb950,c0af7134,c6d23930,c72cc930) at mi_switch+0x122/frame 0xee3fb918 sleepq_switch(c72cc930,0,c11b698a,1a8,c6aa5e3c,...) at sleepq_switch+0x15b/frame 0xee3fb940 sleepq_catch_signals(0,c11b698a,154,ee3fba28,c1036475,...) at sleepq_catch_signals+0x5be/frame 0xee3fb994 sleepq_wait_sig(c6d57da4,0,c11ecb42,101,0,...) at sleepq_wait_sig+0x14/frame 0xee3fb9c0 _cv_wait_sig(c6d57da4,c6d57d90,c0a05c9b,c69a4000,0,...) at _cv_wait_sig+0x17c/frame 0xee3fb9f0 seltdwait(ffffffff,0,0,c72cc930,ee3fbba0,...) at seltdwait+0xcf/frame 0xee3fba28 kern_select(c72cc930,9,28834118,0,0,...) at kern_select+0x8c2/frame 0xee3fbbd0 sys_select(c72cc930,ee3fbca8,804c210,c74c5000,0,...) at sys_select+0x69/frame 0xee3fbc18 syscall(ee3fbce8) at syscall+0x4a6/frame 0xee3fbcdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3fbcdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x281b8277, esp = 0xbfbfdea4, ebp = 0xbfbfee34 --- Tracing command devd pid 649 tid 100071 td 0xc6f68620 sched_switch(c6f68620,0,104,ac,98,...) at sched_switch+0x2d2/frame 0xee3b18d8 mi_switch(104,0,0,c0b134a4,0,c6f68620) at mi_switch+0x122/frame 0xee3b1910 sleepq_switch(c6f68620,0,c11b698a,1a8,4127f8c6,...) at sleepq_switch+0x15b/frame 0xee3b1938 sleepq_catch_signals(c0000000,3,c0b72670,c6f68620,0,...) at sleepq_catch_signals+0x5be/frame 0xee3b198c sleepq_timedwait_sig(c6fa1524,0,48,c0000000,3,...) at sleepq_timedwait_sig+0x14/frame 0xee3b19bc _cv_timedwait_sig_sbt(c6fa1524,c6fa1510,4127f8c6,48,c0000000,...) at _cv_timedwait_sig_sbt+0x1a7/frame 0xee3b19f0 seltdwait(48,c0000000,3,c6f68620,ee3b1ba0,...) at seltdwait+0xc1/frame 0xee3b1a28 kern_select(c6f68620,6,bfbfcda0,0,0,...) at kern_select+0x8c2/frame 0xee3b1bd0 sys_select(c6f68620,ee3b1ca8,c6f65000,4,c088707e,...) at sys_select+0x69/frame 0xee3b1c18 syscall(ee3b1ce8) at syscall+0x4a6/frame 0xee3b1cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xee3b1cdc --- syscall (93, FreeBSD ELF32, sys_select), eip = 0x8097703, esp = 0xbfbfcd3c, ebp = 0xbfbfee54 --- Tracing command vnlru pid 16 tid 100068 td 0xc6f69000 sched_switch(c6f69000,0,104,0,36254e5d,...) at sched_switch+0x2d2/frame 0xee3a6b5c mi_switch(104,0,101,c6f69000,ee3a6be8,c6f658d0) at mi_switch+0x122/frame 0xee3a6b94 sleepq_switch(c6f69000,0,c11b698a,28b,101,...) at sleepq_switch+0x15b/frame 0xee3a6bbc sleepq_timedwait(c6f658d0,60,0,0,0,...) at sleepq_timedwait+0x3f/frame 0xee3a6be8 _sleep(c6f658d0,c1534980,260,c11bd97c,fffffed8,...) at _sleep+0x282/frame 0xee3a6c30 vnlru_proc(0,ee3a6ce8,0,0,0,...) at vnlru_proc+0xcf/frame 0xee3a6ca4 fork_exit(c0bdb7e0,0,ee3a6ce8) at fork_exit+0xa3/frame 0xee3a6cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee3a6cd4 --- trap 0, eip = 0, esp = 0xee3a6d20, ebp = 0 --- Tracing command syncer pid 9 tid 100067 td 0xc6f69310 sched_switch(c6f69310,0,104,36254e5d,0,...) at sched_switch+0x2d2/frame 0xee3a3b68 mi_switch(104,0,101,c6f69310,ee3a3bf4,c15349c4) at mi_switch+0x122/frame 0xee3a3ba0 sleepq_switch(c6f69310,0,c11b698a,28b,101,...) at sleepq_switch+0x15b/frame 0xee3a3bc8 sleepq_timedwait(c15349c4,0,0,0,0,...) at sleepq_timedwait+0x3f/frame 0xee3a3bf4 _cv_timedwait_sbt(c15349c4,c15349a4,fffffed8,0,0,...) at _cv_timedwait_sbt+0x1a7/frame 0xee3a3c28 sched_sync(0,ee3a3ce8,0,0,0,...) at sched_sync+0x7a1/frame 0xee3a3ca4 fork_exit(c0bdaea0,0,ee3a3ce8) at fork_exit+0xa3/frame 0xee3a3cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee3a3cd4 --- trap 0, eip = 0, esp = 0xee3a3d20, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100080 td 0xc71b9c40 sched_switch(c71b9c40,0,104,0,e9db9457,...) at sched_switch+0x2d2/frame 0xee3d3b8c mi_switch(104,0,101,c71b9c40,ee3d3c18,c6fb6a84) at mi_switch+0x122/frame 0xee3d3bc4 sleepq_switch(c71b9c40,0,c11b698a,28b,101,...) at sleepq_switch+0x15b/frame 0xee3d3bec sleepq_timedwait(c6fb6a84,54,0,0,0,...) at sleepq_timedwait+0x3f/frame 0xee3d3c18 _sleep(c6fb6a84,c6fb6a00,54,c11de1de,7fffff6c,...) at _sleep+0x282/frame 0xee3d3c60 softdep_flush(c70f7000,ee3d3ce8,c0da9070,0,ee3d3cd0,...) at softdep_flush+0x16a/frame 0xee3d3ca4 fork_exit(c0da9070,c70f7000,ee3d3ce8) at fork_exit+0xa3/frame 0xee3d3cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee3d3cd4 --- trap 0, eip = 0, esp = 0xee3d3d20, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100079 td 0xc71ba000 sched_switch(c71ba000,0,104,0,d041fadb,...) at sched_switch+0x2d2/frame 0xee3d0b8c mi_switch(104,0,101,c71ba000,ee3d0c18,c6fb6884) at mi_switch+0x122/frame 0xee3d0bc4 sleepq_switch(c71ba000,0,c11b698a,28b,101,...) at sleepq_switch+0x15b/frame 0xee3d0bec sleepq_timedwait(c6fb6884,54,0,0,0,...) at sleepq_timedwait+0x3f/frame 0xee3d0c18 _sleep(c6fb6884,c6fb6800,54,c11de1de,7fffff6c,...) at _sleep+0x282/frame 0xee3d0c60 softdep_flush(c70f72a0,ee3d0ce8,c0da9070,0,ee3d0cd0,...) at softdep_flush+0x16a/frame 0xee3d0ca4 fork_exit(c0da9070,c70f72a0,ee3d0ce8) at fork_exit+0xa3/frame 0xee3d0cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee3d0cd4 --- trap 0, eip = 0, esp = 0xee3d0d20, ebp = 0 --- Tracing command bufdaemon pid 8 tid 100066 td 0xc6f69620 sched_switch(c6f69620,0,104,36254e5d,0,...) at sched_switch+0x2d2/frame 0xee3a0b98 mi_switch(104,0,101,c6f69620,ee3a0c24,c1534104) at mi_switch+0x122/frame 0xee3a0bd0 sleepq_switch(c6f69620,0,c11b698a,28b,101,...) at sleepq_switch+0x15b/frame 0xee3a0bf8 sleepq_timedwait(c1534104,54,0,0,0,...) at sleepq_timedwait+0x3f/frame 0xee3a0c24 _sleep(c1534104,c1534080,54,c11bb3e7,fffffed8,...) at _sleep+0x282/frame 0xee3a0c6c buf_daemon(0,ee3a0ce8,0,0,0,...) at buf_daemon+0xac/frame 0xee3a0ca4 fork_exit(c0bbeac0,0,ee3a0ce8) at fork_exit+0xa3/frame 0xee3a0cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee3a0cd4 --- trap 0, eip = 0, esp = 0xee3a0d20, ebp = 0 --- Tracing command pagezero pid 7 tid 100065 td 0xc6cee620 sched_switch(c6cee620,0,104,79b9a83d,0,...) at sched_switch+0x2d2/frame 0xee39dba0 mi_switch(104,0,101,c6cee620,ee39dc2c,c1539320) at mi_switch+0x122/frame 0xee39dbd8 sleepq_switch(c6cee620,0,c11b698a,28b,101,...) at sleepq_switch+0x15b/frame 0xee39dc00 sleepq_timedwait(c1539320,0,12b,0,0,...) at sleepq_timedwait+0x3f/frame 0xee39dc2c _sleep(c1539320,c1594580,0,c11e2463,fffea520,...) at _sleep+0x282/frame 0xee39dc74 vm_pagezero(0,ee39dce8,0,0,0,...) at vm_pagezero+0xd2/frame 0xee39dca4 fork_exit(c0e09ef0,0,ee39dce8) at fork_exit+0xa3/frame 0xee39dcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee39dcd4 --- trap 0, eip = 0, esp = 0xee39dd20, ebp = 0 --- Tracing command vmdaemon pid 6 tid 100064 td 0xc6cee930 sched_switch(c6cee930,0,104,0,c6cee930,...) at sched_switch+0x2d2/frame 0xee39ab64 mi_switch(104,0,0,0,c6cee930,c1539044) at mi_switch+0x122/frame 0xee39ab9c sleepq_switch(c6cee930,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee39abc4 sleepq_wait(c1539044,74,c11bb3e7,0,0,...) at sleepq_wait+0x3f/frame 0xee39abf0 _sleep(c1539044,c153902c,74,c11bb3e7,0,...) at _sleep+0x2ae/frame 0xee39ac38 vm_daemon(0,ee39ace8,0,0,0,...) at vm_daemon+0x116/frame 0xee39aca4 fork_exit(c0e029a0,0,ee39ace8) at fork_exit+0xa3/frame 0xee39acd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee39acd4 --- trap 0, eip = 0, esp = 0xee39ad20, ebp = 0 --- Tracing command pagedaemon pid 5 tid 100063 td 0xc6ceec40 sched_switch(c6ceec40,0,104,0,36254e5d,...) at sched_switch+0x2d2/frame 0xee397b64 mi_switch(104,0,101,c6ceec40,ee397bf0,c1594604) at mi_switch+0x122/frame 0xee397b9c sleepq_switch(c6ceec40,0,c11b698a,28b,101,...) at sleepq_switch+0x15b/frame 0xee397bc4 sleepq_timedwait(c1594604,54,0,0,0,...) at sleepq_timedwait+0x3f/frame 0xee397bf0 _sleep(c1594604,c1594580,54,c11bb3e7,fffffed8,...) at _sleep+0x282/frame 0xee397c38 vm_pageout(0,ee397ce8,0,0,0,...) at vm_pageout+0x1fb/frame 0xee397ca4 fork_exit(c0e03410,0,ee397ce8) at fork_exit+0xa3/frame 0xee397cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee397cd4 --- trap 0, eip = 0, esp = 0xee397d20, ebp = 0 --- Tracing command sctp_iterator pid 4 tid 100060 td 0xc6d22620 sched_switch(c6d22620,0,104,0,c6d22620,...) at sched_switch+0x2d2/frame 0xee38eba4 mi_switch(104,0,d6fa6fc8,c0b0eedf,c6d22620,c1592874) at mi_switch+0x122/frame 0xee38ebdc sleepq_switch(c6d22620,0,c11b698a,268,c6fa6fc8,...) at sleepq_switch+0x15b/frame 0xee38ec04 sleepq_wait(c1592874,0,c11ce22b,0,0,...) at sleepq_wait+0x3f/frame 0xee38ec30 _sleep(c1592874,c159283c,0,c11ce22b,0,...) at _sleep+0x2ae/frame 0xee38ec78 sctp_iterator_thread(0,ee38ece8,0,0,0,...) at sctp_iterator_thread+0x9c/frame 0xee38eca4 fork_exit(c0c83920,0,ee38ece8) at fork_exit+0xa3/frame 0xee38ecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee38ecd4 --- trap 0, eip = 0, esp = 0xee38ed20, ebp = 0 --- Tracing command fdc0 pid 3 tid 100056 td 0xc6d23310 sched_switch(c6d23310,0,104,0,36254e5d,...) at sched_switch+0x2d2/frame 0xee381b34 mi_switch(104,0,101,c6d23310,ee381bc0,c6baf83c) at mi_switch+0x122/frame 0xee381b6c sleepq_switch(c6d23310,0,c11b698a,28b,101,...) at sleepq_switch+0x15b/frame 0xee381b94 sleepq_timedwait(c6baf83c,5c,0,0,0,...) at sleepq_timedwait+0x3f/frame 0xee381bc0 _sleep(c6baf83c,c6baf8f0,5c,c11a23c4,fffffed8,...) at _sleep+0x282/frame 0xee381c08 fdc_thread(c6baf800,ee381ce8,0,0,0,...) at fdc_thread+0x877/frame 0xee381ca4 fork_exit(c0fc2e20,c6baf800,ee381ce8) at fork_exit+0xa3/frame 0xee381cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee381cd4 --- trap 0, eip = 0, esp = 0xee381d20, ebp = 0 --- Tracing command usb pid 15 tid 100053 td 0xc6d23c40 sched_switch(c6d23c40,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee304bb8 mi_switch(104,0,0,80246,c6d23c40,c6d09d34) at mi_switch+0x122/frame 0xee304bf0 sleepq_switch(c6d23c40,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee304c18 sleepq_wait(c6d09d34,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee304c44 _cv_wait(c6d09d34,c6d09e74,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee304c74 usb_process(c6d09d2c,ee304ce8,0,0,0,...) at usb_process+0xf6/frame 0xee304ca4 fork_exit(c09a4cc0,c6d09d2c,ee304ce8) at fork_exit+0xa3/frame 0xee304cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee304cd4 --- trap 0, eip = 0, esp = 0xee304d20, ebp = 0 --- Tracing command usb pid 15 tid 100052 td 0xc6c85c40 sched_switch(c6c85c40,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee301bb8 mi_switch(104,0,c6c99fc8,c0b3286d,c6c85c40,c6d09d04) at mi_switch+0x122/frame 0xee301bf0 sleepq_switch(c6c85c40,0,c11b698a,268,c6d57640,...) at sleepq_switch+0x15b/frame 0xee301c18 sleepq_wait(c6d09d04,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee301c44 _cv_wait(c6d09d04,c6d09e74,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee301c74 usb_process(c6d09cfc,ee301ce8,0,0,0,...) at usb_process+0xf6/frame 0xee301ca4 fork_exit(c09a4cc0,c6d09cfc,ee301ce8) at fork_exit+0xa3/frame 0xee301cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee301cd4 --- trap 0, eip = 0, esp = 0xee301d20, ebp = 0 --- Tracing command usb pid 15 tid 100051 td 0xc6cec000 sched_switch(c6cec000,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2febb8 mi_switch(104,0,0,80246,c6cec000,c6d09cd4) at mi_switch+0x122/frame 0xee2febf0 sleepq_switch(c6cec000,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2fec18 sleepq_wait(c6d09cd4,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2fec44 _cv_wait(c6d09cd4,c6d09e74,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2fec74 usb_process(c6d09ccc,ee2fece8,0,0,0,...) at usb_process+0xf6/frame 0xee2feca4 fork_exit(c09a4cc0,c6d09ccc,ee2fece8) at fork_exit+0xa3/frame 0xee2fecd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2fecd4 --- trap 0, eip = 0, esp = 0xee2fed20, ebp = 0 --- Tracing command usb pid 15 tid 100050 td 0xc6cec310 sched_switch(c6cec310,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2fbbb8 mi_switch(104,0,0,80246,c6cec310,c6d09ca4) at mi_switch+0x122/frame 0xee2fbbf0 sleepq_switch(c6cec310,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2fbc18 sleepq_wait(c6d09ca4,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2fbc44 _cv_wait(c6d09ca4,c6d09e74,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2fbc74 usb_process(c6d09c9c,ee2fbce8,0,0,0,...) at usb_process+0xf6/frame 0xee2fbca4 fork_exit(c09a4cc0,c6d09c9c,ee2fbce8) at fork_exit+0xa3/frame 0xee2fbcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2fbcd4 --- trap 0, eip = 0, esp = 0xee2fbd20, ebp = 0 --- Tracing command usb pid 15 tid 100049 td 0xc6cec620 sched_switch(c6cec620,0,104,c09a52a9,0,...) at sched_switch+0x2d2/frame 0xee2f5bb8 mi_switch(104,0,0,80246,c6cec620,c6cf2dac) at mi_switch+0x122/frame 0xee2f5bf0 sleepq_switch(c6cec620,0,c11b698a,268,4,...) at sleepq_switch+0x15b/frame 0xee2f5c18 sleepq_wait(c6cf2dac,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2f5c44 _cv_wait(c6cf2dac,c6cf2eec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2f5c74 usb_process(c6cf2da4,ee2f5ce8,0,0,0,...) at usb_process+0xf6/frame 0xee2f5ca4 fork_exit(c09a4cc0,c6cf2da4,ee2f5ce8) at fork_exit+0xa3/frame 0xee2f5cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2f5cd4 --- trap 0, eip = 0, esp = 0xee2f5d20, ebp = 0 --- Tracing command usb pid 15 tid 100048 td 0xc6cec930 sched_switch(c6cec930,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2f2bb8 mi_switch(104,0,c0995672,c158f620,c6cec930,c6cf2d7c) at mi_switch+0x122/frame 0xee2f2bf0 sleepq_switch(c6cec930,0,c11b698a,268,c6d57380,...) at sleepq_switch+0x15b/frame 0xee2f2c18 sleepq_wait(c6cf2d7c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2f2c44 _cv_wait(c6cf2d7c,c6cf2eec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2f2c74 usb_process(c6cf2d74,ee2f2ce8,0,0,0,...) at usb_process+0xf6/frame 0xee2f2ca4 fork_exit(c09a4cc0,c6cf2d74,ee2f2ce8) at fork_exit+0xa3/frame 0xee2f2cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2f2cd4 --- trap 0, eip = 0, esp = 0xee2f2d20, ebp = 0 --- Tracing command usb pid 15 tid 100047 td 0xc6cecc40 sched_switch(c6cecc40,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2efbb8 mi_switch(104,0,0,80246,c6cecc40,c6cf2d4c) at mi_switch+0x122/frame 0xee2efbf0 sleepq_switch(c6cecc40,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2efc18 sleepq_wait(c6cf2d4c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2efc44 _cv_wait(c6cf2d4c,c6cf2eec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2efc74 usb_process(c6cf2d44,ee2efce8,0,0,0,...) at usb_process+0xf6/frame 0xee2efca4 fork_exit(c09a4cc0,c6cf2d44,ee2efce8) at fork_exit+0xa3/frame 0xee2efcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2efcd4 --- trap 0, eip = 0, esp = 0xee2efd20, ebp = 0 --- Tracing command usb pid 15 tid 100046 td 0xc6ced000 sched_switch(c6ced000,0,104,8,0,...) at sched_switch+0x2d2/frame 0xee2ecbb8 mi_switch(104,0,3660,0,c6ced000,c6cf2d1c) at mi_switch+0x122/frame 0xee2ecbf0 sleepq_switch(c6ced000,0,c11b698a,268,4,...) at sleepq_switch+0x15b/frame 0xee2ecc18 sleepq_wait(c6cf2d1c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2ecc44 _cv_wait(c6cf2d1c,c6cf2eec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2ecc74 usb_process(c6cf2d14,ee2ecce8,0,0,0,...) at usb_process+0xf6/frame 0xee2ecca4 fork_exit(c09a4cc0,c6cf2d14,ee2ecce8) at fork_exit+0xa3/frame 0xee2eccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2eccd4 --- trap 0, eip = 0, esp = 0xee2ecd20, ebp = 0 --- Tracing command usb pid 15 tid 100044 td 0xc6ced620 sched_switch(c6ced620,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2e5bb8 mi_switch(104,0,0,80246,c6ced620,c6cdbdac) at mi_switch+0x122/frame 0xee2e5bf0 sleepq_switch(c6ced620,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2e5c18 sleepq_wait(c6cdbdac,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2e5c44 _cv_wait(c6cdbdac,c6cdbeec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2e5c74 usb_process(c6cdbda4,ee2e5ce8,0,0,0,...) at usb_process+0xf6/frame 0xee2e5ca4 fork_exit(c09a4cc0,c6cdbda4,ee2e5ce8) at fork_exit+0xa3/frame 0xee2e5cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2e5cd4 --- trap 0, eip = 0, esp = 0xee2e5d20, ebp = 0 --- Tracing command usb pid 15 tid 100043 td 0xc6ced930 sched_switch(c6ced930,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2e2bb8 mi_switch(104,0,0,0,c6ced930,c6cdbd7c) at mi_switch+0x122/frame 0xee2e2bf0 sleepq_switch(c6ced930,0,c11b698a,268,c6d57140,...) at sleepq_switch+0x15b/frame 0xee2e2c18 sleepq_wait(c6cdbd7c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2e2c44 _cv_wait(c6cdbd7c,c6cdbeec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2e2c74 usb_process(c6cdbd74,ee2e2ce8,0,0,0,...) at usb_process+0xf6/frame 0xee2e2ca4 fork_exit(c09a4cc0,c6cdbd74,ee2e2ce8) at fork_exit+0xa3/frame 0xee2e2cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2e2cd4 --- trap 0, eip = 0, esp = 0xee2e2d20, ebp = 0 --- Tracing command usb pid 15 tid 100042 td 0xc6cedc40 sched_switch(c6cedc40,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2dfbb8 mi_switch(104,0,0,80246,c6cedc40,c6cdbd4c) at mi_switch+0x122/frame 0xee2dfbf0 sleepq_switch(c6cedc40,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2dfc18 sleepq_wait(c6cdbd4c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2dfc44 _cv_wait(c6cdbd4c,c6cdbeec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2dfc74 usb_process(c6cdbd44,ee2dfce8,0,0,0,...) at usb_process+0xf6/frame 0xee2dfca4 fork_exit(c09a4cc0,c6cdbd44,ee2dfce8) at fork_exit+0xa3/frame 0xee2dfcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2dfcd4 --- trap 0, eip = 0, esp = 0xee2dfd20, ebp = 0 --- Tracing command usb pid 15 tid 100041 td 0xc6cee000 sched_switch(c6cee000,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2dcbb8 mi_switch(104,0,0,80246,c6cee000,c6cdbd1c) at mi_switch+0x122/frame 0xee2dcbf0 sleepq_switch(c6cee000,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2dcc18 sleepq_wait(c6cdbd1c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2dcc44 _cv_wait(c6cdbd1c,c6cdbeec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2dcc74 usb_process(c6cdbd14,ee2dcce8,0,0,0,...) at usb_process+0xf6/frame 0xee2dcca4 fork_exit(c09a4cc0,c6cdbd14,ee2dcce8) at fork_exit+0xa3/frame 0xee2dccd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2dccd4 --- trap 0, eip = 0, esp = 0xee2dcd20, ebp = 0 --- Tracing command usb pid 15 tid 100039 td 0xc6c0d310 sched_switch(c6c0d310,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2d5bb8 mi_switch(104,0,0,80246,c6c0d310,c6cc6dac) at mi_switch+0x122/frame 0xee2d5bf0 sleepq_switch(c6c0d310,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2d5c18 sleepq_wait(c6cc6dac,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2d5c44 _cv_wait(c6cc6dac,c6cc6eec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2d5c74 usb_process(c6cc6da4,ee2d5ce8,0,0,0,...) at usb_process+0xf6/frame 0xee2d5ca4 fork_exit(c09a4cc0,c6cc6da4,ee2d5ce8) at fork_exit+0xa3/frame 0xee2d5cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2d5cd4 --- trap 0, eip = 0, esp = 0xee2d5d20, ebp = 0 --- Tracing command usb pid 15 tid 100038 td 0xc6c0d620 sched_switch(c6c0d620,0,104,c1a6c5d0,0,...) at sched_switch+0x2d2/frame 0xee2d2bb8 mi_switch(104,0,c6c99fc8,c0b3286d,c6c0d620,c6cc6d7c) at mi_switch+0x122/frame 0xee2d2bf0 sleepq_switch(c6c0d620,0,c11b698a,268,c6d57900,...) at sleepq_switch+0x15b/frame 0xee2d2c18 sleepq_wait(c6cc6d7c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2d2c44 _cv_wait(c6cc6d7c,c6cc6eec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2d2c74 usb_process(c6cc6d74,ee2d2ce8,0,0,0,...) at usb_process+0xf6/frame 0xee2d2ca4 fork_exit(c09a4cc0,c6cc6d74,ee2d2ce8) at fork_exit+0xa3/frame 0xee2d2cd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2d2cd4 --- trap 0, eip = 0, esp = 0xee2d2d20, ebp = 0 --- Tracing command usb pid 15 tid 100037 td 0xc6c0d930 sched_switch(c6c0d930,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2cfbb8 mi_switch(104,0,0,80246,c6c0d930,c6cc6d4c) at mi_switch+0x122/frame 0xee2cfbf0 sleepq_switch(c6c0d930,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2cfc18 sleepq_wait(c6cc6d4c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2cfc44 _cv_wait(c6cc6d4c,c6cc6eec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2cfc74 usb_process(c6cc6d44,ee2cfce8,0,0,0,...) at usb_process+0xf6/frame 0xee2cfca4 fork_exit(c09a4cc0,c6cc6d44,ee2cfce8) at fork_exit+0xa3/frame 0xee2cfcd4 fork_trampoline() at fork_trampoline+0x8/frame 0xee2cfcd4 --- trap 0, eip = 0, esp = 0xee2cfd20, ebp = 0 --- Tracing command usb pid 15 tid 100036 td 0xc6c0dc40 sched_switch(c6c0dc40,0,104,0,0,...) at sched_switch+0x2d2/frame 0xee2ccbb8 mi_switch(104,0,0,80246,c6c0dc40,c6cc6d1c) at mi_switch+0x122/frame 0xee2ccbf0 sleepq_switch(c6c0dc40,0,c11b698a,268,0,...) at sleepq_switch+0x15b/frame 0xee2ccc18 sleepq_wait(c6cc6d1c,0,c11a23c4,1,0,...) at sleepq_wait+0x3f/frame 0xee2ccc44 _cv_wait(c6cc6d1c,c6cc6eec,c118e08e,73,0,...) at _cv_wait+0x182/frame 0xee2ccc74 us ---------------------------------------------------------------------- config.txt options CONFIG_AUTOGENERATED ident GENERIC machine i386 cpu I686_CPU cpu I586_CPU cpu I486_CPU makeoptions WITH_CTF=1 makeoptions DEBUG=-g options XENHVM options USB_DEBUG options ATH_ENABLE_11N options AH_AR5416_INTERRUPT_MITIGATION options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options SC_PIXEL_MODE options VESA options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options ATA_STATIC_ID options SMP options KDB_TRACE options KDB options DDB options VIMAGE options INCLUDE_CONFIG_FILE options DDB_CTF options KDTRACE_HOOKS options MAC options PROCDESC options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options TCP_OFFLOAD options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options NATIVE options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD options ISAPNP device isa device npx device mem device io device uart_ns8250 device atpic device apic device cpufreq device acpi device eisa device pci device fdc device ahci device ata device mvs device siis device ahb device ahc device ahd device esp device hptiop device isp device mpt device mps device mpr device sym device trm device adv device adw device aha device aic device bt device ncv device nsp device stg device isci device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device asr device ciss device dpt device hptmv device hptnr device hptrr device hpt27xx device iir device ips device mly device twa device tws device aac device aacp device aacraid device ida device mfi device mlx device mrsas device pst device twe device atkbdc device atkbd device psm device kbdmux device vga device splash device sc device vt device vt_vga device agp device pmtimer device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device ppi device puc device bxe device de device em device igb device ixgb device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device cas device dc device et device fxp device gem device hme device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sge device sis device sk device ste device stge device tl device tx device vge device vr device vte device wb device xl device cs device ed device ex device ep device fe device ie device sn device xe device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_pci device ath_hal device ath_rate_sample device ipw device iwi device iwn device malo device mwl device ral device wi device wpi device loop device random device padlock_rng device rdrand_rng device ether device vlan device tun device md device gif device faith device firmware device bpf device uhci device ohci device ehci device xhci device usb device ukbd device umass device sound device snd_cmi device snd_csa device snd_emu10kx device snd_es137x device snd_hda device snd_ich device snd_via8233 device mmc device mmcsd device sdhci device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon device hyperv device xenpci device vmx ---------------------------------------------------------------------- msgbuf.txt Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.1-STABLE #1 r276174: Sat Jan 3 20:50:10 CET 2015 root@verdi.nethelp.no:/usr/src/sys/i386/compile/GENERICV i386 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Pentium(R) D CPU 2.80GHz (2793.06-MHz 686-class CPU) Origin="GenuineIntel" Id=0xf47 Family=0xf Model=0x4 Stepping=7 Features=0xbfebfbff Features2=0x641d AMD Features=0x20100000 AMD Features2=0x1 TSC: P-state invariant real memory = 2147483648 (2048 MB) avail memory = 2083926016 (1987 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. ioapic0: Changing APIC ID to 8 ioapic0 irqs 0-23 on motherboard lapic0: Forcing LINT1 to edge trigger random: initialized kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, f00000 (3) failed acpi0: reservation of 1000000, 7ee8cc00 (3) failed cpu0: on acpi0 cpu1: on acpi0 atrtc0: port 0x70-0x7f irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x5f irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 450 Event timer "HPET1" frequency 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz quality 440 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 pcib2: irq 16 at device 28.0 on pci0 pci2: on pcib2 em0: port 0xdcc0-0xdcdf mem 0xfe880000-0xfe89ffff,0xfe8a0000-0xfe8bffff irq 16 at device 0.0 on pci2 em0: Using an MSI interrupt em0: Ethernet address: 00:15:17:36:bc:e8 em1: port 0xdce0-0xdcff mem 0xfe8c0000-0xfe8dffff,0xfe8e0000-0xfe8fffff irq 17 at device 0.1 on pci2 em1: Using an MSI interrupt em1: Ethernet address: 00:15:17:36:bc:e9 pcib3: irq 16 at device 28.4 on pci0 pci3: on pcib3 bge0: mem 0xfe7f0000-0xfe7fffff irq 16 at device 0.0 on pci3 bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: 00:10:18:1c:60:ce pcib4: irq 17 at device 28.5 on pci0 pci4: on pcib4 bge1: mem 0xfe6f0000-0xfe6fffff irq 17 at device 0.0 on pci4 bge1: CHIP ID 0x00004001; ASIC REV 0x04; CHIP REV 0x40; PCI-E miibus1: on bge1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge1: Ethernet address: 00:13:72:20:b4:6f uhci0: port 0xff80-0xff9f irq 21 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0xff60-0xff7f irq 22 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0xff40-0xff5f irq 18 at device 29.2 on pci0 usbus2 on uhci2 uhci3: port 0xff20-0xff3f irq 23 at device 29.3 on pci0 usbus3 on uhci3 ehci0: mem 0xffa80800-0xffa80bff irq 21 at device 29.7 on pci0 usbus4: EHCI version 1.0 usbus4 on ehci0 pcib5: at device 30.0 on pci0 pci5: on pcib5 vgapci0: port 0xcc80-0xccff mem 0xfc000000-0xfdffffff,0xfe4c0000-0xfe4fffff at device 7.0 on pci5 vgapci0: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf irq 16 at device 31.1 on pci0 ata0: at channel 0 on atapci0 atapci1: port 0xfe00-0xfe07,0xfe10-0xfe13,0xfe20-0xfe27,0xfe30-0xfe33,0xfea0-0xfeaf irq 20 at device 31.2 on pci0 ata2: at channel 0 on atapci1 ata3: at channel 1 on atapci1 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xcafff,0xcb000-0xcbfff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: parallel port not found. p4tcc0: on cpu0 p4tcc1: on cpu1 Timecounters tick every 1.000 msec random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 usbus4: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen2.1: at usbus2 ugen1.1: at usbus1 uhub1: on usbus1 ugen4.1: at usbus4 uhub2: on usbus4 ugen3.1: at usbus3 uhub3: on usbus3 uhub4: on usbus2 ada0 at ata2 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 2.x device ada0: Serial Number 9QM098KE ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 ada1 at ata3 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 2.x device ada1: Serial Number 9QM0D4DL ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada1: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad6 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered lapic1: Forcing LINT1 to edge trigger SMP: AP CPU #1 Launched! Timecounter "TSC-low" frequency 1396531633 Hz quality 1000 Root mount waiting for: usbus4 Root mount waiting for: usbus4 uhub2: 8 ports with 8 removable, self powered Trying to mount root from ufs:/dev/ada0s1a [rw]... <118>Setting hostuuid: 44454c4c-4400-104a-8054-b8c04f43324a. <118>Setting hostid: 0xde1f5676. <118>Starting ddb. <118>Entropy harvesting: interrupts ethernet point_to_point swi. <118>Starting file system checks: <118>/dev/ada0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS <118>/dev/ada0s1a: clean, 4499260 free (1612 frags, 562206 blocks, 0.0% fragmentation) <118>/dev/ada0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS <118>/dev/ada0s1d: clean, 3269748 free (10100 frags, 407456 blocks, 0.2% fragmentation) <118>/dev/ada0s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS <118>/dev/ada0s1e: clean, 2462027 free (147 frags, 307735 blocks, 0.0% fragmentation) <118>Mounting local file systems:. <118>Writing entropy file:. <118>Setting hostname: verdi.nethelp.no. ugen3.2: at usbus3 ukbd0: on usbus3 kbd2 at ukbd0 <118>Starting Network: lo0 em0 em1 bge0 bge1. <118>lo0: flags=8049 metric 0 mtu 16384 <118> options=600003 <118> inet6 ::1 prefixlen 128 <118> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 <118> inet 127.0.0.1 netmask 0xff000000 <118> nd6 options=21 <118>em0: flags=8c02 metric 0 mtu 1500 <118> options=4219b <118> ether 00:15:17:36:bc:e8 <118> nd6 options=29 <118> media: Ethernet autoselect <118> status: no carrier <118>em1: flags=8c02 metric 0 mtu 1500 <118> options=4019b <118> ether 00:15:17:36:bc:e9 <118> nd6 options=29 <118> media: Ethernet autoselect <118> status: no carrier <118>bge0: flags=8802 metric 0 mtu 1500 <118> options=8009b <118> ether 00:10:18:1c:60:ce <118> nd6 options=29 <118> media: Ethernet autoselect <118>bge1: flags=8843 metric 0 mtu 1500 <118> options=8009b <118> ether 00:13:72:20:b4:6f <118> inet 10.0.0.2 netmask 0xfffffffc broadcast 10.0.0.3 <118> nd6 options=29 <118> media: Ethernet autoselect (none) <118> status: no carrier <118>Starting devd. <118>Starting Network: em0. <118>em0: flags=8c02 metric 0 mtu 1500 <118> options=4219b <118> ether 00:15:17:36:bc:e8 <118> nd6 options=29 <118> media: Ethernet autoselect <118> status: no carrier <118>Starting Network: em1. <118>em1: flags=8c02 metric 0 mtu 1500 <118> options=4019b <118> ether 00:15:17:36:bc:e9 <118> nd6 options=29 <118> media: Ethernet autoselect <118> status: no carrier <118>Starting Network: bge0. <118>bge0: flags=8802 metric 0 mtu 1500 <118> options=8009b <118> ether 00:10:18:1c:60:ce <118> nd6 options=29 <118> media: Ethernet autoselect <118>Configuring syscons: keymap. <118>Starting pflogd: <118>add net fe80::: gateway ::1 <118>add net ff02::: gateway ::1 <118>add net ::ffff:0.0.0.0: gateway ::1 <118>add net ::0.0.0.0: gateway ::1 <118>Creating and/or trimming log files. <118>Starting syslogd. <118>No core dumps found. <118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat <118>a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout <118>Clearing /tmp (X related). <118>Updating motd:. <118>Mounting late file systems:. <118>Configuring syscons: keymap blanktime. <118>Performing sanity check on sshd configuration. <118>Starting sshd. <118>Starting sendmail_submit. <118>Starting sendmail_msp_queue. <118>Starting cron. <118>Starting background file system checks in 60 seconds. <118> <118>Sat Jan 3 21:25:26 CET 2015 <118>Jan 3 21:25:31 verdi login: ROOT LOGIN (root) ON ttyv0 panic: vm_fault: fault on nofault entry, addr: cc0e5000 cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper(c11b55f1,fc,0,10000000,1,...) at db_trace_self_wrapper+0x2d/frame 0xf0923638 kdb_backtrace(c1361c84,0,c11e064d,f09236ec,0,...) at kdb_backtrace+0x30/frame 0xf09236a0 panic(c11e064d,cc0e5000,1,f0923804,f09237f4,...) at panic+0x11f/frame 0xf09236e0 vm_fault_hold(c1a6d000,cc0e5000,1,0,0,...) at vm_fault_hold+0x2076/frame 0xf0923840 vm_fault(c1a6d000,cc0e5000,1,0,c0b72331,...) at vm_fault+0x81/frame 0xf0923868 trap_pfault(cc0e5fb8,c1472000,f0923914,c6baf200,f09239a4,...) at trap_pfault+0x1bb/frame 0xf09238e8 trap(f0923a34) at trap+0x674/frame 0xf0923a28 calltrap() at calltrap+0x6/frame 0xf0923a28 --- trap 0xc, eip = 0xc1049023, esp = 0xf0923a74, ebp = 0xf0923a9c --- copyout(f0923b28,cc0e5fb8,4,f0923acc,c142c7a8,...) at copyout+0x37/frame 0xf0923a9c sysctl_ip6_temppltime(c142c7a8,c6a9e3a8,0,f0923b28,f0923b28,...) at sysctl_ip6_temppltime+0x3a/frame 0xf0923ac0 sysctl_root(f0923b28,8,2,c0a058a7,c7017850,...) at sysctl_root+0x2b1/frame 0xf0923b00 userland_sysctl(c769f310,f0923bac,4,28803030,bfbfdb70,...) at userland_sysctl+0x1fa/frame 0xf0923b70 sys___sysctl(c769f310,f0923ca8,2e,f0923c64,c0b329f2,...) at sys___sysctl+0x98/frame 0xf0923c18 syscall(f0923ce8) at syscall+0x4a6/frame 0xf0923cdc Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xf0923cdc --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x281a28b7, esp = 0xbfbfdb00, ebp = 0xbfbfdb28 --- KDB: enter: panic ---------------------------------------------------------------------- panic.txt vm_fault: fault on nofault entry, addr: cc0e5000 ---------------------------------------------------------------------- version.txt FreeBSD 10.1-STABLE #1 r276174: Sat Jan 3 20:50:10 CET 2015 root@verdi.nethelp.no:/usr/src/sys/i386/compile/GENERICV