Date: Sat, 7 Jul 2007 05:25:44 GMT From: Yong Rao <yrao@force10networks.com> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/114370: 6.2 kernel with SMP options failed to dump core on dual cpu board Message-ID: <200707070525.l675PiIV033863@www.freebsd.org> Resent-Message-ID: <200707070530.l675U2GS075057@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 114370 >Category: kern >Synopsis: 6.2 kernel with SMP options failed to dump core on dual cpu board >Confidential: no >Severity: critical >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Jul 07 05:30:02 GMT 2007 >Closed-Date: >Last-Modified: >Originator: Yong Rao >Release: FreeBSD 6.2-RELEASE #2 >Organization: Force10 Networks Inc >Environment: FreeBSD localhost.localdomain 6.2-RELEASE FreeBSD 6.2-RELEASE #2: Thu Jul 5 18:30:02 PDT 2007 root@localhost.localdomain:/usr/src/sys/i386/compile/PNICSMPPOLLDBG i386 The hardware settings. We tried on 2 types of motherboards and CPUs. Both got the same problem. a) the first settings CPU: Dual Core AMD Opteron(tm) Processor 280 (2405.47-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x20f12 Stepping = 2 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,P GE ,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0x1<SSE3> AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow+,3DNow> AMD Features2=0x3<LAHF,CMP> Cores per package: 2 b) The 2nd setting. CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2800.11-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR, PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS, HTT,TM,PBE> Features2=0x4400<CNTX-ID,<b14>> real memory = 2147418112 (2047 MB) avail memory = 2096300032 (1999 MB) ACPI APIC Table: <A M I OEMAPIC > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 6 >Description: Summary: We recent found that the kernel compiled with SMP option has problem in dumping the core when crash happens. This happened ONLY on the motherboard that has either a duo-core cpu, or physically 2 cpus. On ONE cpu board, this problem NEVER happens. Phenomena: When crash happens, just at the begining of core dump, it hang. The following is the print. mem dump: start address = 0x4352,len=0x30 Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x4352 fault code = supervisor read, page not present instruction pointer = 0x20:0xc9e9fc92 stack pointer = 0x28:0xebdbdbdc frame pointer = 0x28:0xebdbdbf8 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 74231 (pnicdbg) trap number = 12 panic: page fault cpuid = 1 Uptime: 1d18h27m42s Dumping 4030 MB (2 chunks) chunk 0: 1MB (154 pages) ... ok chunk 1: 4031MB (1031776 pages) (stopped and hung here) >How-To-Repeat: 1) To isolate the issue. We used the kernel configuration file GENERIC, but just added in the following 2 lines. options SMP # Symmetric MultiProcessor Kernel device apic # I/O APIC, required by SMP options 2) Rebuild the kernel by commands cd /usr/src/sys/i386/conf cp GENERIC TESTCONF vi TESTCONF config TESTCONF cd ../compile/TESTCONF make cleandepend; make clean make depend; make make install; sync reboot 3) We loaded a simple device driver. Run a userland application, which sends an ioctl request to kernel to read the illegal memary address. It caused the crash of the kernel. >Fix: It is not fixed. But here are some debugging information. I am attaching more debugs. After core dump hangs, I am able to get into ddb. But the trace for the thread where core dump is running does not look right. I was able to isolate the dumper function – ad_dump where it seems to be hanging. Hopefully these debugs will give more clues. Please advise on what more to look for. I am new to SMP and still coming up to speed. The following debugs are taken after core dump hangs: a) show pcpu b) bt 100054 -- 100054 is the thread ID, which does the core dump c) alltrace (in attached file “alltrace.txt”) d) step and next command in debug a) --- db> show pcpu cpuid = 1 curthread = 0xc9450d80: pid 16 "swi6: Giant taskq" curpcb = 0xe7fe9d90 fpcurthread = none idlethread = 0xc944f600: pid 10 "idle: cpu1" APIC ID = 1 currentldt = 0x50 spin locks held: db> show pcpu 0 cpuid = 0 curthread = 0xc96eb000: pid 894 "disablemeta" curpcb = 0xe9b4fd90 fpcurthread = none idlethread = 0xc944f780: pid 11 "idle: cpu0" APIC ID = 0 currentldt = 0x50 spin locks held: db> b) --- db> bt 100054 Tracing pid 894 tid 100054 td 0xc96eb000 sched_switch(3225858983,10,0,3382180240,3377804288,...) at sched_switch+331 _end(53,128,606517178,0,0,...) at 3920951848 _end(3377796632,3379230208,3379230476,3377796608,3379229696,...) at 3377796608 _end(3228796064,3228796064,3228796064,3228796064,3228796064,...) at 3379230208 null_method(0,0,0,0,0,...) at null_method db> c) --- Please see attached file “alltrace.txt” for details. d) trace --- db> trace Tracing pid 943 tid 100080 td 0xc998d000 kdb_enter(3228283314) at kdb_enter+43 panic(3390017776,3237235458,3956644984,21554,3389785424,...) at panic+295 mtpioctl(3389997312,3237235458,3389811456,3,3382235136,...) at mtpioctl+193 devfs_ioctl_f(3379694976,3237235458,3389811456,3376546688,3382235136) at devfs_5 ioctl(3382235136,3956645124) at ioctl+874 syscall(59,59,59,3216893976,134516360,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (54, FreeBSD ELF32, ioctl), eip = 672367343, esp = 3216893900, ebp - db> s [thread pid 943 tid 100080 ] Stopped at kdb_enter+44: leave db> s [thread pid 943 tid 100080 ] Stopped at kdb_enter+45: ret db> s [thread pid 943 tid 100080 ] Stopped at panic+295: addl $4,%esp db> s [thread pid 943 tid 100080 ] Stopped at panic+298: pushl $561 db> s [thread pid 943 tid 100080 ] Stopped at panic+303: pushl $3228283350 db> s [thread pid 943 tid 100080 ] Stopped at panic+308: pushl $0 db> s [thread pid 943 tid 100080 ] Stopped at panic+310: pushl $3228776128 db> s [thread pid 943 tid 100080 ] Stopped at panic+315: call _mtx_lock_spin_flags db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags: pushl %ebp db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+1: movl %esp,%ebp db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+3: pushl %edi db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+4: pushl %esi db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+5: pushl %ebx db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+6: movl 8(%ebp),%esi db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+9: movl 20(%ebp),%edi db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+12: pushl %edi db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+13: pushl 16(%ebp) db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+16: movl 12(%ebp),%eax db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+19: orl $9,%eax db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+22: pushl %eax db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+23: pushl %esi db> s [thread pid 943 tid 100080 ] Stopped at _mtx_lock_spin_flags+24: call witness_checkorder db> n After 22 instructions (0 loads, 0 stores), [thread pid 943 tid 100080 ] Stopped at witness_checkorder+1259: ret db> n KDB: stack backtrace: hangs here ... Patch attached with submission follows: db> alltrace Tracing command disablemeta pid 894 tid 100054 td 0xc96eb000 sched_switch(3225858983,10,0,3382180240,3377804288,...) at sched_switch+331 _end(53,128,606517178,0,0,...) at 3920951848 _end(3377796632,3379230208,3379230476,3377796608,3379229696,...) at 3377796608 _end(3228796064,3228796064,3228796064,3228796064,3228796064,...) at 3379230208 null_method(0,0,0,0,0,...) at null_method Tracing command bash pid 884 tid 100058 td 0xc9991c00 sched_switch(3382254592,0,1) at sched_switch+331 mi_switch(1,0,0,3956742568,3226748917,...) at mi_switch+426 sleepq_switch(3377983504) at sleepq_switch+113 sleepq_wait_sig(3377983504) at sleepq_wait_sig+29 msleep(3377983504,0,345,3228302894,0) at msleep+510 ttysleep(3377983488,3377983504,345,3228302894,0,...) at ttysleep+33 ttread(3377983488,3956743356,0) at ttread+1167 ttyread(3377918464,3956743356,0,3228776192,0,...) at ttyread+47 giant_read(3377918464,3956743356,0) at giant_read+45 devfs_read_f(3379716600,3956743356,3387476352,0,3382254592) at devfs_read_f+98 dofileread(3382254592,0,3379716600,3956743356,4294967295,...) at dofileread+133 kern_readv(3382254592,0,3956743356,3217025183,1,...) at kern_readv+54 read(3382254592,3956743428) at read+69 syscall(59,59,59,0,3217025183,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (3, FreeBSD ELF32, read), eip = 135257039, esp = 3217025148, ebp = - Tracing command bash pid 878 tid 100071 td 0xc96ea000 sched_switch(3379470336,0,1) at sched_switch+331 mi_switch(1,0,0,3920829452,3226748917,...) at mi_switch+426 sleepq_switch(3382776344) at sleepq_switch+113 sleepq_wait_sig(3382776344) at sleepq_wait_sig+29 msleep(3382776344,3382776448,348,3228297560,0) at msleep+510 kern_wait(3379470336,4294967295,3920829572,6,0) at kern_wait+1699 wait4(3379470336,3920829700) at wait4+42 syscall(59,59,59,894,0,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (7, FreeBSD ELF32, wait4), eip = 135042415, esp = 3217025340, ebp =- Tracing command login pid 866 tid 100083 td 0xc9992780 sched_switch(3382257536,0,1) at sched_switch+331 mi_switch(1,0,0,3956829196,3226748917,...) at mi_switch+426 sleepq_switch(3385359504) at sleepq_switch+113 sleepq_wait_sig(3385359504) at sleepq_wait_sig+29 msleep(3385359504,3385359608,348,3228297560,0) at msleep+510 kern_wait(3382257536,884,3956829316,0,0) at kern_wait+1699 wait4(3382257536,3956829444) at wait4+42 syscall(59,59,59,582,1,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (7, FreeBSD ELF32, wait4), eip = 672093903, esp = 3217026396, ebp =- Tracing command getty pid 865 tid 100090 td 0xc9c8c000 sched_switch(3385376768,0,1) at sched_switch+331 mi_switch(1,0,0,3957242260,3226748917,...) at mi_switch+426 sleepq_switch(3377985552) at sleepq_switch+113 sleepq_wait_sig(3377985552) at sleepq_wait_sig+29 msleep(3377985552,0,345,3228302894,0) at msleep+510 ttysleep(3377985536,3377985552,345,3228302894,0,...) at ttysleep+33 ttread(3377985536,3957243068,0) at ttread+1167 ttyread(3378041344,3957243068,0,3957242900,3226488553,...) at ttyread+47 scread(3378041344,3957243068,0,3228776192,0,...) at scread+34 giant_read(3378041344,3957243068,0) at giant_read+45 devfs_read_f(3379723360,3957243068,3376727936,0,3385376768) at devfs_read_f+98 dofileread(3385376768,0,3379723360,3957243068,4294967295,...) at dofileread+133 kern_readv(3385376768,0,3957243068,3217026591,1,...) at kern_readv+54 read(3385376768,3957243140) at read+69 syscall(59,59,59,0,134538624,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = - Tracing command getty pid 864 tid 100068 td 0xc96ea480 sched_switch(3379471488,0,1) at sched_switch+331 mi_switch(1,0,0,3920865684,3226748917,...) at mi_switch+426 sleepq_switch(3377979408) at sleepq_switch+113 sleepq_wait_sig(3377979408) at sleepq_wait_sig+29 msleep(3377979408,0,345,3228302894,0) at msleep+510 ttysleep(3377979392,3377979408,345,3228302894,0,...) at ttysleep+33 ttread(3377979392,3920866492,0) at ttread+1167 ttyread(3378041088,3920866492,0,3920866324,3226488553,...) at ttyread+47 scread(3378041088,3920866492,0,3228776192,0,...) at scread+34 giant_read(3378041088,3920866492,0) at giant_read+45 devfs_read_f(3382342424,3920866492,3376727936,0,3379471488) at devfs_read_f+98 dofileread(3379471488,0,3382342424,3920866492,4294967295,...) at dofileread+133 kern_readv(3379471488,0,3920866492,3217026591,1,...) at kern_readv+54 read(3379471488,3920866564) at read+69 syscall(59,59,59,0,134538624,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = - Tracing command getty pid 863 tid 100042 td 0xc9566480 sched_switch(3377882240,0,1) at sched_switch+331 mi_switch(1,0,0,3920550292,3226748917,...) at mi_switch+426 sleepq_switch(3378064400) at sleepq_switch+113 sleepq_wait_sig(3378064400) at sleepq_wait_sig+29 msleep(3378064400,0,345,3228302894,0) at msleep+510 ttysleep(3378064384,3378064400,345,3228302894,0,...) at ttysleep+33 ttread(3378064384,3920551100,0) at ttread+1167 ttyread(3378040832,3920551100,0,3920550932,3226488553,...) at ttyread+47 scread(3378040832,3920551100,0,3228776192,0,...) at scread+34 giant_read(3378040832,3920551100,0) at giant_read+45 devfs_read_f(3382344728,3920551100,3376727936,0,3377882240) at devfs_read_f+98 dofileread(3377882240,0,3382344728,3920551100,4294967295,...) at dofileread+133 kern_readv(3377882240,0,3920551100,3217026591,1,...) at kern_readv+54 read(3377882240,3920551172) at read+69 syscall(59,59,59,0,134538624,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = - Tracing command getty pid 862 tid 100072 td 0xc9a12900 sched_switch(3382782208,0,1) at sched_switch+331 mi_switch(1,0,0,3957016980,3226748917,...) at mi_switch+426 sleepq_switch(3378063376) at sleepq_switch+113 sleepq_wait_sig(3378063376) at sleepq_wait_sig+29 msleep(3378063376,0,345,3228302894,0) at msleep+510 ttysleep(3378063360,3378063376,345,3228302894,0,...) at ttysleep+33 ttread(3378063360,3957017788,0) at ttread+1167 ttyread(3377924352,3957017788,0,3957017620,3226488553,...) at ttyread+47 scread(3377924352,3957017788,0,3228776192,0,...) at scread+34 giant_read(3377924352,3957017788,0) at giant_read+45 devfs_read_f(3382342640,3957017788,3376727936,0,3382782208) at devfs_read_f+98 dofileread(3382782208,0,3382342640,3957017788,4294967295,...) at dofileread+133 kern_readv(3382782208,0,3957017788,3217026591,1,...) at kern_readv+54 read(3382782208,3957017860) at read+69 syscall(59,59,59,0,134538624,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = - Tracing command getty pid 861 tid 100079 td 0xc9992d80 sched_switch(3382259072,0,1) at sched_switch+331 mi_switch(1,0,0,3956877716,3226748917,...) at mi_switch+426 sleepq_switch(3378059280) at sleepq_switch+113 sleepq_wait_sig(3378059280) at sleepq_wait_sig+29 msleep(3378059280,0,345,3228302894,0) at msleep+510 ttysleep(3378059264,3378059280,345,3228302894,0,...) at ttysleep+33 ttread(3378059264,3956878524,0) at ttread+1167 ttyread(3377924608,3956878524,0,3956878356,3226488553,...) at ttyread+47 scread(3377924608,3956878524,0,3228776192,0,...) at scread+34 giant_read(3377924608,3956878524,0) at giant_read+45 devfs_read_f(3379719696,3956878524,3376727936,0,3382259072) at devfs_read_f+98 dofileread(3382259072,0,3379719696,3956878524,4294967295,...) at dofileread+133 kern_readv(3382259072,0,3956878524,3217026591,1,...) at kern_readv+54 read(3382259072,3956878596) at read+69 syscall(59,59,59,0,134538624,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = - Tracing command getty pid 860 tid 100057 td 0xc9991d80 sched_switch(3382254976,0,1) at sched_switch+331 mi_switch(1,0,0,3956754836,3226748917,...) at mi_switch+426 sleepq_switch(3378058256) at sleepq_switch+113 sleepq_wait_sig(3378058256) at sleepq_wait_sig+29 msleep(3378058256,0,345,3228302894,0) at msleep+510 ttysleep(3378058240,3378058256,345,3228302894,0,...) at ttysleep+33 ttread(3378058240,3956755644,0) at ttread+1167 ttyread(3377924864,3956755644,0,3956755476,3226488553,...) at ttyread+47 scread(3377924864,3956755644,0,3228776192,0,...) at scread+34 giant_read(3377924864,3956755644,0) at giant_read+45 devfs_read_f(3379722928,3956755644,3376727936,0,3382254976) at devfs_read_f+98 dofileread(3382254976,0,3379722928,3956755644,4294967295,...) at dofileread+133 kern_readv(3382254976,0,3956755644,3217026591,1,...) at kern_readv+54 read(3382254976,3956755716) at read+69 syscall(59,59,59,0,134538624,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = - Tracing command getty pid 859 tid 100088 td 0xc9c8c300 sched_switch(3385377536,0,1) at sched_switch+331 mi_switch(1,0,0,3957266836,3226748917,...) at mi_switch+426 sleepq_switch(3378062352) at sleepq_switch+113 sleepq_wait_sig(3378062352) at sleepq_wait_sig+29 msleep(3378062352,0,345,3228302894,0) at msleep+510 ttysleep(3378062336,3378062352,345,3228302894,0,...) at ttysleep+33 ttread(3378062336,3957267644,0) at ttread+1167 ttyread(3377925120,3957267644,0,3957267476,3226488553,...) at ttyread+47 scread(3377925120,3957267644,0,3228776192,0,...) at scread+34 giant_read(3377925120,3957267644,0) at giant_read+45 devfs_read_f(3382342928,3957267644,3376727936,0,3385377536) at devfs_read_f+98 dofileread(3385377536,0,3382342928,3957267644,4294967295,...) at dofileread+133 kern_readv(3385377536,0,3957267644,3217026591,1,...) at kern_readv+54 read(3385377536,3957267716) at read+69 syscall(59,59,59,0,134538624,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = - Tracing command login pid 858 tid 100059 td 0xc9991a80 sched_switch(3382254208,0,1) at sched_switch+331 mi_switch(1,0,0,3956730892,3226748917,...) at mi_switch+426 sleepq_switch(3382249032) at sleepq_switch+113 sleepq_wait_sig(3382249032) at sleepq_wait_sig+29 msleep(3382249032,3382249136,348,3228297560,0) at msleep+510 kern_wait(3382254208,878,3956731012,0,0) at kern_wait+1699 wait4(3382254208,3956731140) at wait4+42 syscall(59,59,59,582,1,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (7, FreeBSD ELF32, wait4), eip = 672093903, esp = 3217026396, ebp =- Tracing command cron pid 815 tid 100080 td 0xc9992c00 sched_switch(3382258688,0,1) at sched_switch+331 mi_switch(1,0,0,3956866108,3226749082,...) at mi_switch+426 sleepq_switch(3228779692) at sleepq_switch+113 sleepq_timedwait_sig(3228779692) at sleepq_timedwait_sig+30 msleep(3228779692,0,348,3228286296,60001,...) at msleep+468 kern_nanosleep(3382258688,3956866252,3956866244) at kern_nanosleep+171 nanosleep(3382258688,3956866308) at nanosleep+79 syscall(59,59,59,134545548,60,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 672337699, esp = 3217026268,- Tracing command sendmail pid 808 tid 100065 td 0xc96ea900 sched_switch(3379472640,0,1) at sched_switch+331 mi_switch(1,0,0,3920903260,3226748917,...) at mi_switch+426 sleepq_switch(3382772836) at sleepq_switch+113 sleepq_wait_sig(3382772836) at sleepq_wait_sig+29 msleep(3382772836,3382772888,360,3228284168,0) at msleep+510 kern_sigsuspend(3379472640,0,0,0,0,...) at kern_sigsuspend+163 sigsuspend(3379472640,3920903428) at sigsuspend+51 syscall(59,59,59,0,3217018880,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (341, FreeBSD ELF32, sigsuspend), eip = 673683383, esp = 3217018860- Tracing command sendmail pid 804 tid 100041 td 0xc9566600 sched_switch(3377882624,0,1) at sched_switch+331 mi_switch(1,0,0,3920562976,3226749082,...) at mi_switch+426 sleepq_switch(3229095940) at sleepq_switch+113 sleepq_timedwait_sig(3229095940,3229095940,5001,3920563008,582,...) at sleepq_t0 cv_timedwait_sig(3229095940,3229095904,5001) at cv_timedwait_sig+290 kern_select(3377882624,4,3217016064,0,0,...) at kern_select+1169 select(3377882624,3920563460) at select+68 syscall(59,59,59,1,1,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (93, FreeBSD ELF32, select), eip = 674198223, esp = 3217015820, ebp- Tracing command sshd pid 798 tid 100082 td 0xc9992900 sched_switch(3382257920,0,1) at sched_switch+331 mi_switch(1,0,0,3956841260,3226748917,...) at mi_switch+426 sleepq_switch(3229095940) at sleepq_switch+113 sleepq_wait_sig(3229095940,582,3228776128,3228296630,771,...) at sleepq_wait_si9 cv_wait_sig(3229095940,3229095904) at cv_wait_sig+281 kern_select(3382257920,5,134723872,0,0,...) at kern_select+1191 select(3382257920,3956841732) at select+68 syscall(59,59,59,1,10,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (93, FreeBSD ELF32, select), eip = 674546383, esp = 3217023340, ebp- Tracing command usbd pid 761 tid 100050 td 0xc96eb600 sched_switch(3379475968,0,1) at sched_switch+331 mi_switch(1,0,0,3921001248,3226749082,...) at mi_switch+426 sleepq_switch(3229095940) at sleepq_switch+113 sleepq_timedwait_sig(3229095940,3229095940,30001,3921001280,582,...) at sleepq_0 cv_timedwait_sig(3229095940,3229095904,30001) at cv_timedwait_sig+290 kern_select(3379475968,6,3217026464,3217026336,0,...) at kern_select+1169 select(3379475968,3921001732) at select+68 syscall(59,59,59,3217026336,30,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (93, FreeBSD ELF32, select), eip = 672379599, esp = 3217026252, ebp- Tracing command smbd pid 735 tid 100078 td 0xc9a12000 sched_switch(3382779904,0,1) at sched_switch+331 mi_switch(1,0,0,3956943964,3226748917,...) at mi_switch+426 sleepq_switch(3385356876) at sleepq_switch+113 sleepq_wait_sig(3385356876) at sleepq_wait_sig+29 msleep(3385356876,3385356928,360,3228284168,0) at msleep+510 kern_sigsuspend(3382779904,1073746048,0,0,0,...) at kern_sigsuspend+163 sigsuspend(3382779904,3956944132) at sigsuspend+51 syscall(59,59,59,137408512,137408512,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (341, FreeBSD ELF32, sigsuspend), eip = 676951991, esp = 3217021452- Tracing command smbd pid 714 tid 100086 td 0xc9992300 sched_switch(3382256384,0,1) at sched_switch+331 mi_switch(1,0,0,3956792108,3226748917,...) at mi_switch+426 sleepq_switch(3229095940) at sleepq_switch+113 sleepq_wait_sig(3229095940,582,3228776128,3228296630,771,...) at sleepq_wait_si9 cv_wait_sig(3229095940,3229095904) at cv_wait_sig+281 kern_select(3382256384,22,3217021808,0,0,...) at kern_select+1191 select(3382256384,3956792580) at select+68 syscall(59,59,59,137408512,0,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (93, FreeBSD ELF32, select), eip = 677466831, esp = 3217021372, ebp- Tracing command nmbd pid 710 tid 100089 td 0xc9c8c180 sched_switch(3385377152,0,1) at sched_switch+331 mi_switch(1,0,0,3957254944,3226749082,...) at mi_switch+426 sleepq_switch(3229095940) at sleepq_switch+113 sleepq_timedwait_sig(3229095940,3229095940,10001,3957254976,582,...) at sleepq_0 cv_timedwait_sig(3229095940,3229095904,10001) at cv_timedwait_sig+290 kern_select(3385377152,13,3217025088,0,0,...) at kern_select+1169 select(3385377152,3957255428) at select+68 syscall(59,59,59,13,3217025088,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (93, FreeBSD ELF32, select), eip = 674271951, esp = 3217024796, ebp- Tracing command syslogd pid 631 tid 100044 td 0xc9566180 sched_switch(3377881472,0,1) at sched_switch+331 mi_switch(1,0,0,3920526124,3226748917,...) at mi_switch+426 sleepq_switch(3229095940) at sleepq_switch+113 sleepq_wait_sig(3229095940,582,3228776128,3228296630,771,...) at sleepq_wait_si9 cv_wait_sig(3229095940,3229095904) at cv_wait_sig+281 kern_select(3377881472,9,134570080,0,0,...) at kern_select+1191 select(3377881472,3920526596) at select+68 syscall(59,59,59,134570084,134570080,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (93, FreeBSD ELF32, select), eip = 672445135, esp = 3217021484, ebp- Tracing command devd pid 565 tid 100060 td 0xc9991900 sched_switch(3382253824,0,1) at sched_switch+331 mi_switch(1,0,0,3956718380,3226748917,...) at mi_switch+426 sleepq_switch(3229095940) at sleepq_switch+113 sleepq_wait_sig(3229095940,582,3228776128,3228296630,771,...) at sleepq_wait_si9 cv_wait_sig(3229095940,3229095904) at cv_wait_sig+281 kern_select(3382253824,5,3217025488,0,0,...) at kern_select+1191 select(3382253824,3956718852) at select+68 syscall(59,59,59,0,8,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (93, FreeBSD ELF32, select), eip = 134698999, esp = 3217025388, ebp- Tracing command adjkerntz pid 143 tid 100064 td 0xc96eaa80 sched_switch(3379473024,0,1) at sched_switch+331 mi_switch(1,0,0,3920915548,3226748917,...) at mi_switch+426 sleepq_switch(3382772300) at sleepq_switch+113 sleepq_wait_sig(3382772300) at sleepq_wait_sig+29 msleep(3382772300,3382772352,360,3228284168,0) at msleep+510 kern_sigsuspend(3379473024,0,0,0,0,...) at kern_sigsuspend+163 sigsuspend(3379473024,3920915716) at sigsuspend+51 syscall(59,59,59,0,25200,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (341, FreeBSD ELF32, sigsuspend), eip = 671856567, esp = 3217026460- Tracing command schedcpu pid 42 tid 100046 td 0xc951bd80 sched_switch(3377577344,0,1) at sched_switch+331 mi_switch(1,0,0,3919867052,3226749014,...) at mi_switch+426 sleepq_switch(3919867124,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3919867124) at sleepq_timedwait+34 msleep(3919867124,0,0,3228262727,1000) at msleep+489 schedcpu_thread(0,3919867192,3228776128,0,3228273492,...) at schedcpu_thread+38 fork_exit(3226695036,0,3919867192) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3919867244, ebp = 0 --- Tracing command softdepflush pid 41 tid 100047 td 0xc951bc00 sched_switch(3377576960,0,1) at sched_switch+331 mi_switch(1,0,0,3919854756,3226749014,...) at mi_switch+426 sleepq_switch(3229151220,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3229151220) at sleepq_timedwait+34 msleep(3229151220,3229151168,68,3228349621,1000) at msleep+489 softdep_flush(0,3919854904,3228776128,0,3228273492,...) at softdep_flush+499 fork_exit(3227615092,0,3919854904) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3919854956, ebp = 0 --- Tracing command vnlru pid 40 tid 100048 td 0xc951ba80 sched_switch(3377576576,0,1) at sched_switch+331 mi_switch(1,0,1,3919842460,3226749014,...) at mi_switch+426 sleepq_switch(3379465360,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3379465360) at sleepq_timedwait+34 msleep(3379465360,3229098336,592,3228312087,1000,3229098464) at msleep+489 vnlru_proc(0,3919842616,3228776128,0,3228273492,...) at vnlru_proc+223 fork_exit(3226951860,0,3919842616) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3919842668, ebp = 0 --- Tracing command syncer pid 39 tid 100049 td 0xc951b900 sched_switch(3377576192,0,1) at sched_switch+331 mi_switch(1,0,1,3919830160,3226748861,...) at mi_switch+426 sleepq_switch(3228779036,3228776128,0,3228292512,529,...) at sleepq_switch+113 sleepq_wait(3228779036,0,3226599612,3229098400,8,...) at sleepq_wait+33 msleep(3228779036,0,104,3228312226,0) at msleep+525 sched_sync(0,3919830328,3228776128,0,3228273492,...) at sched_sync+777 fork_exit(3226957540,0,3919830328) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3919830380, ebp = 0 --- Tracing command bufdaemon pid 38 tid 100027 td 0xc951b180 sched_switch(3377574272,0,1) at sched_switch+331 mi_switch(1,0,1,3919768756,3226749014,...) at mi_switch+426 sleepq_switch(3229097324,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3229097324) at sleepq_timedwait+34 msleep(3229097324,3229097344,68,3228307860,1000) at msleep+489 buf_daemon(0,3919768888,3228776128,0,3228273492,...) at buf_daemon+264 fork_exit(3226903868,0,3919768888) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3919768940, ebp = 0 --- Tracing command pagezero pid 37 tid 100028 td 0xc951b000 sched_switch(3377573888,0,1) at sched_switch+331 mi_switch(1,0,0,3919756452,3226749014,...) at mi_switch+426 sleepq_switch(3229155172,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3229155172) at sleepq_timedwait+34 msleep(3229155172,3229153856,512,3228365548,300000,3229153856,0,3228365522,157)9 vm_pagezero(0,3919756600,3228776128,0,3228273492,...) at vm_pagezero+108 fork_exit(3227806548,0,3919756600) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3919756652, ebp = 0 --- Tracing command idlepoll pid 36 tid 100029 td 0xc9454d80 sched_switch(3376762240,0,1) at sched_switch+331 mi_switch(1,0,0,3892346024,3226749014,...) at mi_switch+426 sleepq_switch(3228776544,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3228776544) at sleepq_timedwait+34 msleep(3228776544,0,0,3228281840,3000) at msleep+489 poll_idle(0,3892346168,3228776128,0,3228273492,...) at poll_idle+226 fork_exit(3226607484,0,3892346168) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892346220, ebp = 0 --- Tracing command vmdaemon pid 35 tid 100030 td 0xc9454c00 sched_switch(3376761856,0,1) at sched_switch+331 mi_switch(1,0,1,3892333716,3226748861,...) at mi_switch+426 sleepq_switch(3229153972,3228776128,0,3228292512,529,...) at sleepq_switch+113 sleepq_wait(3229153972,0,3226599551,3228776192,3228365165,...) at sleepq_wait+33 msleep(3229153972,0,104,3228307860,0) at msleep+525 vm_daemon(0,3892333880,3228776128,0,3228273492,...) at vm_daemon+54 fork_exit(3227802892,0,3892333880) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892333932, ebp = 0 --- Tracing command pagedaemon pid 34 tid 100031 td 0xc9454a80 sched_switch(3376761472,0,1) at sched_switch+331 mi_switch(1,0,0,3892321456,3226749014,...) at mi_switch+426 sleepq_switch(3229153904,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3229153904) at sleepq_timedwait+34 msleep(3229153904,3229153856,68,3228307860,5000) at msleep+489 vm_pageout(0,3892321592,3228776128,0,3228273492,...) at vm_pageout+608 fork_exit(3227802060,0,3892321592) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892321644, ebp = 0 --- Tracing command irq7: ppc0 pid 33 tid 100032 td 0xc9454900 fork_trampoline() at fork_trampoline Tracing command swi0: sio pid 32 tid 100033 td 0xc9454780 sched_switch(3376760704,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3377944496,3892297016,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3377944496,3892297016) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892297068, ebp = 0 --- Tracing command fdc0 pid 31 tid 100034 td 0xc9454600 sched_switch(3376760320,0,1) at sched_switch+331 mi_switch(1,0,0,3892284500,3226749014,...) at mi_switch+426 sleepq_switch(3377805884,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3377805884) at sleepq_timedwait+34 msleep(3377805884,3377806064,76,3228262727,1000) at msleep+489 fdc_worker(3377805824,3377806064,0,3228371674,1166) at fdc_worker+458 fdc_thread(3377805824,3892284728,3228776128,0,3228273492,...) at fdc_thread+84 fork_exit(3227861752,3377805824,3892284728) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892284780, ebp = 0 --- Tracing command irq1: atkbd0 pid 30 tid 100035 td 0xc9454480 sched_switch(3376759936,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3377944864,3892272440,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3377944864,3892272440) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892272492, ebp = 0 --- Tracing command irq24: bge1 pid 29 tid 100036 td 0xc9566d80 sched_switch(3377884544,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3377837568,3920624952,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3377837568,3920624952) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3920625004, ebp = 0 --- Tracing command irq27: bge0 pid 28 tid 100037 td 0xc9566c00 sched_switch(3377884160,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3377838352,3920612664,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3377838352,3920612664) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3920612716, ebp = 0 --- Tracing command irq15: ata1 pid 27 tid 100017 td 0xc9450900 fork_trampoline() at fork_trampoline Tracing command irq14: ata0 pid 26 tid 100018 td 0xc9450780 fork_trampoline() at fork_trampoline Tracing command irq16: atapci0 pid 25 tid 100019 td 0xc9450600 sched_switch(3376743936,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3377734416,3892161848,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3377734416,3892161848) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892161900, ebp = 0 --- Tracing command usb1 pid 24 tid 100020 td 0xc9450480 sched_switch(3376743552,0,1) at sched_switch+331 mi_switch(1,0,1,3892149420,3226749014,...) at mi_switch+426 sleepq_switch(3377725968,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3377725968) at sleepq_timedwait+34 msleep(3377725968,0,92,3228259087,60000,3377742272) at msleep+489 usb_event_thread(3377742272,3892149560,3228776128,0,3228273492,...) at usb_even2 fork_exit(3226327968,3377742272,3892149560) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892149612, ebp = 0 --- Tracing command usbtask pid 23 tid 100021 td 0xc9450300 sched_switch(3376743168,0,1) at sched_switch+331 mi_switch(1,0,1,3892137140,3226748861,...) at mi_switch+426 sleepq_switch(3228766148,3228776128,0,3228292512,529,...) at sleepq_switch+113 sleepq_wait(3228766148,0,3226599551,3228776192,3228259073,...) at sleepq_wait+33 msleep(3228766148,0,92,3228259094,0,...) at msleep+525 usb_task_thread(0,3892137272,3228776128,0,3228273492,...) at usb_task_thread+71 fork_exit(3226328112,0,3892137272) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892137324, ebp = 0 --- Tracing command usb0 pid 22 tid 100022 td 0xc9450180 sched_switch(3376742784,0,1) at sched_switch+331 mi_switch(1,0,1,3892124844,3226749014,...) at mi_switch+426 sleepq_switch(3377709584,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3377709584) at sleepq_timedwait+34 msleep(3377709584,0,92,3228259087,60000,3377645696) at msleep+489 usb_event_thread(3377645696,3892124984,3228776128,0,3228273492,...) at usb_even2 fork_exit(3226327968,3377645696,3892124984) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892125036, ebp = 0 --- Tracing command irq19: ohci0 ohci1 pid 21 tid 100023 td 0xc951b780 fork_trampoline() at fork_trampoline Tracing command irq9: acpi0 pid 20 tid 100024 td 0xc951b600 fork_trampoline() at fork_trampoline Tracing command swi6: task queue pid 19 tid 100025 td 0xc951b480 sched_switch(3377575040,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3376511392,3919793464,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3376511392,3919793464) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3919793516, ebp = 0 --- Tracing command kqueue taskq pid 9 tid 100026 td 0xc951b300 sched_switch(3377574656,0,1) at sched_switch+331 mi_switch(1,0,0,3919781032,3226748861,...) at mi_switch+426 sleepq_switch(3377065088,3228776128,0,3228292512,529,...) at sleepq_switch+113 sleepq_wait(3377065088,0,3228293171,73,8,...) at sleepq_wait+33 msleep(3377065088,3377065116,0,3228262727,0) at msleep+525 taskqueue_thread_loop(3228769632,3919781176,3228776128,0,3228273492,...) at tas0 fork_exit(3226753588,3228769632,3919781176) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3919781228, ebp = 0 --- Tracing command acpi_task_2 pid 8 tid 100008 td 0xc944f300 sched_switch(3376739072,0,1) at sched_switch+331 mi_switch(1,0,0,3892014248,3226748861,...) at mi_switch+426 sleepq_switch(3377065344,3228776128,0,3228292512,529,...) at sleepq_switch+113 sleepq_wait(3377065344,0,3228293171,73,8,...) at sleepq_wait+33 msleep(3377065344,3377065372,0,3228262727,0) at msleep+525 taskqueue_thread_loop(3230234752,3892014392,3228776128,0,3228273492,...) at tas0 fork_exit(3226753588,3230234752,3892014392) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892014444, ebp = 0 --- Tracing command acpi_task_1 pid 7 tid 100009 td 0xc944f180 sched_switch(3376738688,0,1) at sched_switch+331 mi_switch(1,0,0,3892001960,3226748861,...) at mi_switch+426 sleepq_switch(3377065344,3228776128,0,3228292512,529,...) at sleepq_switch+113 sleepq_wait(3377065344,0,3228293171,73,8,...) at sleepq_wait+33 msleep(3377065344,3377065372,0,3228262727,0) at msleep+525 taskqueue_thread_loop(3230234752,3892002104,3228776128,0,3228273492,...) at tas0 fork_exit(3226753588,3230234752,3892002104) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892002156, ebp = 0 --- Tracing command acpi_task_0 pid 6 tid 100010 td 0xc944f000 sched_switch(3376738304,0,1) at sched_switch+331 mi_switch(1,0,0,3891989672,3226748861,...) at mi_switch+426 sleepq_switch(3377065344,3228776128,0,3228292512,529,...) at sleepq_switch+113 sleepq_wait(3377065344,0,3228293171,73,3377600608,...) at sleepq_wait+33 msleep(3377065344,3377065372,0,3228262727,0) at msleep+525 taskqueue_thread_loop(3230234752,3891989816,3228776128,0,3228273492,...) at tas0 fork_exit(3226753588,3230234752,3891989816) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3891989868, ebp = 0 --- Tracing command swi2: cambio pid 18 tid 100011 td 0xc9454300 sched_switch(3376759552,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3376511456,3892260152,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3376511456,3892260152) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892260204, ebp = 0 --- Tracing command swi5: + pid 17 tid 100012 td 0xc9454180 fork_trampoline() at fork_trampoline Tracing command thread taskq pid 5 tid 100013 td 0xc9454000 sched_switch(3376758784,0,1) at sched_switch+331 mi_switch(1,0,0,3892235432,3226748861,...) at mi_switch+426 sleepq_switch(3377103104,3228776128,0,3228292512,529,...) at sleepq_switch+113 sleepq_wait(3377103104,0,3228293171,73,8,...) at sleepq_wait+33 msleep(3377103104,3377103132,0,3228262727,0) at msleep+525 taskqueue_thread_loop(3228801864,3892235576,3228776128,0,3228273492,...) at tas0 fork_exit(3226753588,3228801864,3892235576) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892235628, ebp = 0 --- Tracing command swi6: Giant taskq pid 16 tid 100014 td 0xc9450d80 kdb_enter(3228375620) at kdb_enter+43 scgetc(3229266944,2) at scgetc+1348 sckbdevent(3377072384,0,3229266944) at sckbdevent+456 kbdmux_intr(3377072384,0,3377105560,3892223160,3226752756,...) at kbdmux_intr+28 kbdmux_kbd_intr(3377072384,1) at kbdmux_kbd_intr+26 taskqueue_run(3377103360,3892223196,3226555506,0,3376511600,...) at taskqueue_r0 taskqueue_swi_giant_run(0) at taskqueue_swi_giant_run+14 ithread_execute_handlers(3377065984,3377103232) at ithread_execute_handlers+206 ithread_loop(3376511600,3892223288,3228776128,0,3228273492,...) at ithread_loop9 fork_exit(3226555672,3376511600,3892223288) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892223340, ebp = 0 --- Tracing command yarrow pid 15 tid 100015 td 0xc9450c00 sched_switch(3376745472,0,1) at sched_switch+331 mi_switch(1,0,0,3892210848,3226749014,...) at mi_switch+426 sleepq_switch(3228757952,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3228757952) at sleepq_timedwait+34 msleep(3228757952,0,0,3228262727,100) at msleep+489 random_kthread(0,3892211000,3228776128,0,3228273492,...) at random_kthread+369 fork_exit(3226233400,0,3892211000) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892211052, ebp = 0 --- Tracing command g_down pid 4 tid 100016 td 0xc9450a80 sched_switch(3376745088,0,1) at sched_switch+331 mi_switch(1,0,0,3892198556,3226749014,...) at mi_switch+426 sleepq_switch(3228767432,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3228767432) at sleepq_timedwait+34 msleep(3228767432,3228767144,588,3228262727,100) at msleep+489 g_io_schedule_down(3376745088) at g_io_schedule_down+86 g_down_procbody(0,3892198712,3228776128,0,3228273492,...) at g_down_procbody+70 fork_exit(3226426420,0,3892198712) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892198764, ebp = 0 --- Tracing command g_up pid 3 tid 100000 td 0xc9450000 sched_switch(3376742400,0,1) at sched_switch+331 mi_switch(1,0,0,3892112544,3226749014,...) at mi_switch+426 sleepq_switch(3228767428,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3228767428) at sleepq_timedwait+34 msleep(3228767428,3228767208,588,3228262727,100) at msleep+489 g_io_schedule_up(3376742400) at g_io_schedule_up+204 g_up_procbody(0,3892112696,3228776128,0,3228273492,...) at g_up_procbody+70 fork_exit(3226426344,0,3892112696) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892112748, ebp = 0 --- Tracing command g_event pid 2 tid 100001 td 0xc944fd80 sched_switch(3376741760,0,1) at sched_switch+331 mi_switch(1,0,0,3892100260,3226749014,...) at mi_switch+426 sleepq_switch(3228767420,3228776128,0,3228292512,566,...) at sleepq_switch+113 sleepq_timedwait(3228767420) at sleepq_timedwait+34 msleep(3228767420,0,76,3228262727,100) at msleep+489 g_event_procbody(0,3892100408,3228776128,0,3228273492,...) at g_event_procbody+0 fork_exit(3226426496,0,3892100408) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892100460, ebp = 0 --- Tracing command swi3: vm pid 14 tid 100002 td 0xc944fc00 fork_trampoline() at fork_trampoline Tracing command swi4: clock sio pid 13 tid 100003 td 0xc944fa80 sched_switch(3376740992,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3376511136,3892075832,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3376511136,3892075832) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892075884, ebp = 0 --- Tracing command swi1: net pid 12 tid 100004 td 0xc944f900 sched_switch(3376740608,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 ithread_loop(3376511152,3892063544,3228776128,0,3228273492,...) at ithread_loop2 fork_exit(3226555672,3376511152,3892063544) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892063596, ebp = 0 --- Tracing command idle: cpu0 pid 11 tid 100005 td 0xc944f780 sched_switch(3376740224,3376743936,6) at sched_switch+331 mi_switch(6,3376743936,3376744280,3376743936,3892050952,...) at mi_switch+426 maybe_preempt(3376743936) at maybe_preempt+196 sched_add(3376743936,4,3376743936,3377734416,3377063168,...) at sched_add+240 setrunqueue(3376743936,4) at setrunqueue+99 intr_event_schedule_thread(3377063168) at intr_event_schedule_thread+163 intr_execute_handlers(3376699000,3892051072,16,3892051140,3227972099,...) at in0 lapic_handle_intr(50) at lapic_handle_intr+46 Xapic_isr1() at Xapic_isr1+51 --- interrupt, eip = 3230175589, esp = 3892051140, ebp = 3892051140 --- acpi_cpu_c1(3228776128,0,3892051172,1,4294967294,...) at acpi_cpu_c1+5 acpi_cpu_idle(3892051196,3226552629,0,3376736888,3892051236,...) at acpi_cpu_id8 cpu_idle(0,3376736888,3892051236,3226552149,0,...) at cpu_idle+40 idle_proc(0,3892051256,3228776128,0,3228273492,...) at idle_proc+77 fork_exit(3226552552,0,3892051256) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892051308, ebp = 0 --- Tracing command idle: cpu1 pid 10 tid 100006 td 0xc944f600 sched_switch(3376739840,0,1) at sched_switch+331 mi_switch(1,0,3228776128,0,3228273740,...) at mi_switch+426 idle_proc(0,3892038968,3228776128,0,3228273492,...) at idle_proc+123 fork_exit(3226552552,0,3892038968) at fork_exit+97 fork_trampoline() at fork_trampoline+8 --- trap 1, eip = 0, esp = 3892039020, ebp = 0 --- Tracing command init pid 1 tid 100007 td 0xc944f480 sched_switch(3376739456,0,1) at sched_switch+331 mi_switch(1,0,0,3892026380,3226748917,...) at mi_switch+426 sleepq_switch(3376754688) at sleepq_switch+113 sleepq_wait_sig(3376754688) at sleepq_wait_sig+29 msleep(3376754688,3376754792,348,3228297560,0) at msleep+510 kern_wait(3376739456,4294967295,3892026500,0,0) at kern_wait+1699 wait4(3376739456,3892026628) at wait4+42 syscall(59,59,59,0,0,...) at syscall+559 Xint0x80_syscall() at Xint0x80_syscall+31 --- syscall (7, FreeBSD ELF32, wait4), eip = 134557511, esp = 3217026428, ebp =- Tracing command swapper pid 0 tid 0 td 0xc07317e0 sched_switch(3228768224,0,1) at sched_switch+331 mi_switch(1,0) at mi_switch+426 scheduler(0,12708864,12705792,0,3225640485,...) at scheduler+385 mi_startup() at mi_startup+150 begin() at begin+44 db> >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200707070525.l675PiIV033863>