Date: Sat, 16 Sep 2006 13:02:56 +0200 From: Frode Nordahl <frode@nordahl.net> To: freebsd-stable@freebsd.org Subject: RELENG_6 Livelock Message-ID: <FC17EAA4-B5B1-48E4-BA69-1D8EF4E0F047@nordahl.net>
next in thread | raw e-mail | index | archive | help
--Apple-Mail-23--291249582
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=US-ASCII;
delsp=yes;
format=flowed
Hello,
FreeBSD localhost.localdomain 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE
#1: Wed Sep 13 00:10:04 CEST 2006 frode@localhost.localdomain:/
usr/obj/usr/src/sys/PT i386
After running some stress tests for 3 days, I wanted to remove some
large directories.
drwxrwxrwt 2 root wheel 34726400 Sep 16 11:43 uidtmp
drwxrwxrwt 2 root wheel 75099648 Sep 3 10:59 uidtmp.bak.1
drwxrwxrwt 2 root wheel 61301760 Sep 12 23:10 uidtmp.bak.2
drwxrwxrwt 2 root wheel 56146944 Sep 15 00:35 uidtmp.bak.3
I fired up 4 copies of rm -rf that ran on separate directories and
monitored the system with iostat.
The directories to be removed was located in the root of an NFS export.
I saw very little I/O and 100% cpu time spent in sys, so I attempted
to kill off all the rm processes. However one of them seemed to be
stuck. After some time the last rm process finally exited.
I had another go at removing the direcotires, firing up 4 copies of
rm on seperate directories simultaniously, and boom. Everything got
stuck.
The machine responded to ping, but nothing else but the debugger.
After the system is back up, attemping to remove one of these
directories still makes the system behave erraticly:
CPU states: 0.0% user, 0.0% nice, 98.0% system, 0.3% interrupt,
1.7% idle
Mem: 115M Active, 1667M Inact, 154M Wired, 64M Cache, 112M Buf, 2996K
Free
Swap: 4071M Total, 4071M Free
PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
929 root 1 -8 0 1284K 592K biord 7:51 102.93% rm
dmesg, output from debugger and kernel config attached
--
Frode Nordahl
--Apple-Mail-23--291249582
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; x-unix-mode=0644; x-mac-hide-extension=yes;
name=dmesg.txt
Content-Disposition: attachment;
filename=dmesg.txt
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 6.2-PRERELEASE #1: Wed Sep 13 00:10:04 CEST 2006
frode@localhost.localdomain:/usr/obj/usr/src/sys/PT
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel Pentium III (999.72-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x686 Stepping = 6
Features=0x387fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,PN,MMX,FXSR,SSE>
real memory = 2147418112 (2047 MB)
avail memory = 2096590848 (1999 MB)
ACPI APIC Table: <INTEL 024B >
ioapic0 <Version 1.1> irqs 0-15 on motherboard
ioapic1 <Version 1.1> irqs 16-31 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.17.2 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
acpi0: <INTEL 024B> on motherboard
acpi0: Overriding SCI Interrupt from IRQ 9 to IRQ 31
acpi0: Power Button (fixed)
Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x404-0x407 on acpi0
acpi_ec0: <Embedded Controller: GPE 0x4> port 0xca6,0xca7 on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_button0: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <display, VGA> at device 2.0 (no driver attached)
fxp0: <Intel 82559 Pro/100 Ethernet> port 0x1400-0x143f mem 0xf0001000-0xf0001fff,0xf0100000-0xf01fffff irq 18 at device 3.0 on pci0
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:d0:b7:b8:1d:f6
isab0: <PCI-ISA bridge> port 0x580-0x58f at device 15.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <ServerWorks ROSB4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x374-0x377,0x1440-0x144f at device 15.1 on pci0
ata0: <ATA channel 0> on atapci0
ata2: <ATA channel 1> on atapci0
pcib1: <ACPI Host-PCI bridge> on acpi0
pci1: <ACPI PCI bus> on pcib1
ahc0: <Adaptec aic7899 Ultra160 SCSI adapter> port 0x1800-0x18ff mem 0xf2010000-0xf2010fff irq 16 at device 4.0 on pci1
ahc0: [GIANT-LOCKED]
aic7899: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs
ahc1: <Adaptec aic7899 Ultra160 SCSI adapter> port 0x2000-0x20ff mem 0xf2011000-0xf2011fff irq 17 at device 4.1 on pci1
ahc1: [GIANT-LOCKED]
aic7899: Ultra160 Wide Channel B, SCSI Id=7, 32/253 SCBs
pcib2: <PCI-PCI bridge> at device 10.0 on pci1
pci2: <PCI bus> on pcib2
pcib3: <PCI-PCI bridge> at device 0.0 on pci2
pci3: <PCI bus> on pcib3
amr0: <LSILogic MegaRAID 1.53> mem 0xf4000000-0xf7ffffff irq 20 at device 0.0 on pci3
amr0: delete logical drives supported by controller
amr0: <LSILogic MegaRAID Elite 1600> Firmware 111U, BIOS F320, 64MB RAM
pci2: <mass storage, SCSI> at device 1.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection Version - 6.1.4> mem 0xf2020000-0xf203ffff,0xf2000000-0xf200ffff irq 21 at device 11.0 on pci1
em0: Ethernet address: 00:03:47:9a:f2:05
em0: [FAST]
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
acpi_tz2: <Thermal Zone> on acpi0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A, console
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
ppc0: <ECP parallel printer port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc97ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounter "TSC" frequency 999721181 Hz quality 800
Timecounters tick every 1.000 msec
Waiting 5 seconds for SCSI devices to settle
amr0: delete logical drives supported by controller
amrd0: <LSILogic MegaRAID logical drive> on amr0
amrd0: 561184MB (1149304832 sectors) RAID 5 (optimal)
em0: link state changed to UP
Trying to mount root from ufs:/dev/amrd0s1a
WARNING: / was not properly dismounted
--Apple-Mail-23--291249582
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
x-unix-mode=0644;
name=debugger.txt
Content-Disposition: attachment;
filename=debugger.txt
KDB: enter: Line break on console
[thread pid 15042 tid 100144 ]
Stopped at kdb_enter+0x2b: nop
db> where
Tracing pid 15042 tid 100144 td 0xcfb11d80
kdb_enter(c0921739) at kdb_enter+0x2b
siointr1(c6554000) at siointr1+0xce
siointr(c6554000) at siointr+0x38
intr_execute_handlers(c63854c4,ecfce8f8,4,ecfce950,c087f853,...) at intr_execute_handlers+0xe1
lapic_handle_intr(38) at lapic_handle_intr+0x2e
Xapic_isr1() at Xapic_isr1+0x33
--- interrupt, eip = 0xc07d79dc, esp = 0xecfce93c, ebp = 0xecfce950 ---
ufsdirhash_adjfree(ca780200,a9d720,fffffff0) at ufsdirhash_adjfree+0x80
ufsdirhash_build(c68414a4) at ufsdirhash_build+0x4f0
ufs_lookup(ecfcea7c) at ufs_lookup+0xf9
VOP_CACHEDLOOKUP_APV(c09b5300,ecfcea7c) at VOP_CACHEDLOOKUP_APV+0x38
vfs_cache_lookup(ecfceb18) at vfs_cache_lookup+0xb2
VOP_LOOKUP_APV(c09b5300,ecfceb18) at VOP_LOOKUP_APV+0x43
lookup(ecfceba0) at lookup+0x4c1
namei(ecfceba0) at namei+0x38e
kern_lstat(cfb11d80,804f2a8,0,ecfcec74) at kern_lstat+0x47
lstat(cfb11d80,ecfced04) at lstat+0x1b
syscall(3b,3b,3b,804f248,804f200,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (190, FreeBSD ELF32, lstat), eip = 0x2812d473, esp = 0xbfbfeb3c, ebp = 0xbfbfebd8 ---
db> show allpcpu
Current CPU: 0
cpuid = 0
curthread = 0xcfb11d80: pid 15042 "rm"
curpcb = 0xecfced90
fpcurthread = none
idlethread = 0xc638b600: pid 10 "idle"
APIC ID = 0
currentldt = 0x50
db> show lock
db> show lockedvnods
Locked vnodes
0xcc21f880: tag ufs, type VDIR
usecount 3, writecount 0, refcount 685 mountedhere 0
flags ()
v_object 0xcc1208c4 ref 0 pages 2724
lock type ufs: EXCL (count 1) by thread 0xcfb11d80 (pid 15042)
ino 43490161, on dev amrd0s1g
0xcc04bdd0: tag ufs, type VDIR
usecount 3, writecount 0, refcount 682 mountedhere 0
flags ()
v_object 0xc7538108 ref 0 pages 2712
lock type ufs: EXCL (count 1) by thread 0xc697f000 (pid 15041)
ino 31210587, on dev amrd0s1g
db> allt
Tracing command rm pid 15044 tid 100155 td 0xcfb47000
sched_switch(cfb47000,0,2) at sched_switch+0x14b
mi_switch(2,0,cfb47000,b4) at mi_switch+0x1ba
ast(ed034d38) at ast+0x3c5
doreti_ast() at doreti_ast+0x17
Tracing command rm pid 15043 tid 100114 td 0xcfb11a80
sched_switch(cfb11a80,0,2) at sched_switch+0x14b
mi_switch(2,0) at mi_switch+0x1ba
critical_exit(ecfc8bc8,ecfc8b94,c087fa80,0,cfb10008,...) at critical_exit+0x78
lapic_handle_timer(0) at lapic_handle_timer+0xc9
Xtimerint(ecfc8ba0) at Xtimerint+0x30
kern_lstat(cfb11a80,805dfa8,0,ecfc8c74) at kern_lstat+0x47
lstat(cfb11a80,ecfc8d04) at lstat+0x1b
syscall(3b,3b,3b,805df48,805df00,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (190, FreeBSD ELF32, lstat), eip = 0x2812d473, esp = 0xbfbfeb3c, ebp = 0xbfbfebd8 ---
Tracing command rm pid 15042 tid 100144 td 0xcfb11d80
kdb_enter(c0921739) at kdb_enter+0x2b
siointr1(c6554000) at siointr1+0xce
siointr(c6554000) at siointr+0x38
intr_execute_handlers(c63854c4,ecfce8f8,4,ecfce950,c087f853,...) at intr_execute_handlers+0xe1
lapic_handle_intr(38) at lapic_handle_intr+0x2e
Xapic_isr1() at Xapic_isr1+0x33
--- interrupt, eip = 0xc07d79dc, esp = 0xecfce93c, ebp = 0xecfce950 ---
ufsdirhash_adjfree(ca780200,a9d720,fffffff0) at ufsdirhash_adjfree+0x80
ufsdirhash_build(c68414a4) at ufsdirhash_build+0x4f0
ufs_lookup(ecfcea7c) at ufs_lookup+0xf9
VOP_CACHEDLOOKUP_APV(c09b5300,ecfcea7c) at VOP_CACHEDLOOKUP_APV+0x38
vfs_cache_lookup(ecfceb18) at vfs_cache_lookup+0xb2
VOP_LOOKUP_APV(c09b5300,ecfceb18) at VOP_LOOKUP_APV+0x43
lookup(ecfceba0) at lookup+0x4c1
namei(ecfceba0) at namei+0x38e
kern_lstat(cfb11d80,804f2a8,0,ecfcec74) at kern_lstat+0x47
lstat(cfb11d80,ecfced04) at lstat+0x1b
syscall(3b,3b,3b,804f248,804f200,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (190, FreeBSD ELF32, lstat), eip = 0x2812d473, esp = 0xbfbfeb3c, ebp = 0xbfbfebd8 ---
Tracing command rm pid 15041 tid 100075 td 0xc697f000
sched_switch(c697f000,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
turnstile_wait(c09e6800,cfb11d80) at turnstile_wait+0x280
_mtx_lock_sleep(c09e6800,c697f000,0,0,0) at _mtx_lock_sleep+0xb4
msleep(da608828,c09ec1a0,4c,c090b351,0) at msleep+0x29b
bwait(da608828,4c,c090b351) at bwait+0x5b
bufwait(da608828,1,0,0,2a4,...) at bufwait+0x1a
breadn(cc04bdd0,2a4,0,4000,0,...) at breadn+0x1e6
bread(cc04bdd0,2a4,0,4000,0,ecef6938) at bread+0x20
ffs_blkatoff(cc04bdd0,a90000,0,0,ecef6984) at ffs_blkatoff+0x9e
ufsdirhash_build(d3bbb420) at ufsdirhash_build+0x428
ufs_lookup(ecef6a7c) at ufs_lookup+0xf9
VOP_CACHEDLOOKUP_APV(c09b5300,ecef6a7c) at VOP_CACHEDLOOKUP_APV+0x38
vfs_cache_lookup(ecef6b18) at vfs_cache_lookup+0xb2
VOP_LOOKUP_APV(c09b5300,ecef6b18) at VOP_LOOKUP_APV+0x43
lookup(ecef6ba0) at lookup+0x4c1
namei(ecef6ba0) at namei+0x38e
kern_lstat(c697f000,804f2a8,0,ecef6c74) at kern_lstat+0x47
lstat(c697f000,ecef6d04) at lstat+0x1b
syscall(3b,3b,3b,804f248,804f200,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (190, FreeBSD ELF32, lstat), eip = 0x2812d473, esp = 0xbfbfeb3c, ebp = 0xbfbfebd8 ---
Tracing command gstat pid 15038 tid 100064 td 0xc6737000
sched_switch(c6737000,0,1) at sched_switch+0x14b
mi_switch(1,0,c09e70ec,eacc1c48,c069bf6a,...) at mi_switch+0x1ba
sleepq_switch(c09e70ec) at sleepq_switch+0x86
sleepq_timedwait_sig(c09e70ec) at sleepq_timedwait_sig+0x1e
msleep(c09e70ec,0,15c,c0907bb7,1f5,...) at msleep+0x200
kern_nanosleep(c6737000,eacc1ccc,eacc1cc4) at kern_nanosleep+0xab
nanosleep(c6737000,eacc1d04) at nanosleep+0x4f
syscall(3b,3b,3b,8055214,c,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28199f63, esp = 0xbfbfea0c, ebp = 0xbfbfea38 ---
Tracing command bash pid 14979 tid 100140 td 0xcfb2c480
sched_switch(cfb2c480,0,1) at sched_switch+0x14b
mi_switch(1,0,c6516010,ecff49bc,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6516010) at sleepq_switch+0x86
sleepq_wait_sig(c6516010) at sleepq_wait_sig+0x1d
msleep(c6516010,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6516000,c6516010,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6516000,ecff4cbc,0) at ttread+0x50f
ptsread(c7f34d00,ecff4cbc,0) at ptsread+0x2d
giant_read(c7f34d00,ecff4cbc,0,1,c7f34d00,...) at giant_read+0x48
devfs_read_f(c69564c8,ecff4cbc,c83d9d00,0,cfb2c480) at devfs_read_f+0x62
dofileread(cfb2c480,0,c69564c8,ecff4cbc,ffffffff,...) at dofileread+0x85
kern_readv(cfb2c480,0,ecff4cbc,bfbfe80f,1,...) at kern_readv+0x36
read(cfb2c480,ecff4d04) at read+0x45
syscall(80f003b,3b,bfbf003b,0,bfbfe80f,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x282f638f, esp = 0xbfbfe7ec, ebp = 0xbfbfe818 ---
Tracing command sshd pid 14975 tid 100078 td 0xc6738a80
sched_switch(c6738a80,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,eacf4b28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(c6738a80,8,807ab50,807ab60,0,...) at kern_select+0x56f
select(c6738a80,eacf4d04) at select+0x44
syscall(807003b,807003b,bfbf003b,0,bfbfde7c,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2834730f, esp = 0xbfbfde0c, ebp = 0xbfbfde58 ---
Tracing command bash pid 14970 tid 100143 td 0xcfb2c000
sched_switch(cfb2c000,0,1) at sched_switch+0x14b
mi_switch(1,0,cfb46648,ecfebc18,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(cfb46648) at sleepq_switch+0x86
sleepq_wait_sig(cfb46648) at sleepq_wait_sig+0x1d
msleep(cfb46648,cfb466b0,15c,c0907390,0) at msleep+0x22a
kern_wait(cfb2c000,ffffffff,ecfebc84,6,0) at kern_wait+0x89c
wait4(cfb2c000,ecfebd04) at wait4+0x2a
syscall(80f003b,3b,bfbf003b,3abe,0,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x2828d70f, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 ---
Tracing command sshd pid 14967 tid 100152 td 0xcfb47480
sched_switch(cfb47480,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,ed03db28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(cfb47480,8,807ab50,807ab60,0,...) at kern_select+0x56f
select(cfb47480,ed03dd04) at select+0x44
syscall(bfbf003b,bfbf003b,bfbf003b,0,bfbfde7c,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2834730f, esp = 0xbfbfde0c, ebp = 0xbfbfde58 ---
Tracing command bash pid 14952 tid 100171 td 0xcfb48900
sched_switch(cfb48900,0,1) at sched_switch+0x14b
mi_switch(1,0,c6552010,ed0649bc,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6552010) at sleepq_switch+0x86
sleepq_wait_sig(c6552010) at sleepq_wait_sig+0x1d
msleep(c6552010,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6552000,c6552010,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6552000,ed064cbc,0) at ttread+0x50f
ptsread(c6a3e000,ed064cbc,0) at ptsread+0x2d
giant_read(c6a3e000,ed064cbc,0,1,c6a3e000,...) at giant_read+0x48
devfs_read_f(c87d9510,ed064cbc,cf16c900,0,cfb48900) at devfs_read_f+0x62
dofileread(cfb48900,0,c87d9510,ed064cbc,ffffffff,...) at dofileread+0x85
kern_readv(cfb48900,0,ed064cbc,bfbfe80f,1,...) at kern_readv+0x36
read(cfb48900,ed064d04) at read+0x45
syscall(3b,3b,3b,0,bfbfe80f,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x282f638f, esp = 0xbfbfe7ec, ebp = 0xbfbfe818 ---
Tracing command sshd pid 14949 tid 100168 td 0xcfb48d80
sched_switch(cfb48d80,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,ed06db28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(cfb48d80,8,807ab50,807ab60,0,...) at kern_select+0x56f
select(cfb48d80,ed06dd04) at select+0x44
syscall(bfbf003b,807003b,bfbf003b,0,bfbfde7c,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2834730f, esp = 0xbfbfde0c, ebp = 0xbfbfde58 ---
Tracing command getty pid 852 tid 100067 td 0xc697fc00
sched_switch(c697fc00,0,1) at sched_switch+0x14b
mi_switch(1,0,c6385c10,ecf0e9c4,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6385c10) at sleepq_switch+0x86
sleepq_wait_sig(c6385c10) at sleepq_wait_sig+0x1d
msleep(c6385c10,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6385c00,c6385c10,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6385c00,ecf0ecbc,0) at ttread+0x50f
ttyread(c6557400,ecf0ecbc,0) at ttyread+0x2f
giant_read(c6557400,ecf0ecbc,0,1,c6557400,...) at giant_read+0x48
devfs_read_f(c6759ca8,ecf0ecbc,c6388d80,0,c697fc00) at devfs_read_f+0x62
dofileread(c697fc00,0,c6759ca8,ecf0ecbc,ffffffff,...) at dofileread+0x85
kern_readv(c697fc00,0,ecf0ecbc,bfbfee1f,1,...) at kern_readv+0x36
read(c697fc00,ecf0ed04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command getty pid 851 tid 100063 td 0xc6737180
sched_switch(c6737180,0,1) at sched_switch+0x14b
mi_switch(1,0,c6569010,eacc49b0,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6569010) at sleepq_switch+0x86
sleepq_wait_sig(c6569010) at sleepq_wait_sig+0x1d
msleep(c6569010,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6569000,c6569010,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6569000,eacc4cbc,0) at ttread+0x50f
ttyread(c6558600,eacc4cbc,0,eacc4c14,c0653514,...) at ttyread+0x2f
scread(c6558600,eacc4cbc,0) at scread+0x22
giant_read(c6558600,eacc4cbc,0,1,c6558600,...) at giant_read+0x48
devfs_read_f(c675ad80,eacc4cbc,c6388d80,0,c6737180) at devfs_read_f+0x62
dofileread(c6737180,0,c675ad80,eacc4cbc,ffffffff,...) at dofileread+0x85
kern_readv(c6737180,0,eacc4cbc,bfbfee1f,1,...) at kern_readv+0x36
read(c6737180,eacc4d04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command getty pid 850 tid 100068 td 0xc697fa80
sched_switch(c697fa80,0,1) at sched_switch+0x14b
mi_switch(1,0,c6568c10,ecf0b9b0,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6568c10) at sleepq_switch+0x86
sleepq_wait_sig(c6568c10) at sleepq_wait_sig+0x1d
msleep(c6568c10,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6568c00,c6568c10,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6568c00,ecf0bcbc,0) at ttread+0x50f
ttyread(c6558700,ecf0bcbc,0,ecf0bc14,c0653514,...) at ttyread+0x2f
scread(c6558700,ecf0bcbc,0) at scread+0x22
giant_read(c6558700,ecf0bcbc,0,1,c6558700,...) at giant_read+0x48
devfs_read_f(c675a5a0,ecf0bcbc,c6388d80,0,c697fa80) at devfs_read_f+0x62
dofileread(c697fa80,0,c675a5a0,ecf0bcbc,ffffffff,...) at dofileread+0x85
kern_readv(c697fa80,0,ecf0bcbc,bfbfee1f,1,...) at kern_readv+0x36
read(c697fa80,ecf0bd04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command getty pid 849 tid 100066 td 0xc697fd80
sched_switch(c697fd80,0,1) at sched_switch+0x14b
mi_switch(1,0,c6569c10,ecf119b0,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6569c10) at sleepq_switch+0x86
sleepq_wait_sig(c6569c10) at sleepq_wait_sig+0x1d
msleep(c6569c10,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6569c00,c6569c10,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6569c00,ecf11cbc,0) at ttread+0x50f
ttyread(c6558800,ecf11cbc,0,ecf11c14,c0653514,...) at ttyread+0x2f
scread(c6558800,ecf11cbc,0) at scread+0x22
giant_read(c6558800,ecf11cbc,0,1,c6558800,...) at giant_read+0x48
devfs_read_f(c675a798,ecf11cbc,c6388d80,0,c697fd80) at devfs_read_f+0x62
dofileread(c697fd80,0,c675a798,ecf11cbc,ffffffff,...) at dofileread+0x85
kern_readv(c697fd80,0,ecf11cbc,bfbfee1f,1,...) at kern_readv+0x36
read(c697fd80,ecf11d04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command getty pid 848 tid 100071 td 0xc697f600
sched_switch(c697f600,0,1) at sched_switch+0x14b
mi_switch(1,0,c6569810,ecf029b0,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6569810) at sleepq_switch+0x86
sleepq_wait_sig(c6569810) at sleepq_wait_sig+0x1d
msleep(c6569810,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6569800,c6569810,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6569800,ecf02cbc,0) at ttread+0x50f
ttyread(c6558900,ecf02cbc,0,ecf02c14,c0653514,...) at ttyread+0x2f
scread(c6558900,ecf02cbc,0) at scread+0x22
giant_read(c6558900,ecf02cbc,0,1,c6558900,...) at giant_read+0x48
devfs_read_f(c675ac60,ecf02cbc,c6388d80,0,c697f600) at devfs_read_f+0x62
dofileread(c697f600,0,c675ac60,ecf02cbc,ffffffff,...) at dofileread+0x85
kern_readv(c697f600,0,ecf02cbc,bfbfee1f,1,...) at kern_readv+0x36
read(c697f600,ecf02d04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command getty pid 847 tid 100069 td 0xc697f900
sched_switch(c697f900,0,1) at sched_switch+0x14b
mi_switch(1,0,c6568410,ecf089b0,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6568410) at sleepq_switch+0x86
sleepq_wait_sig(c6568410) at sleepq_wait_sig+0x1d
msleep(c6568410,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6568400,c6568410,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6568400,ecf08cbc,0) at ttread+0x50f
ttyread(c6558a00,ecf08cbc,0,ecf08c14,c0653514,...) at ttyread+0x2f
scread(c6558a00,ecf08cbc,0) at scread+0x22
giant_read(c6558a00,ecf08cbc,0,1,c6558a00,...) at giant_read+0x48
devfs_read_f(c675a1f8,ecf08cbc,c6388d80,0,c697f900) at devfs_read_f+0x62
dofileread(c697f900,0,c675a1f8,ecf08cbc,ffffffff,...) at dofileread+0x85
kern_readv(c697f900,0,ecf08cbc,bfbfee1f,1,...) at kern_readv+0x36
read(c697f900,ecf08d04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command getty pid 846 tid 100073 td 0xc697f300
sched_switch(c697f300,0,1) at sched_switch+0x14b
mi_switch(1,0,c6568010,ecefc9b0,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6568010) at sleepq_switch+0x86
sleepq_wait_sig(c6568010) at sleepq_wait_sig+0x1d
msleep(c6568010,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6568000,c6568010,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6568000,ecefccbc,0) at ttread+0x50f
ttyread(c655aa00,ecefccbc,0,ecefcc14,c0653514,...) at ttyread+0x2f
scread(c655aa00,ecefccbc,0) at scread+0x22
giant_read(c655aa00,ecefccbc,0,1,c655aa00,...) at giant_read+0x48
devfs_read_f(c675a828,ecefccbc,c6388d80,0,c697f300) at devfs_read_f+0x62
dofileread(c697f300,0,c675a828,ecefccbc,ffffffff,...) at dofileread+0x85
kern_readv(c697f300,0,ecefccbc,bfbfee1f,1,...) at kern_readv+0x36
read(c697f300,ecefcd04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command getty pid 845 tid 100070 td 0xc697f780
sched_switch(c697f780,0,1) at sched_switch+0x14b
mi_switch(1,0,c6567810,ecf059b0,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6567810) at sleepq_switch+0x86
sleepq_wait_sig(c6567810) at sleepq_wait_sig+0x1d
msleep(c6567810,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6567800,c6567810,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6567800,ecf05cbc,0) at ttread+0x50f
ttyread(c655a300,ecf05cbc,0,ecf05c14,c0653514,...) at ttyread+0x2f
scread(c655a300,ecf05cbc,0) at scread+0x22
giant_read(c655a300,ecf05cbc,0,1,c655a300,...) at giant_read+0x48
devfs_read_f(c675ac18,ecf05cbc,c6388d80,0,c697f780) at devfs_read_f+0x62
dofileread(c697f780,0,c675ac18,ecf05cbc,ffffffff,...) at dofileread+0x85
kern_readv(c697f780,0,ecf05cbc,bfbfee1f,1,...) at kern_readv+0x36
read(c697f780,ecf05d04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command getty pid 844 tid 100072 td 0xc697f480
sched_switch(c697f480,0,1) at sched_switch+0x14b
mi_switch(1,0,c6567c10,eceff9b0,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6567c10) at sleepq_switch+0x86
sleepq_wait_sig(c6567c10) at sleepq_wait_sig+0x1d
msleep(c6567c10,0,159,c0909f78,0) at msleep+0x22a
ttysleep(c6567c00,c6567c10,159,c0909f78,0,...) at ttysleep+0x21
ttread(c6567c00,eceffcbc,0) at ttread+0x50f
ttyread(c655a000,eceffcbc,0,eceffc14,c0653514,...) at ttyread+0x2f
scread(c655a000,eceffcbc,0) at scread+0x22
giant_read(c655a000,eceffcbc,0,1,c655a000,...) at giant_read+0x48
devfs_read_f(c675a5e8,eceffcbc,c6388d80,0,c697f480) at devfs_read_f+0x62
dofileread(c697f480,0,c675a5e8,eceffcbc,ffffffff,...) at dofileread+0x85
kern_readv(c697f480,0,eceffcbc,bfbfee1f,1,...) at kern_readv+0x36
read(c697f480,eceffd04) at read+0x45
syscall(3b,3b,3b,0,804e580,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814738f, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
Tracing command cron pid 802 tid 100055 td 0xc6737d80
sched_switch(c6737d80,0,1) at sched_switch+0x14b
mi_switch(1,0,c09e70ec,eacdcc48,c069bf6a,...) at mi_switch+0x1ba
sleepq_switch(c09e70ec) at sleepq_switch+0x86
sleepq_timedwait_sig(c09e70ec) at sleepq_timedwait_sig+0x1e
msleep(c09e70ec,0,15c,c0907bb7,ea61,...) at msleep+0x200
kern_nanosleep(c6737d80,eacdcccc,eacdccc4) at kern_nanosleep+0xab
nanosleep(c6737d80,eacdcd04) at nanosleep+0x4f
syscall(805003b,3b,bfbf003b,805008c,3c,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28130f63, esp = 0xbfbfecdc, ebp = 0xbfbfed08 ---
Tracing command sendmail pid 796 tid 100059 td 0xc6737780
sched_switch(c6737780,0,1) at sched_switch+0x14b
mi_switch(1,0,c68e067c,eacd0c68,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c68e067c) at sleepq_switch+0x86
sleepq_wait_sig(c68e067c) at sleepq_wait_sig+0x1d
msleep(c68e067c,c68e06b0,168,c08d5a58,0) at msleep+0x22a
kern_sigsuspend(c6737780,0,0,0,0,...) at kern_sigsuspend+0xbf
sigsuspend(c6737780,eacd0d04) at sigsuspend+0x33
syscall(3b,bfbf003b,bfbf003b,0,bfbfd000,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x282797f7, esp = 0xbfbfcfec, ebp = 0xbfbfd018 ---
Tracing command sendmail pid 792 tid 100058 td 0xc6737900
sched_switch(c6737900,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,eacd3b1c,c069bf6a,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_timedwait_sig(c09ebb64,c09ebb64,1389,c09ebb64,c09ebb40,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c09ebb64,c09ebb40,1389) at cv_timedwait_sig+0x11e
kern_select(c6737900,5,bfbfc500,0,0,...) at kern_select+0x559
select(c6737900,eacd3d04) at select+0x44
syscall(80d003b,bfbf003b,bfbf003b,1,1,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x282f730f, esp = 0xbfbfc40c, ebp = 0xbfbfd008 ---
Tracing command sshd pid 786 tid 100050 td 0xc6738600
sched_switch(c6738600,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,eacebb28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(c6738600,7,807a8e0,0,0,...) at kern_select+0x56f
select(c6738600,eacebd04) at select+0x44
syscall(3b,3b,bfbf003b,1,a,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2834730f, esp = 0xbfbfdecc, ebp = 0xbfbfee78 ---
Tracing command ntpd pid 765 tid 100043 td 0xc6533300
sched_switch(c6533300,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,e6c71b28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(c6533300,b,bfbfed60,0,0,...) at kern_select+0x56f
select(c6533300,e6c71d04) at select+0x44
syscall(c087003b,3b,3b,bfbfed60,0,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2828930f, esp = 0xbfbfed1c, ebp = 0xbfbfedf8 ---
Tracing command rpc.lockd pid 731 tid 100051 td 0xc6738480
sched_switch(c6738480,0,1) at sched_switch+0x14b
mi_switch(1,0,c09f4128,eace8bcc,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09f4128) at sleepq_switch+0x86
sleepq_wait_sig(c09f4128) at sleepq_wait_sig+0x1d
msleep(c09f4128,c09f4100,158,c0914b2b,0) at msleep+0x22a
nfslock_read(c6412000,eace8cbc,4,194,c6412000,...) at nfslock_read+0x7f
devfs_read_f(c675a8b8,eace8cbc,c6902000,0,c6738480) at devfs_read_f+0x62
dofileread(c6738480,8,c675a8b8,eace8cbc,ffffffff,...) at dofileread+0x85
kern_readv(c6738480,8,eace8cbc,bfbfec60,194,...) at kern_readv+0x36
read(c6738480,eace8d04) at read+0x45
syscall(3b,3b,3b,4,806e300,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (3, FreeBSD ELF32, read), eip = 0x2815438f, esp = 0xbfbfec2c, ebp = 0xbfbfee08 ---
Tracing command rpc.lockd pid 725 tid 100038 td 0xc6533a80
sched_switch(c6533a80,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,e6c80b1c,c069bf6a,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_timedwait_sig(c09ebb64,c09ebb64,7531,c09ebb64,c09ebb40,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c09ebb64,c09ebb40,7531) at cv_timedwait_sig+0x11e
kern_select(c6533a80,7,bfbfed70,0,0,...) at kern_select+0x559
select(c6533a80,e6c80d04) at select+0x44
syscall(2817003b,bfbf003b,bfbf003b,2817a3a0,bfbfed70,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815430f, esp = 0xbfbfecac, ebp = 0xbfbfee08 ---
Tracing command rpc.statd pid 720 tid 100052 td 0xc6738300
sched_switch(c6738300,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,eace5b1c,c069bf6a,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_timedwait_sig(c09ebb64,c09ebb64,7531,c09ebb64,c09ebb40,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c09ebb64,c09ebb40,7531) at cv_timedwait_sig+0x11e
kern_select(c6738300,8,bfbfed80,0,0,...) at kern_select+0x559
select(c6738300,eace5d04) at select+0x44
syscall(2816003b,bfbf003b,bfbf003b,281683a0,bfbfed80,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2814230f, esp = 0xbfbfecbc, ebp = 0xbfbfee18 ---
Tracing command nfsd pid 718 tid 100053 td 0xc6738180
sched_switch(c6738180,0,1) at sched_switch+0x14b
mi_switch(1,0,c671f000,eace2c40,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c671f000) at sleepq_switch+0x86
sleepq_wait_sig(c671f000) at sleepq_wait_sig+0x1d
msleep(c671f000,c09f91c0,158,c0902a56,0,...) at msleep+0x22a
nfssvc_nfsd(c6738180) at nfssvc_nfsd+0xdd
nfssvc(c6738180,eace2d04) at nfssvc+0x1bc
syscall(3b,3b,3b,1,0,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280bd1f7, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 ---
Tracing command nfsd pid 717 tid 100062 td 0xc6737300
sched_switch(c6737300,0,1) at sched_switch+0x14b
mi_switch(1,0,c6720800,eacc7c40,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6720800) at sleepq_switch+0x86
sleepq_wait_sig(c6720800) at sleepq_wait_sig+0x1d
msleep(c6720800,c09f91c0,158,c0902a56,0,...) at msleep+0x22a
nfssvc_nfsd(c6737300) at nfssvc_nfsd+0xdd
nfssvc(c6737300,eacc7d04) at nfssvc+0x1bc
syscall(3b,3b,3b,1,0,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280bd1f7, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 ---
Tracing command nfsd pid 716 tid 100057 td 0xc6737a80
sched_switch(c6737a80,0,1) at sched_switch+0x14b
mi_switch(1,0,c6715800,eacd6c40,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c6715800) at sleepq_switch+0x86
sleepq_wait_sig(c6715800) at sleepq_wait_sig+0x1d
msleep(c6715800,c09f91c0,158,c0902a56,0,...) at msleep+0x22a
nfssvc_nfsd(c6737a80) at nfssvc_nfsd+0xdd
nfssvc(c6737a80,eacd6d04) at nfssvc+0x1bc
syscall(3b,3b,3b,1,0,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280bd1f7, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 ---
Tracing command nfsd pid 712 tid 100056 td 0xc6737c00
sched_switch(c6737c00,0,1) at sched_switch+0x14b
mi_switch(1,0,c671f200,eacd9c40,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c671f200) at sleepq_switch+0x86
sleepq_wait_sig(c671f200) at sleepq_wait_sig+0x1d
msleep(c671f200,c09f91c0,158,c0902a56,0,...) at msleep+0x22a
nfssvc_nfsd(c6737c00) at nfssvc_nfsd+0xdd
nfssvc(c6737c00,eacd9d04) at nfssvc+0x1bc
syscall(3b,3b,3b,1,0,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x280bd1f7, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 ---
Tracing command nfsd pid 710 tid 100042 td 0xc6533480
sched_switch(c6533480,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,e6c74b28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(c6533480,5,bfbfed30,0,0,...) at kern_select+0x56f
select(c6533480,e6c74d04) at select+0x44
syscall(3b,3b,3b,bfbfee40,4,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2813a30f, esp = 0xbfbfeb2c, ebp = 0xbfbfee68 ---
Tracing command mountd pid 708 tid 100040 td 0xc6533780
sched_switch(c6533780,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,e6c7ab28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(c6533780,8,bfbfede0,0,0,...) at kern_select+0x56f
select(c6533780,e6c7ad04) at select+0x44
syscall(3b,3b,3b,8,0,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2814a30f, esp = 0xbfbfed3c, ebp = 0xbfbfee78 ---
Tracing command rpcbind pid 639 tid 100039 td 0xc6533900
sched_switch(c6533900,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,e6c7db60,c069bf6a,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_timedwait_sig(c09ebb64,c09ebb64,7531,c09ebb64,c09ebb40,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c09ebb64,c09ebb40,7531) at cv_timedwait_sig+0x11e
poll(c6533900,e6c7dd04) at poll+0x48f
syscall(3b,3b,bfbf003b,1,7,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (209, FreeBSD ELF32, poll), eip = 0x2810adff, esp = 0xbfbfcc5c, ebp = 0xbfbfee28 ---
Tracing command syslogd pid 604 tid 100054 td 0xc6738000
sched_switch(c6738000,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,eacdfb28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(c6738000,9,8056060,0,0,...) at kern_select+0x56f
select(c6738000,eacdfd04) at select+0x44
syscall(805003b,bfbf003b,bfbf003b,8056064,8056060,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x2814b30f, esp = 0xbfbfda2c, ebp = 0xbfbfee78 ---
Tracing command devd pid 530 tid 100061 td 0xc6737480
sched_switch(c6737480,0,1) at sched_switch+0x14b
mi_switch(1,0,c09ebb64,eaccab28,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c09ebb64) at sleepq_switch+0x86
sleepq_wait_sig(c09ebb64,c09ebb64,c09ebb40,c09094fa,101,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c09ebb64,c09ebb40) at cv_wait_sig+0x113
kern_select(c6737480,5,bfbfe9d0,0,0,...) at kern_select+0x56f
select(c6737480,eaccad04) at select+0x44
syscall(3b,3b,3b,1,8,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (93, FreeBSD ELF32, select), eip = 0x80757f7, esp = 0xbfbfe96c, ebp = 0xbfbfee78 ---
Tracing command schedcpu pid 43 tid 100045 td 0xc6533000
sched_switch(c6533000,0,1) at sched_switch+0x14b
mi_switch(1,0,c6533000,e6c6bccc,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(e6c6bd08,e6c6bd08,e6c6bcec,c06835ed,e6c6bd08,...) at sleepq_switch+0x86
sleepq_timedwait(e6c6bd08) at sleepq_timedwait+0x36
msleep(e6c6bd08,0,0,c0902a56,3e8) at msleep+0x215
schedcpu_thread(0,e6c6bd38) at schedcpu_thread+0x26
fork_exit(c068e408,0,e6c6bd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6c6bd6c, ebp = 0 ---
Tracing command softdepflush pid 42 tid 100046 td 0xc6493d80
sched_switch(c6493d80,0,1) at sched_switch+0x14b
mi_switch(1,0,c6493d80,e6a2ccc0,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c09f9554,c6493d80,e6a2cce0,c06835ed,c09f9554,...) at sleepq_switch+0x86
sleepq_timedwait(c09f9554) at sleepq_timedwait+0x36
msleep(c09f9554,c09f9520,44,c0918caf,3e8) at msleep+0x215
softdep_flush(0,e6a2cd38) at softdep_flush+0x2b9
fork_exit(c07c4b6c,0,e6a2cd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6a2cd6c, ebp = 0 ---
Tracing command syncer pid 41 tid 100047 td 0xc6493c00
sched_switch(c6493c00,0,1) at sched_switch+0x14b
mi_switch(1,0,c6493c00,e6a29cb0,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c09e6e5c) at sleepq_switch+0x86
sleepq_wait(c09e6e5c,0,c6493c00,0,c651e700,...) at sleepq_wait+0x36
msleep(c09e6e5c,0,68,c090c238,0) at msleep+0x239
sched_sync(0,e6a29d38) at sched_sync+0x3d6
fork_exit(c06d4960,0,e6a29d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6a29d6c, ebp = 0 ---
Tracing command vnlru pid 40 tid 100048 td 0xc6493a80
sched_switch(c6493a80,0,1) at sched_switch+0x14b
mi_switch(1,0,c6493a80,e6a26cb8,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c6735c90,c6493a80,e6a26cd8,c06835ed,c6735c90,...) at sleepq_switch+0x86
sleepq_timedwait(c6735c90) at sleepq_timedwait+0x36
msleep(c6735c90,c09ec4c0,250,c090c19d,3e8,c09ec540) at msleep+0x215
vnlru_proc(0,e6a26d38) at vnlru_proc+0x113
fork_exit(c06d2d20,0,e6a26d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6a26d6c, ebp = 0 ---
Tracing command bufdaemon pid 39 tid 100049 td 0xc6493900
sched_switch(c6493900,0,1) at sched_switch+0x14b
mi_switch(1,0,c6493900,e6a23cd0,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c09ec0cc,c6493900,e6a23cf0,c06835ed,c09ec0cc,...) at sleepq_switch+0x86
sleepq_timedwait(c09ec0cc) at sleepq_timedwait+0x36
msleep(c09ec0cc,c09ec0e0,44,c090b1fa,3e8) at msleep+0x215
buf_daemon(0,e6a23d38) at buf_daemon+0x16b
fork_exit(c06c55f0,0,e6a23d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6a23d6c, ebp = 0 ---
Tracing command pagezero pid 38 tid 100027 td 0xc6493180
sched_switch(c6493180,0,1) at sched_switch+0x14b
mi_switch(1,0,c6493180,e6a14cd4,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c09fa4c4,4,e6a14cf4,c06835ed,c09fa4c4,...) at sleepq_switch+0x86
sleepq_timedwait(c09fa4c4) at sleepq_timedwait+0x36
msleep(c09fa4c4,c09f9fa0,200,c091c89d,493e0) at msleep+0x215
vm_pagezero(0,e6a14d38) at vm_pagezero+0x8a
fork_exit(c07fa45c,0,e6a14d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6a14d6c, ebp = 0 ---
Tracing command vmdaemon pid 37 tid 100028 td 0xc6493000
sched_switch(c6493000,0,1) at sched_switch+0x14b
mi_switch(1,0,c6493000,e6a11ca8,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c09fa014) at sleepq_switch+0x86
sleepq_wait(c09fa014,0,c6493000,c6530000,c07f9394,...) at sleepq_wait+0x36
msleep(c09fa014,0,68,c090b1fa,0) at msleep+0x239
vm_daemon(0,e6a11d38) at vm_daemon+0x4f
fork_exit(c07f9394,0,e6a11d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6a11d6c, ebp = 0 ---
Tracing command pagedaemon pid 36 tid 100029 td 0xc6391d80
sched_switch(c6391d80,0,1) at sched_switch+0x14b
mi_switch(1,0,c6391d80,e4febccc,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c09f9fd0,c6391d80,e4febcec,c06835ed,c09f9fd0,...) at sleepq_switch+0x86
sleepq_timedwait(c09f9fd0) at sleepq_timedwait+0x36
msleep(c09f9fd0,c09f9fa0,44,c090b1fa,1388) at msleep+0x215
vm_pageout(0,e4febd38) at vm_pageout+0x280
fork_exit(c07f9004,0,e4febd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4febd6c, ebp = 0 ---
Tracing command irq1: atkbd0 pid 35 tid 100030 td 0xc6391c00
fork_trampoline() at fork_trampoline
Tracing command fdc0 pid 34 tid 100031 td 0xc6391a80
sched_switch(c6391a80,0,1) at sched_switch+0x14b
mi_switch(1,0,c6391a80,e4fe5c88,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c6560a3c,c6391a80,e4fe5ca8,c06835ed,c6560a3c,...) at sleepq_switch+0x86
sleepq_timedwait(c6560a3c) at sleepq_timedwait+0x36
msleep(c6560a3c,c6560af0,4c,c0902a56,3e8) at msleep+0x215
fdc_worker(c6560a00) at fdc_worker+0x256
fdc_thread(c6560a00,e4fe5d38) at fdc_thread+0x86
fork_exit(c0857064,c6560a00,e4fe5d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fe5d6c, ebp = 0 ---
Tracing command irq7: ppc0 pid 33 tid 100032 td 0xc6391900
fork_trampoline() at fork_trampoline
Tracing command swi0: sio pid 32 tid 100033 td 0xc6391780
sched_switch(c6391780,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c65590f0,e4fdfd38) at ithread_loop+0xb3
fork_exit(c0667d78,c65590f0,e4fdfd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fdfd6c, ebp = 0 ---
Tracing command acpi_thermal pid 31 tid 100034 td 0xc6391600
sched_switch(c6391600,c6533d80,6) at sched_switch+0x14b
mi_switch(6,c6533d80,c6533ed8,c6533d80,e4fdcaac,...) at mi_switch+0x1ba
maybe_preempt(c6533d80) at maybe_preempt+0xc4
sched_add(c6533d80,4,c6391600,c6533d80,c6388380,...) at sched_add+0x27
setrunqueue(c6533d80,4) at setrunqueue+0x63
intr_event_schedule_thread(c6388380) at intr_event_schedule_thread+0xb5
intr_execute_handlers(c63850c4,e4fdcb24,14,e4fdcb8c,c087f853,...) at intr_execute_handlers+0x118
lapic_handle_intr(36) at lapic_handle_intr+0x2e
Xapic_isr1() at Xapic_isr1+0x33
--- interrupt, eip = 0xc0b485ab, esp = 0xe4fdcb68, ebp = 0xe4fdcb8c ---
AcpiNsGetNextNode(c445,c655c400,c649d080,e4fdcbc0,c0b4a55e,...) at AcpiNsGetNextNode+0x3f
AcpiDsTerminateControlMethod(c655c400,0,c649d080,0,c655d800,...) at AcpiDsTerminateControlMethod+0x7b
AcpiPsParseAml(c655d800,c649d780,c63d6380,0,e4fdcc38,...) at AcpiPsParseAml+0x1a6
AcpiPsxExecute(e4fdcc38,e4fdcc38,e4fdcc38,e4fdcc04,c0b47174,...) at AcpiPsxExecute+0x1b9
AcpiNsExecuteControlMethod(e4fdcc38,0,e4fdcc24,c0b472a8,e4fdcc38,...) at AcpiNsExecuteControlMethod+0x3b
AcpiNsEvaluateByHandle(e4fdcc38,c63d3b80,c9d083f0,1001,c63d3ba0,...) at AcpiNsEvaluateByHandle+0x84
AcpiNsEvaluateRelative(c0b6a5bd,e4fdcc38,c6391600,c63d3b80,0,...) at AcpiNsEvaluateRelative+0xa4
AcpiEvaluateObject(c63d3ba0,c0b6a5bd,0,e4fdcc6c,10,...) at AcpiEvaluateObject+0x74
acpi_GetInteger(c63d3ba0,c0b6a5bd,e4fdcca4,0,0,...) at acpi_GetInteger+0x30
acpi_tz_get_temperature(c06726a9,c097a200,10,0,c104e960,...) at acpi_tz_get_temperature+0x19
acpi_tz_monitor(cad2f9b0,d25c20c0,3,c6530c90,c0b59d48,...) at acpi_tz_monitor+0x10
acpi_tz_thread(0,e4fdcd38) at acpi_tz_thread+0x143
fork_exit(c0b59d48,0,e4fdcd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fdcd6c, ebp = 0 ---
Tracing command em0 taskq pid 30 tid 100035 td 0xc6391480
sched_switch(c6391480,0,1) at sched_switch+0x14b
mi_switch(1,0,c6391480,e4fd9cd0,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c651dd00) at sleepq_switch+0x86
sleepq_wait(c651dd00,c069ce91,c651dd00,c651dd1c,c651dd00,...) at sleepq_wait+0x36
msleep_spin(c651dd00,c651dd1c,c0902a56,0) at msleep_spin+0x132
taskqueue_thread_loop(c6487a14,e4fd9d38) at taskqueue_thread_loop+0x85
fork_exit(c069d1f0,c6487a14,e4fd9d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fd9d6c, ebp = 0 ---
Tracing command irq20: amr0 pid 29 tid 100036 td 0xc6533d80
sched_switch(c6533d80,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c651c360,e6c86d38) at ithread_loop+0xb3
fork_exit(c0667d78,c651c360,e6c86d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6c86d6c, ebp = 0 ---
Tracing command aic_recovery1 pid 28 tid 100037 td 0xc6533c00
sched_switch(c6533c00,0,1) at sched_switch+0x14b
mi_switch(1,0,c6533c00,e6c83cd0,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c651b000) at sleepq_switch+0x86
sleepq_wait(c651b000,0,c6533c00,c651b000,c04b6cb0,...) at sleepq_wait+0x36
msleep(c651b000,0,a0,c08d39c8,0) at msleep+0x239
ahc_recovery_thread(c651b000,e6c83d38) at ahc_recovery_thread+0x5b
fork_exit(c04b6cb0,c651b000,e6c83d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6c83d6c, ebp = 0 ---
Tracing command irq17: ahc1 pid 27 tid 100017 td 0xc638c900
sched_switch(c638c900,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c6518950,e4fc4d38) at ithread_loop+0xb3
fork_exit(c0667d78,c6518950,e4fc4d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fc4d6c, ebp = 0 ---
Tracing command aic_recovery1 pid 26 tid 100018 td 0xc638c780
sched_switch(c638c780,0,1) at sched_switch+0x14b
mi_switch(1,0,c638c780,e4fc1cd0,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c651b000) at sleepq_switch+0x86
sleepq_wait(c651b000,0,c638c780,c651b000,c04b6cb0,...) at sleepq_wait+0x36
msleep(c651b000,0,a0,c08d39c8,0) at msleep+0x239
ahc_recovery_thread(c651b000,e4fc1d38) at ahc_recovery_thread+0x5b
fork_exit(c04b6cb0,c651b000,e4fc1d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fc1d6c, ebp = 0 ---
Tracing command aic_recovery0 pid 25 tid 100019 td 0xc638c600
sched_switch(c638c600,0,1) at sched_switch+0x14b
mi_switch(1,0,c638c600,e4fbecd0,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c648a600) at sleepq_switch+0x86
sleepq_wait(c648a600,0,c638c600,c648a600,c04b6cb0,...) at sleepq_wait+0x36
msleep(c648a600,0,a0,c08d39c8,0) at msleep+0x239
ahc_recovery_thread(c648a600,e4fbed38) at ahc_recovery_thread+0x5b
fork_exit(c04b6cb0,c648a600,e4fbed38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fbed6c, ebp = 0 ---
Tracing command irq16: ahc0 pid 24 tid 100020 td 0xc638c480
sched_switch(c638c480,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c651c130,e4fbbd38) at ithread_loop+0xb3
fork_exit(c0667d78,c651c130,e4fbbd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fbbd6c, ebp = 0 ---
Tracing command aic_recovery0 pid 23 tid 100021 td 0xc638c300
sched_switch(c638c300,0,1) at sched_switch+0x14b
mi_switch(1,0,c638c300,e4fb8cd0,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c648a600) at sleepq_switch+0x86
sleepq_wait(c648a600,0,c638c300,c648a600,c04b6cb0,...) at sleepq_wait+0x36
msleep(c648a600,0,a0,c08d39c8,0) at msleep+0x239
ahc_recovery_thread(c648a600,e4fb8d38) at ahc_recovery_thread+0x5b
fork_exit(c04b6cb0,c648a600,e4fb8d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fb8d6c, ebp = 0 ---
Tracing command irq15: ata2 pid 22 tid 100022 td 0xc638c180
fork_trampoline() at fork_trampoline
Tracing command irq14: ata0 pid 21 tid 100023 td 0xc6493780
fork_trampoline() at fork_trampoline
Tracing command irq18: fxp0 pid 20 tid 100024 td 0xc6493600
fork_trampoline() at fork_trampoline
Tracing command irq31: acpi0 pid 19 tid 100025 td 0xc6493480
sched_switch(c6493480,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c6488000,e6a1ad38) at ithread_loop+0xb3
fork_exit(c0667d78,c6488000,e6a1ad38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe6a1ad6c, ebp = 0 ---
Tracing command swi6: Giant taskq pid 18 tid 100026 td 0xc6493300
fork_trampoline() at fork_trampoline
Tracing command swi6: task queue pid 17 tid 100008 td 0xc638b300
sched_switch(c638b300,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c6488610,e4f9ad38) at ithread_loop+0xb3
fork_exit(c0667d78,c6488610,e4f9ad38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4f9ad6c, ebp = 0 ---
Tracing command acpi_task_2 pid 9 tid 100009 td 0xc638b180
sched_switch(c638b180,0,1) at sched_switch+0x14b
mi_switch(1,0,c638b180,e4f97cc8,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c63dfb00) at sleepq_switch+0x86
sleepq_wait(c63dfb00,0,c638b180,c63dfb00,c63dfb1c,...) at sleepq_wait+0x36
msleep(c63dfb00,c63dfb1c,0,c0902a56,0) at msleep+0x239
taskqueue_thread_loop(c0b70860,e4f97d38) at taskqueue_thread_loop+0x9c
fork_exit(c069d1f0,c0b70860,e4f97d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4f97d6c, ebp = 0 ---
Tracing command acpi_task_1 pid 8 tid 100010 td 0xc638b000
sched_switch(c638b000,0,1) at sched_switch+0x14b
mi_switch(1,0,c638b000,e4f94cc8,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c63dfb00) at sleepq_switch+0x86
sleepq_wait(c63dfb00,0,c638b000,c63dfb00,c63dfb1c,...) at sleepq_wait+0x36
msleep(c63dfb00,c63dfb1c,0,c0902a56,0) at msleep+0x239
taskqueue_thread_loop(c0b70860,e4f94d38) at taskqueue_thread_loop+0x9c
fork_exit(c069d1f0,c0b70860,e4f94d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4f94d6c, ebp = 0 ---
Tracing command acpi_task_0 pid 7 tid 100011 td 0xc6391300
sched_switch(c6391300,0,1) at sched_switch+0x14b
mi_switch(1,0,c6391300,e4fd6cc8,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c63dfb00) at sleepq_switch+0x86
sleepq_wait(c63dfb00,0,c6391300,c63dfb00,c63dfb1c,...) at sleepq_wait+0x36
msleep(c63dfb00,c63dfb1c,0,c0902a56,0) at msleep+0x239
taskqueue_thread_loop(c0b70860,e4fd6d38) at taskqueue_thread_loop+0x9c
fork_exit(c069d1f0,c0b70860,e4fd6d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fd6d6c, ebp = 0 ---
Tracing command kqueue taskq pid 6 tid 100012 td 0xc6391180
sched_switch(c6391180,0,1) at sched_switch+0x14b
mi_switch(1,0,c6391180,e4fd3cc8,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c63dfb80) at sleepq_switch+0x86
sleepq_wait(c63dfb80,0,c6391180,c63dfb80,c63dfb9c,...) at sleepq_wait+0x36
msleep(c63dfb80,c63dfb9c,0,c0902a56,0) at msleep+0x239
taskqueue_thread_loop(c09e4e60,e4fd3d38) at taskqueue_thread_loop+0x9c
fork_exit(c069d1f0,c09e4e60,e4fd3d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fd3d6c, ebp = 0 ---
Tracing command swi2: cambio pid 16 tid 100013 td 0xc6391000
sched_switch(c6391000,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c6488650,e4fd0d38) at ithread_loop+0xb3
fork_exit(c0667d78,c6488650,e4fd0d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fd0d6c, ebp = 0 ---
Tracing command swi5: + pid 15 tid 100014 td 0xc638cd80
sched_switch(c638cd80,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c641e380,e4fcdd38) at ithread_loop+0xb3
fork_exit(c0667d78,c641e380,e4fcdd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fcdd6c, ebp = 0 ---
Tracing command thread taskq pid 5 tid 100015 td 0xc638cc00
sched_switch(c638cc00,0,1) at sched_switch+0x14b
mi_switch(1,0,c638cc00,e4fcacc8,c069be42,...) at mi_switch+0x1ba
sleepq_switch(c63dfe00) at sleepq_switch+0x86
sleepq_wait(c63dfe00,0,c638cc00,c63dfe00,c63dfe1c,...) at sleepq_wait+0x36
msleep(c63dfe00,c63dfe1c,0,c0902a56,0) at msleep+0x239
taskqueue_thread_loop(c09ea628,e4fcad38) at taskqueue_thread_loop+0x9c
fork_exit(c069d1f0,c09ea628,e4fcad38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fcad6c, ebp = 0 ---
Tracing command yarrow pid 14 tid 100016 td 0xc638ca80
sched_switch(c638ca80,0,1) at sched_switch+0x14b
mi_switch(1,0,c638ca80,e4fc7cbc,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c09e1aa0,c638ca80,e4fc7cdc,c06835ed,c09e1aa0,...) at sleepq_switch+0x86
sleepq_timedwait(c09e1aa0) at sleepq_timedwait+0x36
msleep(c09e1aa0,0,0,c0902a56,64) at msleep+0x215
random_kthread(0,e4fc7d38) at random_kthread+0x1bb
fork_exit(c05c6bdc,0,e4fc7d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fc7d6c, ebp = 0 ---
Tracing command g_down pid 4 tid 100000 td 0xc638c000
sched_switch(c638c000,0,1) at sched_switch+0x14b
mi_switch(1,0,c638c000,e4fb2cb8,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c09e45c8,c638c000,e4fb2cd8,c06835ed,c09e45c8,...) at sleepq_switch+0x86
sleepq_timedwait(c09e45c8) at sleepq_timedwait+0x36
msleep(c09e45c8,c09e44a8,24c,c0902a56,64) at msleep+0x215
g_io_schedule_down(c638c000) at g_io_schedule_down+0x56
g_down_procbody(0,e4fb2d38) at g_down_procbody+0x6e
fork_exit(c0643bb4,0,e4fb2d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fb2d6c, ebp = 0 ---
Tracing command g_up pid 3 tid 100001 td 0xc638bd80
sched_switch(c638bd80,0,1) at sched_switch+0x14b
mi_switch(1,0,c638bd80,e4fafcbc,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c09e45c4,c638bd80,e4fafcdc,c06835ed,c09e45c4,...) at sleepq_switch+0x86
sleepq_timedwait(c09e45c4) at sleepq_timedwait+0x36
msleep(c09e45c4,c09e44e8,24c,c0902a56,64) at msleep+0x215
g_io_schedule_up(c638bd80) at g_io_schedule_up+0xcc
g_up_procbody(0,e4fafd38) at g_up_procbody+0x6e
fork_exit(c0643b40,0,e4fafd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fafd6c, ebp = 0 ---
Tracing command g_event pid 2 tid 100002 td 0xc638bc00
sched_switch(c638bc00,0,1) at sched_switch+0x14b
mi_switch(1,0,c638bc00,e4faccc4,c069bf12,...) at mi_switch+0x1ba
sleepq_switch(c09e45bc,c638bc00,e4facce4,c06835ed,c09e45bc,...) at sleepq_switch+0x86
sleepq_timedwait(c09e45bc) at sleepq_timedwait+0x36
msleep(c09e45bc,0,4c,c0902a56,64) at msleep+0x215
g_event_procbody(0,e4facd38) at g_event_procbody+0xa6
fork_exit(c0643c28,0,e4facd38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4facd6c, ebp = 0 ---
Tracing command swi3: vm pid 13 tid 100003 td 0xc638ba80
fork_trampoline() at fork_trampoline
Tracing command swi4: clock sio pid 12 tid 100004 td 0xc638b900
sched_switch(c638b900,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
turnstile_wait(c09e6800,cfb11d80) at turnstile_wait+0x280
_mtx_lock_sleep(c09e6800,c638b900,0,0,0) at _mtx_lock_sleep+0xb4
softclock(0) at softclock+0x1e7
ithread_execute_handlers(c638a860,c6389e00) at ithread_execute_handlers+0x121
ithread_loop(c636e640,e4fa6d38) at ithread_loop+0x54
fork_exit(c0667d78,c636e640,e4fa6d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fa6d6c, ebp = 0 ---
Tracing command swi1: net pid 11 tid 100005 td 0xc638b780
sched_switch(c638b780,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
ithread_loop(c636e650,e4fa3d38) at ithread_loop+0xb3
fork_exit(c0667d78,c636e650,e4fa3d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fa3d6c, ebp = 0 ---
Tracing command idle pid 10 tid 100006 td 0xc638b600
sched_switch(c638b600,0,2) at sched_switch+0x14b
mi_switch(2,0) at mi_switch+0x1ba
critical_exit(0,c638b600,0,c646e818,c646e800,...) at critical_exit+0x78
intr_execute_handlers(c63850e8,e4fa0c98,15,e4fa0cdc,c087f853,...) at intr_execute_handlers+0x10c
lapic_handle_intr(37) at lapic_handle_intr+0x2e
Xapic_isr1() at Xapic_isr1+0x33
--- interrupt, eip = 0xc0b62135, esp = 0xe4fa0cdc, ebp = 0xe4fa0cdc ---
acpi_cpu_c1(0,c638b600,3e8,c638ac90,c0666e88,...) at acpi_cpu_c1+0x5
acpi_cpu_idle(e4fa0d10,c0666e99,c638ac90,e4fa0d24,c0666c48,...) at acpi_cpu_idle+0x152
cpu_idle(c638ac90,e4fa0d24,c0666c48,0,e4fa0d38,...) at cpu_idle+0x1f
idle_proc(0,e4fa0d38) at idle_proc+0x11
fork_exit(c0666e88,0,e4fa0d38) at fork_exit+0x70
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xe4fa0d6c, ebp = 0 ---
Tracing command init pid 1 tid 100007 td 0xc638b480
sched_switch(c638b480,0,1) at sched_switch+0x14b
mi_switch(1,0,c638f000,e4f9dc18,c069be8d,...) at mi_switch+0x1ba
sleepq_switch(c638f000) at sleepq_switch+0x86
sleepq_wait_sig(c638f000) at sleepq_wait_sig+0x1d
msleep(c638f000,c638f068,15c,c0907390,0) at msleep+0x22a
kern_wait(c638b480,ffffffff,e4f9dc84,0,0) at kern_wait+0x89c
wait4(c638b480,e4f9dd04) at wait4+0x2a
syscall(3b,3b,3b,0,0,...) at syscall+0x2b7
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x8052f47, esp = 0xbfbfed7c, ebp = 0xbfbfed98 ---
Tracing command swapper pid 0 tid 0 td 0xc09e48e0
sched_switch(c09e48e0,0,1) at sched_switch+0x14b
mi_switch(1,0) at mi_switch+0x1ba
scheduler(0,c1ec00,c1e000,0,c044f085,...) at scheduler+0x1e1
mi_startup() at mi_startup+0x96
begin() at begin+0x2c
db>
--Apple-Mail-23--291249582
Content-Transfer-Encoding: 7bit
Content-Type: application/octet-stream;
x-unix-mode=0644;
name=KERNCONF
Content-Disposition: attachment;
filename=KERNCONF
include GENERIC
ident PT
options KDB # Enable kernel debugger support.
options BREAK_TO_DEBUGGER
options DDB # Support DDB.
options GDB # Support remote GDB.
options QUOTA
--Apple-Mail-23--291249582
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=US-ASCII;
format=flowed
--Apple-Mail-23--291249582--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?FC17EAA4-B5B1-48E4-BA69-1D8EF4E0F047>