From owner-freebsd-stable@FreeBSD.ORG Mon Jun 29 12:55:36 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9010E1065675 for ; Mon, 29 Jun 2009 12:55:36 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id 3FC448FC0C for ; Mon, 29 Jun 2009 12:55:35 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: by bwz12 with SMTP id 12so313737bwz.43 for ; Mon, 29 Jun 2009 05:55:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=HcZzeufZ5Oq+1kbfPMKnSslDO3zxLXlCCSYTBFo10KM=; b=s7Zggnvf+cO2q2twWdlViIG4NmLESjqfLD7bNpFcXXWTQX5cxh3Zk/mRk9Q5HT87rz 30iVIFZ1Mg9RpqbW07vBprN79YncmYzsZ5QQuZGZgv+VdSbrlnlgxURBffwEldNV/wVh OqDJeqRT42NR0hOgcjXf9GYO+c2F6/aPj16dI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Xo3FoZeI5UZb6a+AHhLr6V1Xxy03Zy+53IZjJxhZpWYt7IttLynJS5qCGy5z7T5DtL Mzf0gRLywuRQaiwiFWNRDv78Cqi7Rye0SLVK0rqfgJOVgAQe3QrytagGQp8dAfrYX3Ea JIVkHMCGn8nfgv16S3vXS6FNRLl+8dw4EKREY= MIME-Version: 1.0 Received: by 10.204.117.16 with SMTP id o16mr7137962bkq.100.1246280134138; Mon, 29 Jun 2009 05:55:34 -0700 (PDT) In-Reply-To: <3bbf2fe10906290458v3d57441ar44c4ed8f36c957f@mail.gmail.com> References: <3bbf2fe10906290458v3d57441ar44c4ed8f36c957f@mail.gmail.com> Date: Mon, 29 Jun 2009 16:55:34 +0400 Message-ID: From: pluknet To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable Subject: Re: [nfs] process locks in "bo_wwait" on 6.4 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2009 12:55:37 -0000 2009/6/29 Attilio Rao : > 2009/6/29 pluknet : >> 2009/6/26 pluknet : >>> 2009/6/26 pluknet : >>>> Hello. >>>> >>>> While building a module on nfs mounted /usr/src >>>> I got an unkillable process waiting forever in bo_wwait. >>> >>> Small note: iface on NFS server has mtu changed from 1500 to 1450. >>> Can this be a source of the problem? >> >> This is 100% reproducible. Lock in the same place. Any hints? > > Can you also show the value of ps? > A precise map of what processes are doing would give an help. > Also would be useful to printout traces for other threads and not only > the stucked one. > >From another run: db> ps pid ppid pgrp uid state wmesg wchan cmd 1228 1205 1205 0 S+ bo_wwait 0xc9887c10 awk 1205 1196 1205 0 S+ wait 0xc893ea78 make 1202 0 0 0 SL nfsreq 0xc9637a00 [nfsiod 0] 1196 1125 1196 0 Ss+ wait 0xc8942648 bash 1194 1 1194 0 Ss+ ttyin 0xc82e9010 getty 1193 1 1193 0 Ss+ ttyin 0xc82d7c10 getty 1192 1 1192 0 Ss+ ttyin 0xc82f7010 getty 1191 1 1191 0 Ss+ ttyin 0xc82f7410 getty 1190 1 1190 0 Ss+ ttyin 0xc82f2410 getty 1189 1 1189 0 Ss+ ttyin 0xc82e9810 getty 1188 1 1188 0 Ss+ ttyin 0xc82e7410 getty 1187 1 1187 0 Ss+ ttyin 0xc82d7410 getty 1186 1 1186 0 Ss+ ttyin 0xc82f0410 getty 1185 1 1185 0 Ss+ ttyin 0xc82f1810 getty 1171 1 1171 0 Ss select 0xc0a8d044 inetd 1134 1 1134 0 Ss nanslp 0xc0a3b4ec cron 1125 1064 1125 0 Ss select 0xc0a8d044 sshd 1064 1 1064 0 Ss select 0xc0a8d044 sshd 901 1 901 0 Ss select 0xc0a8d044 ntpd 796 1 796 0 Ss select 0xc0a8d044 syslogd 767 0 0 0 SL - 0xc0a38b40 [accounting] 734 1 734 0 Ss select 0xc0a8d044 devd 50 0 0 0 SL sdflush 0xc0a9ae74 [softdepflush] 49 0 0 0 SL syncer 0xc0a3b25c [syncer] 48 0 0 0 SL vlruwt 0xc828ba78 [vnlru] 47 0 0 0 SL psleep 0xc0a8d5c0 [bufdaemon] 46 0 0 0 SL pgzero 0xc0a9bea4 [pagezero] 45 0 0 0 SL psleep 0xc0a9b9b4 [vmdaemon] 44 0 0 0 SL psleep 0xc0a9b968 [pagedaemon] 43 0 0 0 WL [irq1: atkbd0] 42 0 0 0 WL [swi0: sio] 41 0 0 0 WL [irq15: ata1] 40 0 0 0 WL [irq14: ata0] 39 0 0 0 SL usbevt 0xc81eb210 [usb4] 38 0 0 0 SL usbevt 0xc82c4210 [usb3] 37 0 0 0 SL usbevt 0xc82ab210 [usb2] 36 0 0 0 SL usbevt 0xc82af210 [usb1] 35 0 0 0 WL [irq22: uhci1 uhci3] 34 0 0 0 SL usbtsk 0xc0a37ba4 [usbtask] 33 0 0 0 SL usbevt 0xc8278210 [usb0] 32 0 0 0 WL [irq23: uhci0 uhci+] 31 0 0 0 WL [irq257: bce1] 30 0 0 0 WL [irq256: bce0] 29 0 0 0 SL aifthd 0xc828b218 [aac0aif] 28 0 0 0 WL [irq17: aac0] 27 0 0 0 WL [irq9: acpi0] 26 0 0 0 WL [swi5: +] 25 0 0 0 SL - 0xc8150100 [thread taskq] 24 0 0 0 WL [swi6: Giant taskq] 9 0 0 0 SL - 0xc8150280 [acpi_task_2] 8 0 0 0 SL - 0xc8150280 [acpi_task_1] 7 0 0 0 SL - 0xc8150280 [acpi_task_0] 23 0 0 0 WL [swi6: task queue] 6 0 0 0 SL - 0xc8150400 [kqueue taskq] 22 0 0 0 WL [swi2: cambio] 5 0 0 0 SL ccb_scan 0xc0a1e204 [xpt_thrd] 21 0 0 0 SL - 0xc0a358c0 [yarrow] 4 0 0 0 SL - 0xc0a38488 [g_down] 3 0 0 0 SL - 0xc0a38484 [g_up] 2 0 0 0 SL - 0xc0a3847c [g_event] 20 0 0 0 WL [swi1: net] 19 0 0 0 WL [swi3: vm] 18 0 0 0 WL [swi4: clock sio] 17 0 0 0 RL CPU 0 [idle: cpu0] 16 0 0 0 RL CPU 1 [idle: cpu1] 15 0 0 0 RL CPU 2 [idle: cpu2] 14 0 0 0 RL CPU 3 [idle: cpu3] 13 0 0 0 RL CPU 4 [idle: cpu4] 12 0 0 0 RL CPU 5 [idle: cpu5] 11 0 0 0 RL CPU 6 [idle: cpu6] 10 0 0 0 RL CPU 7 [idle: cpu7] 1 0 1 0 SLs wait 0xc80fc000 [init] 0 0 0 0 WLs [swapper] db> alltrace Tracing command awk pid 1228 tid 100108 td 0xc88ccd00 sched_switch(c88ccd00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c9887c10,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c9887c10,0,c095cdf5,aed,c0951c9f,...) at sleepq_wait+0x46 msleep(c9887c10,c9887ba8,4d,c094fe6b,0) at msleep+0x27d bufobj_wwait(c9887bec,0,0,c9887ba8,9,...) at bufobj_wwait+0x5f nfs_flush(c9887ae0,1,c88ccd00,0,2,...) at nfs_flush+0x790 nfs_close(eed4ab68) at nfs_close+0xb5 VOP_CLOSE_APV(c09dbec0,eed4ab68) at VOP_CLOSE_APV+0x9b vn_close(c9887ae0,2,c9628700,c88ccd00) at vn_close+0x8c vn_closefile(c88993f0,c88ccd00) at vn_closefile+0xca fdrop_locked(c88993f0,c88ccd00,c80cd400,0,c094365f,...) at fdrop_locked+0xb0 fdrop(c88993f0,c88ccd00,6b0,c0a468a0,0,...) at fdrop+0x24 closef(c88993f0,c88ccd00,0,c88ccd00,0,...) at closef+0x367 kern_close(c88ccd00,3,eed4ad30,c08bc8db,c88ccd00,...) at kern_close+0x1aa close(c88ccd00,eed4ad04) at close+0x10 syscall(3b,3b,3b,0,28190a20,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (6, FreeBSD ELF32, close), eip = 0x2816c1e7, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 --- Tracing command make pid 1205 tid 100117 td 0xc82dad00 sched_switch(c82dad00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c893ea78) at sleepq_switch+0xc1 sleepq_wait_sig(c893ea78) at sleepq_wait_sig+0x1d msleep(c893ea78,c893eae0,15c,c094c5d4,0) at msleep+0x26e kern_wait(c82dad00,ffffffff,eca39c84,0,0) at kern_wait+0x6bb wait4(c82dad00,eca39d04) at wait4+0x2a syscall(3b,3b,3b,bfbfde38,bfbfde90,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (7, FreeBSD ELF32, wait4), eip = 0x8065d03, esp = 0xbfbfddfc, ebp = 0xbfbfde18 --- Tracing command nfsiod 0 pid 1202 tid 100101 td 0xc88d39c0 sched_switch(c88d39c0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c9637a00,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c9637a00,0,c095c856,2ba,c095c856,...) at sleepq_wait+0x46 msleep(c9637a00,c0a95120,53,c095c8cb,0) at msleep+0x27d nfs_reply(c9637a00,0,2a0,0,c847fd00,...) at nfs_reply+0x1ec nfs_request(c9887ae0,c8924400,7,0,c9628700,...) at nfs_request+0x371 nfs_writerpc(c9887ae0,eed5fcb4,c9628700,eed5fca4,eed5fca8) at nfs_writerpc+0x242 nfs_doio(c9887ae0,dc234984,c9628700,0) at nfs_doio+0x3f4 nfssvc_iod(c0a95180,eed5fd38,c0a95180,c0790a54,0,...) at nfssvc_iod+0x194 fork_exit(c0790a54,c0a95180,eed5fd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xeed5fd6c, ebp = 0 --- Tracing command bash pid 1196 tid 100122 td 0xc8943820 sched_switch(c8943820,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c8942648) at sleepq_switch+0xc1 sleepq_wait_sig(c8942648) at sleepq_wait_sig+0x1d msleep(c8942648,c89426b0,15c,c094c5d4,0) at msleep+0x26e kern_wait(c8943820,ffffffff,eedacc84,6,0) at kern_wait+0x6bb wait4(c8943820,eedacd04) at wait4+0x2a syscall(3b,3b,3b,4b5,0,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (7, FreeBSD ELF32, wait4), eip = 0x2828cc37, esp = 0xbfbfe8bc, ebp = 0xbfbfe8d8 --- Tracing command getty pid 1194 tid 100171 td 0xc8e4d340 sched_switch(c8e4d340,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82e9010) at sleepq_switch+0xc1 sleepq_wait_sig(c82e9010) at sleepq_wait_sig+0x1d msleep(c82e9010,0,159,c094db27,0) at msleep+0x26e ttysleep(c82e9000,c82e9010,159,c094db27,0,...) at ttysleep+0x21 ttread(c82e9000,eee5bcbc,0) at ttread+0x48f ttyread(c82ebe00,eee5bcbc,0,c0a3ac00,0,...) at ttyread+0x2f giant_read(c82ebe00,eee5bcbc,0) at giant_read+0x3f devfs_read_f(c8a34750,eee5bcbc,c80f4700,0,c8e4d340) at devfs_read_f+0x62 dofileread(c8e4d340,0,c8a34750,eee5bcbc,ffffffff,...) at dofileread+0x85 kern_readv(c8e4d340,0,eee5bcbc,bfbfee0f,1,...) at kern_readv+0x36 read(c8e4d340,eee5bd04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedec, ebp = 0xbfbfee18 --- Tracing command getty pid 1193 tid 100172 td 0xc8e4d1a0 sched_switch(c8e4d1a0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82d7c10) at sleepq_switch+0xc1 sleepq_wait_sig(c82d7c10) at sleepq_wait_sig+0x1d msleep(c82d7c10,0,159,c094db27,0) at msleep+0x26e ttysleep(c82d7c00,c82d7c10,159,c094db27,0,...) at ttysleep+0x21 ttread(c82d7c00,eee55cbc,0) at ttread+0x48f ttyread(c82a3500,eee55cbc,0,c0a3ac00,0,...) at ttyread+0x2f giant_read(c82a3500,eee55cbc,0) at giant_read+0x3f devfs_read_f(c8899dc8,eee55cbc,c80f4700,0,c8e4d1a0) at devfs_read_f+0x62 dofileread(c8e4d1a0,0,c8899dc8,eee55cbc,ffffffff,...) at dofileread+0x85 kern_readv(c8e4d1a0,0,eee55cbc,bfbfee1f,1,...) at kern_readv+0x36 read(c8e4d1a0,eee55d04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command getty pid 1192 tid 100173 td 0xc8e4ed00 sched_switch(c8e4ed00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82f7010) at sleepq_switch+0xc1 sleepq_wait_sig(c82f7010) at sleepq_wait_sig+0x1d msleep(c82f7010,0,159,c094db27,0) at msleep+0x26e ttysleep(c82f7000,c82f7010,159,c094db27,0,...) at ttysleep+0x21 ttread(c82f7000,eee9acbc,0) at ttread+0x48f ttyread(c82eb600,eee9acbc,0,eee9ac14,c067a05b,...) at ttyread+0x2f scread(c82eb600,eee9acbc,0,c0a3ac00,0,...) at scread+0x22 giant_read(c82eb600,eee9acbc,0) at giant_read+0x3f devfs_read_f(c8d95828,eee9acbc,c80f4700,0,c8e4ed00) at devfs_read_f+0x62 dofileread(c8e4ed00,0,c8d95828,eee9acbc,ffffffff,...) at dofileread+0x85 kern_readv(c8e4ed00,0,eee9acbc,bfbfee1f,1,...) at kern_readv+0x36 read(c8e4ed00,eee9ad04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command getty pid 1191 tid 100174 td 0xc8e4eb60 sched_switch(c8e4eb60,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82f7410) at sleepq_switch+0xc1 sleepq_wait_sig(c82f7410) at sleepq_wait_sig+0x1d msleep(c82f7410,0,159,c094db27,0) at msleep+0x26e ttysleep(c82f7400,c82f7410,159,c094db27,0,...) at ttysleep+0x21 ttread(c82f7400,eee97cbc,0) at ttread+0x48f ttyread(c82eb500,eee97cbc,0,eee97c14,c067a05b,...) at ttyread+0x2f scread(c82eb500,eee97cbc,0,c0a3ac00,0,...) at scread+0x22 giant_read(c82eb500,eee97cbc,0) at giant_read+0x3f devfs_read_f(c960f9d8,eee97cbc,c80f4700,0,c8e4eb60) at devfs_read_f+0x62 dofileread(c8e4eb60,0,c960f9d8,eee97cbc,ffffffff,...) at dofileread+0x85 kern_readv(c8e4eb60,0,eee97cbc,bfbfee1f,1,...) at kern_readv+0x36 read(c8e4eb60,eee97d04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command getty pid 1190 tid 100094 td 0xc88a4d00 sched_switch(c88a4d00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82f2410) at sleepq_switch+0xc1 sleepq_wait_sig(c82f2410) at sleepq_wait_sig+0x1d msleep(c82f2410,0,159,c094db27,0) at msleep+0x26e ttysleep(c82f2400,c82f2410,159,c094db27,0,...) at ttysleep+0x21 ttread(c82f2400,eed22cbc,0) at ttread+0x48f ttyread(c82eb400,eed22cbc,0,eed22c14,c067a05b,...) at ttyread+0x2f scread(c82eb400,eed22cbc,0,c0a3ac00,0,...) at scread+0x22 giant_read(c82eb400,eed22cbc,0) at giant_read+0x3f devfs_read_f(c8899cf0,eed22cbc,c80f4700,0,c88a4d00) at devfs_read_f+0x62 dofileread(c88a4d00,0,c8899cf0,eed22cbc,ffffffff,...) at dofileread+0x85 kern_readv(c88a4d00,0,eed22cbc,bfbfee1f,1,...) at kern_readv+0x36 read(c88a4d00,eed22d04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command getty pid 1189 tid 100129 td 0xc88d4b60 sched_switch(c88d4b60,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82e9810) at sleepq_switch+0xc1 sleepq_wait_sig(c82e9810) at sleepq_wait_sig+0x1d msleep(c82e9810,0,159,c094db27,0) at msleep+0x26e ttysleep(c82e9800,c82e9810,159,c094db27,0,...) at ttysleep+0x21 ttread(c82e9800,eed7dcbc,0) at ttread+0x48f ttyread(c82eb300,eed7dcbc,0,eed7dc14,c067a05b,...) at ttyread+0x2f scread(c82eb300,eed7dcbc,0,c0a3ac00,0,...) at scread+0x22 giant_read(c82eb300,eed7dcbc,0) at giant_read+0x3f devfs_read_f(c860f828,eed7dcbc,c80f4700,0,c88d4b60) at devfs_read_f+0x62 dofileread(c88d4b60,0,c860f828,eed7dcbc,ffffffff,...) at dofileread+0x85 kern_readv(c88d4b60,0,eed7dcbc,bfbfee1f,1,...) at kern_readv+0x36 read(c88d4b60,eed7dd04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command getty pid 1188 tid 100052 td 0xc828ad00 sched_switch(c828ad00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82e7410) at sleepq_switch+0xc1 sleepq_wait_sig(c82e7410) at sleepq_wait_sig+0x1d msleep(c82e7410,0,159,c094db27,0) at msleep+0x26e ttysleep(c82e7400,c82e7410,159,c094db27,0,...) at ttysleep+0x21 ttread(c82e7400,e8a08cbc,0) at ttread+0x48f ttyread(c82eb200,e8a08cbc,0,e8a08c14,c067a05b,...) at ttyread+0x2f scread(c82eb200,e8a08cbc,0,c0a3ac00,0,...) at scread+0x22 giant_read(c82eb200,e8a08cbc,0) at giant_read+0x3f devfs_read_f(c960f900,e8a08cbc,c80f4700,0,c828ad00) at devfs_read_f+0x62 dofileread(c828ad00,0,c960f900,e8a08cbc,ffffffff,...) at dofileread+0x85 kern_readv(c828ad00,0,e8a08cbc,bfbfee1f,1,...) at kern_readv+0x36 read(c828ad00,e8a08d04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command getty pid 1187 tid 100133 td 0xc88d44e0 sched_switch(c88d44e0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82d7410) at sleepq_switch+0xc1 sleepq_wait_sig(c82d7410) at sleepq_wait_sig+0x1d msleep(c82d7410,0,159,c094db27,0) at msleep+0x26e ttysleep(c82d7400,c82d7410,159,c094db27,0,...) at ttysleep+0x21 ttread(c82d7400,eed71cbc,0) at ttread+0x48f ttyread(c82eb100,eed71cbc,0,eed71c14,c067a05b,...) at ttyread+0x2f scread(c82eb100,eed71cbc,0,c0a3ac00,0,...) at scread+0x22 giant_read(c82eb100,eed71cbc,0) at giant_read+0x3f devfs_read_f(c8a34870,eed71cbc,c80f4700,0,c88d44e0) at devfs_read_f+0x62 dofileread(c88d44e0,0,c8a34870,eed71cbc,ffffffff,...) at dofileread+0x85 kern_readv(c88d44e0,0,eed71cbc,bfbfee1f,1,...) at kern_readv+0x36 read(c88d44e0,eed71d04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command getty pid 1186 tid 100103 td 0xc88d3680 sched_switch(c88d3680,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82f0410) at sleepq_switch+0xc1 sleepq_wait_sig(c82f0410) at sleepq_wait_sig+0x1d msleep(c82f0410,0,159,c094db27,0) at msleep+0x26e ttysleep(c82f0400,c82f0410,159,c094db27,0,...) at ttysleep+0x21 ttread(c82f0400,eed59cbc,0) at ttread+0x48f ttyread(c82eb000,eed59cbc,0,eed59c14,c067a05b,...) at ttyread+0x2f scread(c82eb000,eed59cbc,0,c0a3ac00,0,...) at scread+0x22 giant_read(c82eb000,eed59cbc,0) at giant_read+0x3f devfs_read_f(c8521828,eed59cbc,c80f4700,0,c88d3680) at devfs_read_f+0x62 dofileread(c88d3680,0,c8521828,eed59cbc,ffffffff,...) at dofileread+0x85 kern_readv(c88d3680,0,eed59cbc,bfbfee1f,1,...) at kern_readv+0x36 read(c88d3680,eed59d04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command getty pid 1185 tid 100102 td 0xc88d3820 sched_switch(c88d3820,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82f1810) at sleepq_switch+0xc1 sleepq_wait_sig(c82f1810) at sleepq_wait_sig+0x1d msleep(c82f1810,0,159,c094db27,0) at msleep+0x26e ttysleep(c82f1800,c82f1810,159,c094db27,0,...) at ttysleep+0x21 ttread(c82f1800,eed5ccbc,0) at ttread+0x48f ttyread(c82eae00,eed5ccbc,0,eed5cc14,c067a05b,...) at ttyread+0x2f scread(c82eae00,eed5ccbc,0,c0a3ac00,0,...) at scread+0x22 giant_read(c82eae00,eed5ccbc,0) at giant_read+0x3f devfs_read_f(c85217e0,eed5ccbc,c80f4700,0,c88d3820) at devfs_read_f+0x62 dofileread(c88d3820,0,c85217e0,eed5ccbc,ffffffff,...) at dofileread+0x85 kern_readv(c88d3820,0,eed5ccbc,bfbfee1f,1,...) at kern_readv+0x36 read(c88d3820,eed5cd04) at read+0x45 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (3, FreeBSD ELF32, read), eip = 0x28149227, esp = 0xbfbfedfc, ebp = 0xbfbfee28 --- Tracing command inetd pid 1171 tid 100136 td 0xc8d86820 sched_switch(c8d86820,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a8d044) at sleepq_switch+0xc1 sleepq_wait_sig(c0a8d044,246,c0a3abc0,c094c027,303,...) at sleepq_wait_sig+0x1d cv_wait_sig(c0a8d044,c0a8d020) at cv_wait_sig+0x18b kern_select(c8d86820,7,bfbfe890,0,0,...) at kern_select+0x4bf select(c8d86820,eede2d04) at select+0x44 syscall(3b,3b,3b,bfbfe890,bfbfee20,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (93, FreeBSD ELF32, select), eip = 0x2815b1a7, esp = 0xbfbfdb6c, ebp = 0xbfbfee58 --- Tracing command cron pid 1134 tid 100128 td 0xc88d4d00 sched_switch(c88d4d00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a3b4ec) at sleepq_switch+0xc1 sleepq_timedwait_sig(c0a3b4ec) at sleepq_timedwait_sig+0x1e msleep(c0a3b4ec,0,15c,c0948631,e291,...) at msleep+0x244 kern_nanosleep(c88d4d00,eed80ccc,eed80cc4) at kern_nanosleep+0xab nanosleep(c88d4d00,eed80d04) at nanosleep+0x4f syscall(3b,3b,3b,805008c,3a,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28132d0f, esp = 0xbfbfeccc, ebp = 0xbfbfecf8 -- - Tracing command sshd pid 1125 tid 100098 td 0xc88d4000 sched_switch(c88d4000,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a8d044) at sleepq_switch+0xc1 sleepq_wait_sig(c0a8d044,246,c0a3abc0,c094c027,303,...) at sleepq_wait_sig+0x1d cv_wait_sig(c0a8d044,c0a8d020) at cv_wait_sig+0x18b kern_select(c88d4000,8,807bbe0,807bbf0,0,...) at kern_select+0x4bf select(c88d4000,eed68d04) at select+0x44 syscall(3b,3b,bfbf003b,0,bfbfe10c,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (93, FreeBSD ELF32, select), eip = 0x283531a7, esp = 0xbfbfe09c, ebp = 0xbfbfe0e8 --- Tracing command sshd pid 1064 tid 100069 td 0xc86114e0 sched_switch(c86114e0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a8d044) at sleepq_switch+0xc1 sleepq_wait_sig(c0a8d044,246,c0a3abc0,c094c027,303,...) at sleepq_wait_sig+0x1d cv_wait_sig(c0a8d044,c0a8d020) at cv_wait_sig+0x18b kern_select(c86114e0,8,807b930,0,0,...) at kern_select+0x4bf select(c86114e0,eec9ad04) at select+0x44 syscall(3b,3b,bfbf003b,1,64,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (93, FreeBSD ELF32, select), eip = 0x283531a7, esp = 0xbfbfe15c, ebp = 0xbfbfee68 --- Tracing command ntpd pid 901 tid 100116 td 0xc8519000 sched_switch(c8519000,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a8d044) at sleepq_switch+0xc1 sleepq_wait_sig(c0a8d044,246,c0a3abc0,c094c027,303,...) at sleepq_wait_sig+0x1d cv_wait_sig(c0a8d044,c0a8d020) at cv_wait_sig+0x18b kern_select(c8519000,1f,bfbfed50,0,0,...) at kern_select+0x4bf select(c8519000,eec4ed04) at select+0x44 syscall(3b,3b,3b,0,bfbfed50,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (93, FreeBSD ELF32, select), eip = 0x282971a7, esp = 0xbfbfed0c, ebp = 0xbfbfede8 --- Tracing command syslogd pid 796 tid 100118 td 0xc82dab60 sched_switch(c82dab60,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a8d044) at sleepq_switch+0xc1 sleepq_wait_sig(c0a8d044,246,c0a3abc0,c094c027,303,...) at sleepq_wait_sig+0x1d cv_wait_sig(c0a8d044,c0a8d020) at cv_wait_sig+0x18b kern_select(c82dab60,9,8059190,0,0,...) at kern_select+0x4bf select(c82dab60,eca36d04) at select+0x44 syscall(3b,3b,3b,8059194,8059190,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (93, FreeBSD ELF32, select), eip = 0x2814d1a7, esp = 0xbfbfda1c, ebp = 0xbfbfee68 --- Tracing command accounting pid 767 tid 100056 td 0xc851d340 sched_switch(c851d340,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a38b40,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a38b40) at sleepq_timedwait+0x4a msleep(c0a38b40,0,7f,c094005e,3a98,c0a38b00,c09428ab,213) at msleep+0x259 acct_thread(0,eec6fd38,0,c067756c,0,...) at acct_thread+0xd5 fork_exit(c067756c,0,eec6fd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xeec6fd6c, ebp = 0 --- Tracing command devd pid 734 tid 100086 td 0xc8615d00 sched_switch(c8615d00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a8d044) at sleepq_switch+0xc1 sleepq_wait_sig(c0a8d044,246,c0a3abc0,c094c027,303,...) at sleepq_wait_sig+0x1d cv_wait_sig(c0a8d044,c0a8d020) at cv_wait_sig+0x18b kern_select(c8615d00,5,bfbfe9d0,0,0,...) at kern_select+0x4bf select(c8615d00,eecc4d04) at select+0x44 syscall(3b,3b,3b,1,8,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (93, FreeBSD ELF32, select), eip = 0x8075acb, esp = 0xbfbfe96c, ebp = 0xbfbfee78 --- Tracing command softdepflush pid 50 tid 100038 td 0xc8287680 sched_switch(c8287680,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a9ae74,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a9ae74) at sleepq_timedwait+0x4a msleep(c0a9ae74,c0a9ae40,44,c0961c53,3e8) at msleep+0x259 softdep_flush(0,e89e1d38,0,c07eec04,0,...) at softdep_flush+0x251 fork_exit(c07eec04,0,e89e1d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89e1d6c, ebp = 0 --- Tracing command syncer pid 49 tid 100039 td 0xc82874e0 sched_switch(c82874e0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a3b25c,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c0a3b25c,0,c06987ee,c0a8da20,c095c856,...) at sleepq_wait+0x46 msleep(c0a3b25c,0,68,c0951419,0) at msleep+0x27d sched_sync(0,e89ded38,0,c06f9860,0,...) at sched_sync+0x309 fork_exit(c06f9860,0,e89ded38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89ded6c, ebp = 0 --- Tracing command vnlru pid 48 tid 100040 td 0xc8287340 sched_switch(c8287340,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c828ba78,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c828ba78) at sleepq_timedwait+0x4a msleep(c828ba78,c0a8d9e0,250,c095110b,3e8,c0a8da60) at msleep+0x259 vnlru_proc(0,e89dbd38,0,c06f7d4c,0,...) at vnlru_proc+0xdf fork_exit(c06f7d4c,0,e89dbd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89dbd6c, ebp = 0 --- Tracing command bufdaemon pid 47 tid 100041 td 0xc82871a0 sched_switch(c82871a0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a8d5c0,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a8d5c0) at sleepq_timedwait+0x4a msleep(c0a8d5c0,c0a8d5e0,44,c094f8fb,3e8) at msleep+0x259 buf_daemon(0,e89d8d38,0,c06eb238,0,...) at buf_daemon+0x108 fork_exit(c06eb238,0,e89d8d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89d8d6c, ebp = 0 --- Tracing command pagezero pid 46 tid 100042 td 0xc8287000 sched_switch(c8287000,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a9bea4,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a9bea4) at sleepq_timedwait+0x4a msleep(c0a9bea4,c0a9b920,200,c0967069,493e0,c0a9b920,0,c096704b,9d) at msleep+0x259 vm_pagezero(0,e89d5d38,0,c08223a4,0,...) at vm_pagezero+0x6c fork_exit(c08223a4,0,e89d5d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89d5d6c, ebp = 0 --- Tracing command vmdaemon pid 45 tid 100043 td 0xc81f2d00 sched_switch(c81f2d00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a9b9b4,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c0a9b9b4,0,c0966cec,659,0,...) at sleepq_wait+0x46 msleep(c0a9b9b4,c0a9b9c0,68,c094f8fb,0,...) at msleep+0x27d vm_daemon(0,e87cad38,0,c08213b8,0,...) at vm_daemon+0x36 fork_exit(c08213b8,0,e87cad38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe87cad6c, ebp = 0 --- Tracing command pagedaemon pid 44 tid 100044 td 0xc81f2b60 sched_switch(c81f2b60,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a9b968,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a9b968) at sleepq_timedwait+0x4a msleep(c0a9b968,c0a9b920,44,c094f8fb,1388) at msleep+0x259 vm_pageout(0,e87c7d38,0,c0821040,0,...) at vm_pageout+0x260 fork_exit(c0821040,0,e87c7d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe87c7d6c, ebp = 0 --- Tracing command irq1: atkbd0 pid 43 tid 100045 td 0xc81f29c0 fork_trampoline() at fork_trampoline Tracing command swi0: sio pid 42 tid 100046 td 0xc82da820 sched_switch(c82da820,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 ithread_loop(c82e3160,eca30d38,c82e3160,c068bd80,0,...) at ithread_loop+0xda fork_exit(c068bd80,c82e3160,eca30d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xeca30d6c, ebp = 0 --- Tracing command irq15: ata1 pid 41 tid 100047 td 0xc82da680 sched_switch(c82da680,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 ithread_loop(c82e3190,eca2dd38,c82e3190,c068bd80,0,...) at ithread_loop+0xda fork_exit(c068bd80,c82e3190,eca2dd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xeca2dd6c, ebp = 0 --- Tracing command irq14: ata0 pid 40 tid 100048 td 0xc82da4e0 fork_trampoline() at fork_trampoline Tracing command usb4 pid 39 tid 100049 td 0xc82da340 sched_switch(c82da340,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c81eb210,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c81eb210) at sleepq_timedwait+0x4a msleep(c81eb210,0,5c,c093cc30,ea60,c82aa980) at msleep+0x259 usb_event_thread(c82aa980,eca27d38,c82aa980,c063d35c,0,...) at usb_event_thread+0x70 fork_exit(c063d35c,c82aa980,eca27d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xeca27d6c, ebp = 0 --- Tracing command usb3 pid 38 tid 100027 td 0xc80fdd00 sched_switch(c80fdd00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82c4210,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c82c4210) at sleepq_timedwait+0x4a msleep(c82c4210,0,5c,c093cc30,ea60,c82bf4c0) at msleep+0x259 usb_event_thread(c82bf4c0,e6d90d38,c82bf4c0,c063d35c,0,...) at usb_event_thread+0x70 fork_exit(c063d35c,c82bf4c0,e6d90d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d90d6c, ebp = 0 --- Tracing command usb2 pid 37 tid 100028 td 0xc80fdb60 sched_switch(c80fdb60,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82ab210,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c82ab210) at sleepq_timedwait+0x4a msleep(c82ab210,0,5c,c093cc30,ea60,c82bfd80) at msleep+0x259 usb_event_thread(c82bfd80,e6d8dd38,c82bfd80,c063d35c,0,...) at usb_event_thread+0x70 fork_exit(c063d35c,c82bfd80,e6d8dd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d8dd6c, ebp = 0 --- Tracing command usb1 pid 36 tid 100029 td 0xc80fd9c0 sched_switch(c80fd9c0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c82af210,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c82af210) at sleepq_timedwait+0x4a msleep(c82af210,0,5c,c093cc30,ea60,c82a9300) at msleep+0x259 usb_event_thread(c82a9300,e6d8ad38,c82a9300,c063d35c,0,...) at usb_event_thread+0x70 fork_exit(c063d35c,c82a9300,e6d8ad38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d8ad6c, ebp = 0 --- Tracing command irq22: uhci1 uhci3 pid 35 tid 100030 td 0xc80fd820 fork_trampoline() at fork_trampoline Tracing command usbtask pid 34 tid 100031 td 0xc80fd680 sched_switch(c80fd680,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a37ba4,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c0a37ba4,0,c0698748,c0a3ac00,c093cc15,...) at sleepq_wait+0x46 msleep(c0a37ba4,0,5c,c093cc37,0,...) at msleep+0x27d usb_task_thread(0,e6d84d38,0,c063d3ec,0,...) at usb_task_thread+0x47 fork_exit(c063d3ec,0,e6d84d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d84d6c, ebp = 0 --- Tracing command usb0 pid 33 tid 100032 td 0xc80fd4e0 sched_switch(c80fd4e0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c8278210,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c8278210) at sleepq_timedwait+0x4a msleep(c8278210,0,5c,c093cc30,ea60,c829da80) at msleep+0x259 usb_event_thread(c829da80,e6d81d38,c829da80,c063d35c,0,...) at usb_event_thread+0x70 fork_exit(c063d35c,c829da80,e6d81d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d81d6c, ebp = 0 --- Tracing command irq23: uhci0 uhci+ pid 32 tid 100033 td 0xc828a000 fork_trampoline() at fork_trampoline Tracing command irq257: bce1 pid 31 tid 100034 td 0xc8287d00 sched_switch(c8287d00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 ithread_loop(c82a4020,e89edd38,c82a4020,c068bd80,0,...) at ithread_loop+0xda fork_exit(c068bd80,c82a4020,e89edd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89edd6c, ebp = 0 --- Tracing command irq256: bce0 pid 30 tid 100035 td 0xc8287b60 sched_switch(c8287b60,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 ithread_loop(c8299680,e89ead38,c8299680,c068bd80,0,...) at ithread_loop+0xda fork_exit(c068bd80,c8299680,e89ead38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89ead6c, ebp = 0 --- Tracing command aac0aif pid 29 tid 100036 td 0xc82879c0 sched_switch(c82879c0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c828b218,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c828b218) at sleepq_timedwait+0x4a msleep(c828b218,c822e130,4c,c09085dd,4e20) at msleep+0x259 aac_command_thread(c822e000,e89e7d38,c822e000,c047bec4,0,...) at aac_command_thread+0x65 fork_exit(c047bec4,c822e000,e89e7d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89e7d6c, ebp = 0 --- Tracing command irq17: aac0 pid 28 tid 100037 td 0xc8287820 sched_switch(c8287820,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 ithread_loop(c8227920,e89e4d38,c8227920,c068bd80,0,...) at ithread_loop+0xda fork_exit(c068bd80,c8227920,e89e4d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe89e4d6c, ebp = 0 --- Tracing command irq9: acpi0 pid 27 tid 100017 td 0xc80f9680 fork_trampoline() at fork_trampoline Tracing command swi5: + pid 26 tid 100018 td 0xc80f94e0 fork_trampoline() at fork_trampoline Tracing command thread taskq pid 25 tid 100019 td 0xc80f9340 sched_switch(c80f9340,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c8150100,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c8150100,0,c094a991,49,8,...) at sleepq_wait+0x46 msleep(c8150100,c815011c,0,c094005e,0) at msleep+0x27d taskqueue_thread_loop(c0a45388,e6d63d38,c0a45388,c06c2bc8,0,...) at taskqueue_thread_loop+0x78 fork_exit(c06c2bc8,c0a45388,e6d63d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d63d6c, ebp = 0 --- Tracing command swi6: Giant taskq pid 24 tid 100020 td 0xc80f91a0 fork_trampoline() at fork_trampoline Tracing command acpi_task_2 pid 9 tid 100021 td 0xc81f2820 sched_switch(c81f2820,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c8150280,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c8150280,0,c094a991,49,8,...) at sleepq_wait+0x46 msleep(c8150280,c815029c,0,c094005e,0) at msleep+0x27d taskqueue_thread_loop(c0c204e0,e87c1d38,c0c204e0,c06c2bc8,0,...) at taskqueue_thread_loop+0x78 fork_exit(c06c2bc8,c0c204e0,e87c1d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe87c1d6c, ebp = 0 --- Tracing command acpi_task_1 pid 8 tid 100022 td 0xc81f2680 sched_switch(c81f2680,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c8150280,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c8150280,0,c094a991,49,8,...) at sleepq_wait+0x46 msleep(c8150280,c815029c,0,c094005e,0) at msleep+0x27d taskqueue_thread_loop(c0c204e0,e87bed38,c0c204e0,c06c2bc8,0,...) at taskqueue_thread_loop+0x78 fork_exit(c06c2bc8,c0c204e0,e87bed38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe87bed6c, ebp = 0 --- Tracing command acpi_task_0 pid 7 tid 100023 td 0xc81f24e0 sched_switch(c81f24e0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c8150280,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c8150280,0,c094a991,49,c813f2c0,...) at sleepq_wait+0x46 msleep(c8150280,c815029c,0,c094005e,0) at msleep+0x27d taskqueue_thread_loop(c0c204e0,e87bbd38,c0c204e0,c06c2bc8,0,...) at taskqueue_thread_loop+0x78 fork_exit(c06c2bc8,c0c204e0,e87bbd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe87bbd6c, ebp = 0 --- Tracing command swi6: task queue pid 23 tid 100024 td 0xc81f2340 sched_switch(c81f2340,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 ithread_loop(c8185580,e87b8d38,c8185580,c068bd80,0,...) at ithread_loop+0xda fork_exit(c068bd80,c8185580,e87b8d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe87b8d6c, ebp = 0 --- Tracing command kqueue taskq pid 6 tid 100025 td 0xc81f21a0 sched_switch(c81f21a0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c8150400,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c8150400,0,c094a991,49,8,...) at sleepq_wait+0x46 msleep(c8150400,c815041c,0,c094005e,0) at msleep+0x27d taskqueue_thread_loop(c0a38d40,e87b5d38,c0a38d40,c06c2bc8,0,...) at taskqueue_thread_loop+0x78 fork_exit(c06c2bc8,c0a38d40,e87b5d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe87b5d6c, ebp = 0 --- Tracing command swi2: cambio pid 22 tid 100026 td 0xc81f2000 fork_trampoline() at fork_trampoline Tracing command xpt_thrd pid 5 tid 100008 td 0xc80f71a0 sched_switch(c80f71a0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a1e204,c0a3abc0,0,c094a337,21f,...) at sleepq_switch+0xc1 sleepq_wait(c0a1e204,0,c0698748,c0a3ac00,c090126d,...) at sleepq_wait+0x46 msleep(c0a1e204,0,4c,c0901288,0) at msleep+0x27d xpt_scanner_thread(0,e6d45d38,0,c0459fdc,0,...) at xpt_scanner_thread+0x35 fork_exit(c0459fdc,0,e6d45d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d45d6c, ebp = 0 --- Tracing command yarrow pid 21 tid 100009 td 0xc80f7000 sched_switch(c80f7000,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a358c0,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a358c0) at sleepq_timedwait+0x4a msleep(c0a358c0,0,0,c094005e,64) at msleep+0x259 random_kthread(0,e6d42d38,0,c05ec5fc,0,...) at random_kthread+0x194 fork_exit(c05ec5fc,0,e6d42d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d42d6c, ebp = 0 --- Tracing command g_down pid 4 tid 100010 td 0xc80fd340 sched_switch(c80fd340,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a38488,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a38488) at sleepq_timedwait+0x4a msleep(c0a38488,c0a38368,24c,c094005e,64) at msleep+0x259 g_io_schedule_down(c80fd340) at g_io_schedule_down+0x56 g_down_procbody(0,e6d7ed38,0,c0669918,0,...) at g_down_procbody+0x5a fork_exit(c0669918,0,e6d7ed38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d7ed6c, ebp = 0 --- Tracing command g_up pid 3 tid 100011 td 0xc80fd1a0 sched_switch(c80fd1a0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a38484,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a38484) at sleepq_timedwait+0x4a msleep(c0a38484,c0a383a8,24c,c094005e,64) at msleep+0x259 g_io_schedule_up(c80fd1a0) at g_io_schedule_up+0x127 g_up_procbody(0,e6d7bd38,0,c06698b8,0,...) at g_up_procbody+0x5a fork_exit(c06698b8,0,e6d7bd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d7bd6c, ebp = 0 --- Tracing command g_event pid 2 tid 100012 td 0xc80fd000 sched_switch(c80fd000,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 sleepq_switch(c0a3847c,c0a3abc0,0,c094a337,244,...) at sleepq_switch+0xc1 sleepq_timedwait(c0a3847c) at sleepq_timedwait+0x4a msleep(c0a3847c,0,4c,c094005e,64) at msleep+0x259 g_event_procbody(0,e6d78d38,0,c0669978,0,...) at g_event_procbody+0x9e fork_exit(c0669978,0,e6d78d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d78d6c, ebp = 0 --- Tracing command swi1: net pid 20 tid 100013 td 0xc80f9d00 sched_switch(c80f9d00,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 ithread_loop(c80cb970,e6d75d38,c80cb970,c068bd80,0,...) at ithread_loop+0xda fork_exit(c068bd80,c80cb970,e6d75d38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d75d6c, ebp = 0 --- Tracing command swi3: vm pid 19 tid 100014 td 0xc80f9b60 fork_trampoline() at fork_trampoline Tracing command swi4: clock sio pid 18 tid 100015 td 0xc80f99c0 sched_switch(c80f99c0,0,1) at sched_switch+0x15b mi_switch(1,0) at mi_switch+0x270 ithread_loop(c80cb990,e6d6fd38,c80cb990,c068bd80,0,...) at ithread_loop+0xda fork_exit(c068bd80,c80cb990,e6d6fd38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d6fd6c, ebp = 0 --- Tracing command idle: cpu0 pid 17 tid 100016 td 0xc80f9820 sched_switch(c81efa18,e6d6cc34,c06b488b,c0a3abc0,2,...) at sched_switch+0x15b _end(0,c8102440,0,c80fa90c,0,...) at 0xc81efa00 _end(c80fa900,c80fa904,c80f9820,c80f9828,0,...) at 0xc80fa900 _end(c80fb000,c80fb004,c80f9000,c80f9008,0,...) at 0xc8102218 _end(c80faea0,c80faea4,c80f7d00,c80f7d08,0,...) at 0xc8102430 _end(c80fae40,c80fae44,c80f7b60,c80f7b68,0,...) at 0xc80f6000 _end(c80fade0,c80fade4,c80f79c0,c80f79c8,0,...) at 0xc80f6218 _end(c80fad80,c80fad84,c80f7820,c80f7828,0,...) at 0xc80f6430 _end(c80fad20,c80fad24,c80f7680,c80f7688,0,...) at 0xc80f6648 _end(c80facc0,c80facc4,c80f74e0,c80f74e8,0,...) at 0xc80f6860 _end(c80fac60,c80fac64,c80f7340,c80f7348,0,...) at 0xc80f6a78 _end(c0a38520,c0a38524,c0a387a0,c0a387a8,0,...) at 0xc80f6c90 _end() at 0xc80fc000 Tracing command idle: cpu1 pid 16 tid 100000 td 0xc80f9000 sched_switch(28,46,c08a84b1,c0ac2ff0,e6d50008,...) at sched_switch+0x15b _end(0,0,0,0,0,...) at 0xc80f0028 _end() at 0xc80f0008 Tracing command idle: cpu2 pid 15 tid 100001 td 0xc80f7d00 kdb_enter(c096bc03) at kdb_enter+0x2b siointr1(c82d7800,c0aa8ae0,0,c096ba0f,56e,...) at siointr1+0xce siointr(c82d7800) at siointr+0x21 intr_execute_handlers(c80eb4c8,e6d5ac94,4,e6d5acd8,c08a8123,...) at intr_execute_handlers+0xe1 lapic_handle_intr(38) at lapic_handle_intr+0x2e Xapic_isr1() at Xapic_isr1+0x33 --- interrupt, eip = 0xc0c115bd, esp = 0xe6d5acd8, ebp = 0xe6d5acd8 --- acpi_cpu_c1(c0a3abc0,4,c80f6218,4,c80f6218,...) at acpi_cpu_c1+0x5 acpi_cpu_idle(e6d5ad0c,c068b0d9,c068b07c,e6d5ad24,c068aebc,...) at acpi_cpu_idle+0x162 cpu_idle(c068b07c,e6d5ad24,c068aebc,0,e6d5ad38,...) at cpu_idle+0x28 idle_proc(0,e6d5ad38,0,c068b07c,0,...) at idle_proc+0x5d fork_exit(c068b07c,0,e6d5ad38) at fork_exit+0xa0 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip = 0, esp = 0xe6d5ad6c, ebp = 0 --- Tracing command idle: cpu3 pid 14 tid 100002 td 0xc80f7b60 sched_switch(28,46,c08a84b1,c0ac34d0,e6d50008,...) at sched_switch+0x15b _end(0,0,0,0,0,...) at 0xc80f0028 _end() at 0xc80f0008 Tracing command idle: cpu4 pid 13 tid 100003 td 0xc80f79c0 sched_switch(28,46,c08a84b1,c0ac3740,e6d50008,...) at sched_switch+0x15b _end(0,0,0,0,0,...) at 0xc80f0028 _end() at 0xc80f0008 Tracing command idle: cpu5 pid 12 tid 100004 td 0xc80f7820 sched_switch(28,46,c08a84b1,c0ac39b0,e6d50008,...) at sched_switch+0x15b _end(0,0,0,0,0,...) at 0xc80f0028 _end() at 0xc80f0008 Tracing command idle: cpu6 pid 11 tid 100005 td 0xc80f7680 sched_switch(e6d4ec1c,c06c7a70,c0a468a0,0,c094b67e,...) at sched_switch+0x15b w_locklistdata(5657e589,84d8b53,8b107d8b,a1641475,0,...) at 0xc0a8ce58 *** error reading from address c9fc5d8f *** db> -- wbr, pluknet