Date: Wed, 19 Oct 2005 13:04:27 -0400 From: John Baldwin <jhb@freebsd.org> To: freebsd-current@freebsd.org Cc: Robert Watson <rwatson@freebsd.org> Subject: Re: Problem remains with FreeBSD 6.0-RC1 as seen in RELENG_5 Message-ID: <200510191304.28422.jhb@freebsd.org> In-Reply-To: <20051019175020.S60849@fledge.watson.org> References: <200510191623.j9JGNSfr007356@magus.nostrum.com> <20051019175020.S60849@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday 19 October 2005 12:54 pm, Robert Watson wrote: > On Wed, 19 Oct 2005, Philip Kizer wrote: > > I have a problem I reported on freebsd-stable several weeks ago in: > > > > > > http://www.FreeBSD.org/cgi/getmsg.cgi?fetch=47770+187449+/usr/local/www/d > >b/text/2005/freebsd-stable/20051009.freebsd-stable > > > > I upgraded a test box to see if all of the reports were true that > > threading and most other major problems were better in the 6.x branch, > > but I have had the same kind of hangs with 6.0-RC1 that I was having > > with RELENG_5. > > > > I get notified that some of my services are unavailable and I verify > > that new connection attempts from remote just hang. Attempts at issuing > > commands from my existing ssh connections will let me send a <return> > > and see a new prompt generated, but any attempt at execing/etc will then > > hang that process. > > This appears to be a problem with file descriptor passing and garbage > collection. I've seen one report of a lock order reversal along these > lines, but it was not believed to be symptomatic of an actual hang, just > an architectural issue. This could be a sign that we need to address the > source of the reversal, although it sounds like you don't get a reversal > warning? > > Could I have you try the following DDB commands also: > > show alllocks > traceall > > It's possible that 'traceall' isn't present in RC1, in which case it > will be present in RC2. Regardless, the output of "show alllocks" > would be helpful. > > I'll read some code this evening and ponder. He'll need witness for 'show alllocks' but that will require having the 'hptlock' spin mutex added to witness' spin lock order. Also, the psuedo-rw lock phk created for filedesc isn't witness checked since it isn't a mutex or sx lock, and it maybe that there is a reversal involving that lock, but it will be tricky to tell if that is the case. > Thanks, > > Robert N M Watson > > > Moving to the console, I get the same behaviour such that if it is > > already logged in, I can hit return in the shell and get a new shell > > prompt, but any command I try (i.e. 'uptime' or 'uname') then results in > > the same lack of any further response. [Though, obviously from below, > > sending a break to activate DDB still works.] > > > > If the console is not already logged in and I hit return at "login:", I > > get a new "login:" prompt; but, as soon as I enter a username+^M or even > > Ctrl-D it, I cease to receive any feedback besides terminal echo from my > > input. > > > > > > This is: > > FreeBSD [host-elided] 6.0-RC1 FreeBSD 6.0-RC1 #0: Mon Oct 17 15:52:35 CDT > > 2005 root@[host-elided]:/usr/obj/usr/src/sys/SMP_DDB i386 > > > > # grep '^[a-z]' /sys/i386/conf/SMP_DDB > > include GENERIC > > ident DDB > > options SMP # Symmetric MultiProcessor Kernel > > options INCLUDE_CONFIG_FILE # Include this file in kernel > > options DDB #Enable the kernel debugger > > options DDB_NUMSYM #Print numerical value of symbols too > > options KDB > > options KDB_TRACE > > options BREAK_TO_DEBUGGER > > > > [After notification of a problem, this is from the serial console...a few > > further inline comments included in brackets "[]".] > > > > # > > [Typed: Ctrl-T] > > load: 1.82 cmd: sh 795 [ttyin] 0.00u 0.00s 0% 168k > > # uptime > > [Waited a while to be sure it was hung and not returning. Retyped: > > Ctrl-T] load: 1.82 cmd: uptime 83303 [filelist lock] 0.00u 0.00s 0% 180k > > [Sent BREAK] > > KDB: enter: Line break on console > > [thread pid 82885 tid 100133 ] > > Stopped at 0xc0659b37 = kdb_enter+0x2b: nop > > db> set $lines=0 > > db> trace > > Tracing pid 82885 tid 100133 td 0xc3c34c00 > > kdb_enter(c0895a5e) at 0xc0659b37 = kdb_enter+0x2b > > siointr1(c372ec00) at 0xc07f09b2 = siointr1+0xce > > siointr(c372ec00) at 0xc07f0792 = siointr+0x5e > > intr_execute_handlers(c3528090,f1325b68,4,f1325bcc,c0805073) at > > 0xc0808985 = intr_execute_handlers+0x85 lapic_handle_intr(34) at > > 0xc080b042 = lapic_handle_intr+0x2e > > Xapic_isr1() at 0xc0805073 = Xapic_isr1+0x33 > > --- interrupt, eip = 0xc0685d59, esp = 0xf1325bac, ebp = 0xf1325bcc --- > > unp_gc(c43be2c8) at 0xc0685d59 = unp_gc+0xf1 > > unp_detach(c43b7000,f1325c08,c067a188,c43be2c8,0) at 0xc06840fa = > > unp_detach+0xae uipc_detach(c43be2c8) at 0xc0682f8e = uipc_detach+0x82 > > soclose(c43be2c8,c40fb6c0,0,f1325c34,c0620c10) at 0xc067a188 = > > soclose+0x278 soo_close(c40fb6c0,c3c34c00) at 0xc0669fb3 = soo_close+0x63 > > fdrop_locked(c40fb6c0,c3c34c00,c3a3d800,f1325cb4,c061f0fb) at 0xc0620c10 > > = fdrop_locked+0xac fdrop(c40fb6c0,c3c34c00,c066371f,c40fb6c0,c878ac00) > > at 0xc0620b5d = fdrop+0x41 closef(c40fb6c0,c3c34c00,0,c3c34c00,6) at > > 0xc061f0fb = closef+0x427 close(c3c34c00,f1325d04,1,0,292) at 0xc061c4bd > > = close+0x211 > > syscall(3b,1003b,bfbf003b,80d58e7,80e8780) at 0xc0818557 = syscall+0x2bf > > Xint0x80_syscall() at 0xc0804d0f = Xint0x80_syscall+0x1f > > --- syscall (6, FreeBSD ELF32, close), eip = 0x282cb7d3, esp = > > 0xbfbfae4c, ebp = 0xbfbfae68 --- db> show reg > > cs 0x20 > > ds 0xc0870028 > > es 0xc0650028 parse_dynamic+0x4 > > fs 0xf1320008 > > ss 0x28 > > eax 0x22 > > ecx 0xc1033000 > > edx 0 > > ebx 0xc3c34c00 > > esp 0xf1325afc > > ebp 0xf1325afc > > esi 0xc372ec00 > > edi 0xf9 > > eip 0xc0659b37 kdb_enter+0x2b > > efl 0x86 > > dr0 0 > > dr1 0 > > dr2 0 > > dr3 0 > > dr4 0xffff0ff0 > > dr5 0x400 > > dr6 0xffff0ff0 > > dr7 0x400 > > 0xc0659b37 = kdb_enter+0x2b: nop > > db> show witness > > No such command > > db> show pcpu > > cpuid = 1 > > curthread = 0xc3c34c00: pid 82885 \"sendmail\" > > curpcb = 0xf1325d90 > > fpcurthread = none > > idlethread = 0xc34d8780: pid 11 \"idle: cpu1\" > > APIC ID = 0 > > currentldt = 0x50 > > db> show pcpuu0 > > cpuid = 0 > > curthread = 0xc34d8900: pid 12 \"idle: cpu0\" > > curpcb = 0xe4f9bd90 > > fpcurthread = none > > idlethread = 0xc34d8900: pid 12 \"idle: cpu0\" > > APIC ID = 1 > > currentldt = 0x50 > > db> show pcpu 1 > > cpuid = 1 > > curthread = 0xc3c34c00: pid 82885 \"sendmail\" > > curpcb = 0xf1325d90 > > fpcurthread = none > > idlethread = 0xc34d8780: pid 11 \"idle: cpu1\" > > APIC ID = 0 > > currentldt = 0x50 > > db> show pcpu 2 > > CPU 2 not found > > db> print sysctllock > > c093ace0 > > db> ps > > pid proc uid ppid pgrp flag stat wmesg wchan cmd > > 83305 c43b2830 0 504 504 0000000 [SLPQ filelist lock > > 0xc0938bd0][SLP] cron 83304 c3d91a3c 21802 605 605 0000000 [SLPQ > > filelist lock 0xc0938bd0][SLP] perl 83303 c3d12c48 0 795 83303 > > 0004002 [SLPQ filelist lock 0xc0938bd0][SLP] uptime 83302 c5b13418 0 > > 686 686 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83301 > > c3ba1c48 0 680 680 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83300 c40c320c 0 504 504 0000000 [SLPQ filelist lock > > 0xc0938bd0][SLP] cron 83299 c43a2830 0 504 504 0000000 [SLPQ > > filelist lock 0xc0938bd0][SLP] cron 83298 c38dec48 0 504 504 > > 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] cron 83297 c3f0020c 21802 > > 605 605 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 83296 > > c3c36624 21802 605 605 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] > > perl 83295 c406620c 21802 605 605 0000000 [SLPQ filelist lock > > 0xc0938bd0][SLP] perl 83294 c38de20c 21802 605 605 0000000 [SLPQ > > filelist lock 0xc0938bd0][SLP] perl 83293 c41af418 0 686 686 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83292 c4244a3c 0 > > 680 680 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83291 > > c40c0624 0 504 504 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] > > cron 83290 c43b1a3c 0 504 504 0000000 [SLPQ filelist lock > > 0xc0938bd0][SLP] cron 83289 c5fffc48 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83288 c3e9d830 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83287 c3cbbc48 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83286 > > c43a2000 0 636 636 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail > > 83285 c40c4c48 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83284 c413b624 0 636 636 0000100 [SLPQ > > select 0xc0941aa4][SLP] sendmail 83283 c6003418 0 664 664 0000100 > > [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83282 c4140000 0 650 > > 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83281 c43ac000 > > 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail > > 83280 c3d91830 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83279 c43afc48 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83278 c3cbb418 0 664 664 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83277 c4244c48 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83276 > > c40c0830 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83275 c4136624 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83274 c4140a3c 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83273 c6035830 0 636 636 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83272 c43b120c 0 > > 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83271 > > c41ada3c 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83270 c3fca624 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83269 c6035c48 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83268 c5b13c48 0 664 664 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83267 c3f82c48 0 > > 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83266 > > c424320c 0 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83265 c43a720c 0 636 636 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83264 c3e9d418 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83263 c3d18000 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83261 c6036418 0 > > 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83260 > > c5b0120c 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83259 c4244000 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83258 c4243830 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83257 c43af830 0 1 83257 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83255 c5b01830 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83254 > > c5b01000 0 650 650 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail > > 83248 c3b99c48 0 680 680 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83247 c42d5c48 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83246 c3f00a3c 0 664 664 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83245 c6036000 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83244 > > c40c3c48 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83243 c5fff830 0 636 636 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83242 c5b01418 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83241 c42a2624 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83240 c41af000 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83239 > > c41ad830 0 636 636 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83238 c3d1820c 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83237 c3f00624 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83236 c5b13a3c 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83235 c42a7a3c 0 > > 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83234 > > c3f82830 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83233 c43a7624 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83231 c4376418 0 683 683 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83230 c406e830 0 664 664 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83229 c42d520c 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83228 > > c43af418 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83227 c413ba3c 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83226 c3e9d20c 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83225 c43a220c 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83224 c3b99418 0 > > 636 636 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83223 > > c3c33c48 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83222 c4375000 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83221 c3e9d624 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83220 c38de830 0 636 636 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83219 c3f8020c 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83218 > > c3f02a3c 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83213 c4375a3c 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83212 c5b09418 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83211 c43af624 0 664 664 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83208 c4135830 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83207 > > c41afa3c 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83206 c406ea3c 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83205 c413b418 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83195 c3f80624 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83193 c5b0920c 0 > > 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83192 > > c3c33624 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83191 c3c36a3c 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83190 c4140418 0 636 636 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83189 c4135418 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83188 c437620c 0 > > 636 636 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83187 > > c406e624 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83186 c43a7418 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83185 c3c36c48 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83184 c3cbb830 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83183 c406e000 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83182 > > c43a2c48 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83181 c40c3000 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83180 c4140c48 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83179 c42a2418 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83178 c437e000 0 > > 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83175 > > c5fff20c 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83174 c3f82624 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83168 c3ba1830 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83167 c4243a3c 0 664 664 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83164 c6003000 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83162 > > c4066a3c 0 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83157 c40c4000 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83154 c600320c 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83153 c40c0c48 0 664 664 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83152 c3d18418 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83150 > > c41adc48 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83149 c3e9dc48 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83148 c3cba624 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83146 c3c36000 0 650 650 > > 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail 83145 c3f01830 0 650 > > 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83142 c41af20c > > 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail > > 83141 c3e9da3c 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83140 c5fff624 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83134 c41af624 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83132 c4136830 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83131 > > c603620c 0 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83130 c3f00418 0 650 650 0000100 [SLPQ select > > 0xc0941aa4][SLP] sendmail 83129 c3fcac48 0 664 664 0000100 [SLPQ > > select 0xc0941aa4][SLP] sendmail 83128 c40c4418 0 650 650 0000100 > > [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83125 c42d4c48 0 664 > > 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83124 c3fca20c > > 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail > > 83123 c437e418 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83121 c4135c48 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83119 c41af830 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83118 c38dd000 0 > > 650 650 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail 83117 c42d4a3c > > 0 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail > > 83114 c42a720c 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83113 c41aac48 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83112 c42d420c 0 664 664 > > 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail 83109 c40c4a3c 0 650 > > 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83108 c3f8220c > > 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail > > 83107 c3d10830 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83105 c41aa000 0 650 650 0000100 [SLPQ > > select 0xc0941aa4][SLP][SWAP] sendmail 83101 c4066624 0 664 664 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83100 c40c3624 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83097 > > c42a2000 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83094 c41aa830 0 650 650 0000100 [SLPQ select > > 0xc0941aa4][SLP] sendmail 83093 c43a7a3c 0 664 664 0000100 [SLPQ > > select 0xc0941aa4][SLP] sendmail 83092 c43b1000 0 650 650 0000100 > > [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83091 c437ec48 0 650 > > 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83087 c5fff000 > > 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail > > 83070 c42a2830 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83069 c6036c48 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83056 c6035624 0 664 664 > > 0000100 [SLPQ select 0xc0941aa4][SLP][SWAP] sendmail 83051 c41ad20c 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83050 > > c4244830 0 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 83041 c40c0a3c 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 83038 c424420c 0 664 664 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 83037 c43a2624 0 664 664 > > 0000100 [SLPQ select 0xc0941aa4][SLP][SWAP] sendmail 83034 c413b000 0 > > 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82994 > > c43b2000 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82993 c38d8c48 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 82992 c5b01a3c 21802 551 551 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 82991 c5b01c48 21802 550 550 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 82990 c43b2624 21802 551 551 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 82989 c5b09000 21802 550 550 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82987 c6036830 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82986 c42d5624 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82984 c5b09624 21802 > > 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82983 c5b09830 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82982 c4135a3c > > 21802 551 551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 82981 > > c3f82000 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82976 c42d5830 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 82975 c5b09c48 21802 550 550 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 82974 c43acc48 21802 551 551 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 82973 c3f02418 21802 550 550 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 82972 c5b13000 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82971 c43ac20c 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82970 c3cbb20c 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82969 c43a2418 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82967 c43a7000 21802 > > 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82966 c40c0000 > > 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82965 > > c4376c48 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82964 c43b1c48 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 82962 c41ad624 21802 551 551 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 82961 c38dea3c 21802 550 550 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 82960 c38a820c 21802 551 551 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 82959 c43b0c48 21802 550 550 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82952 c42a2a3c 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82951 c6035000 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82948 c43b220c 21802 > > 551 551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 82947 > > c6035a3c 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82945 c43b2418 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 82944 c41aaa3c 21802 550 550 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 82938 c40c420c 21802 551 551 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 82937 c6036a3c 21802 550 550 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 82932 c3f01a3c 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82931 c42d4624 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82930 c41aa418 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82929 c3d94c48 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82928 c3f02830 21802 > > 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82927 c3d94a3c > > 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82926 > > c413620c 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82925 c3fcbc48 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 82924 c42d5a3c 21802 551 551 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 82923 c4140830 21802 550 550 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 82922 c43ac830 21802 551 551 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 82921 c3f0220c 21802 550 550 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82919 c43b0418 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82918 c3cbba3c > > 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82917 > > c4066830 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82916 c3ba1624 0 636 636 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 82915 c4376000 21802 551 551 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 82914 c3f01c48 21802 550 550 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 82913 c43b1418 0 636 636 0000100 > > [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82909 c4243c48 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82908 c3c33000 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82907 c4243000 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82906 > > c437ea3c 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82905 c3ba1000 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 82904 c43b1624 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 82903 c40c3a3c 21802 551 551 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 82902 c3f82a3c 21802 550 550 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 82896 c43b0830 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82895 c4136000 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82894 c42a7000 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82893 c3e9d000 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82892 c3c3320c 21802 > > 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82891 c42a220c > > 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82890 > > c4375c48 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 82889 c3ba1a3c 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 82888 c43af000 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 82887 c42a2c48 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82886 c3c33418 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82885 c3c36830 0 636 > > 636 0000100 [CPU 1] sendmail > > 82877 c42a7418 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 82876 c3cbaa3c 21802 550 550 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 82874 c413bc48 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 82870 c43a7c48 0 650 650 > > 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82865 c437520c > > 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82864 > > c4376830 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82851 c3f80418 0 650 650 0000100 [SLPQ select > > 0xc0941aa4][SLP][SWAP] sendmail 82837 c3f02c48 0 650 650 0000100 > > [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82782 c42d5000 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82781 c3d9120c 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82769 c43aca3c 0 > > 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82766 > > c40c020c 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 82765 c41aa20c 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 82763 c3d1220c 0 664 664 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 82762 c3d9420c 21802 551 551 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 82761 c42a7624 21802 550 550 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 82740 c6036624 0 664 664 0000100 > > [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82738 c4136a3c 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82737 c4243418 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82729 c3f82418 0 > > 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82676 > > c4376624 0 664 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 82621 c3c33a3c 0 650 650 0000100 [SLPQ filelist lock > > 0xc0938bd0][SLP] sendmail 82607 c4376a3c 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 81310 c6035418 0 664 664 > > 0000100 [SLPQ select 0xc0941aa4][SLP][SWAP] sendmail 80076 c42d4000 0 > > 650 650 0000100 [SLPQ select 0xc0941aa4][SLP][SWAP] sendmail 79627 > > c4135624 0 650 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 79198 c3cba830 21802 551 551 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 79197 c414020c 21802 550 550 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 79193 c4066418 21802 551 551 0000000 > > [SLPQ filelist lock 0xc0938bd0][SLP] perl 79192 c40c3830 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79189 c3f00000 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79188 c437e830 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79186 c3d18830 21802 > > 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79185 c3fca418 > > 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79184 > > c4136418 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 79183 c3fcaa3c 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 79179 c3f80000 21802 551 551 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 79178 c3fcb20c 21802 550 550 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 79168 c603520c 21802 551 551 0000000 > > [SLPQ filelist lock 0xc0938bd0][SLP] perl 79167 c4243624 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79083 c4375830 0 664 > > 664 0000100 [SLPQ sbwait 0xc4401900][SLP][SWAP] sendmail 78838 c3f01000 > > 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 78837 > > c3f80830 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 78491 c40c4624 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 78490 c3d94830 21802 550 550 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 78484 c3f02000 21802 551 551 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 78483 c406ec48 21802 550 550 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 73847 c43ac418 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 73846 c42d4418 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 68027 c43b2a3c 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 67937 c3fcb000 21802 > > 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 67936 c3fcb624 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 67933 c42d4830 > > 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 67932 > > c38de624 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 67904 c413b830 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 67903 c3cbb624 21802 550 550 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 66287 c43afa3c 0 650 650 0000100 [SLPQ > > filelist lock 0xc0938bd0][SLP] sendmail 60279 c3f00830 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 60278 c4244624 21802 551 > > 551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 60277 c3fcba3c > > 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 60276 > > c3fca000 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 60249 c406e418 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 60248 c3f00c48 21802 550 550 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 51444 c3c3620c 21802 551 551 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 51443 c4066000 21802 550 550 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 51430 c3b99624 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51429 c41aa624 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51427 c40c4830 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51426 c3f0120c 21802 > > 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51231 c4375624 21802 > > 551 551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 51230 > > c3fcb830 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 51228 c3d10a3c 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 51227 c4140624 21802 550 550 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 51222 c413b20c 21802 551 551 0000000 [SLPQ > > filelist lock 0xc0938bd0][SLP] perl 51221 c38de418 21802 550 550 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51218 c43b020c 21802 551 > > 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51217 c3cbac48 21802 550 > > 550 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 51215 c43a2a3c > > 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51214 > > c437e20c 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 51213 c4244418 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] > > perl 51212 c3f02624 21802 550 550 0000000 [SLPQ select > > 0xc0941aa4][SLP] perl 51211 c3f80a3c 21802 551 551 0000000 [SLPQ > > select 0xc0941aa4][SLP] perl 51206 c43b0624 21802 550 550 0000000 > > [SLPQ select 0xc0941aa4][SLP] perl 51161 c413520c 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51160 c42d5418 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51153 c40c0418 21802 551 > > 551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 50987 c3c36418 > > 0 1 50987 0000100 [SLPQ sbwait 0xc3da3c40][SLP] sendmail 36954 c4135000 > > 21802 550 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 36941 > > c38dd418 21802 551 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 32803 c43af20c 0 32793 32803 0004003 [SLPQ ttyin 0xc3871810][SLP][SWAP] > > zsh 32793 c43b2c48 127 32792 32793 0004002 [SLPQ pause > > 0xc43b2c7c][SLP][SWAP] zsh 32792 c42a7c48 127 32781 32781 0000100 [SLPQ > > select 0xc0941aa4][SLP][SWAP] sshd 32781 c3fcb418 0 491 32781 0004100 > > [SLPQ sbwait 0xc4547e90][SLP][SWAP] sshd 31218 c3cbb000 21802 551 551 > > 0000000 [SLPQ select 0xc0941aa4][SLP] perl 31217 c3f01418 21802 550 > > 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 5491 c43b0a3c 0 1 5491 > > 0000100 [SLPQ sbwait 0xc45473e8][SLP] sendmail 795 c3d91c48 0 778 > > 795 0004002 [SLPQ wait 0xc3d91c48][SLP] sh 778 c3d94418 0 760 778 > > 0004002 [SLPQ pause 0xc3d9444c][SLP][SWAP] zsh 760 c3ea1000 0 1 760 > > 0004102 [SLPQ wait 0xc3ea1000][SLP][SWAP] login 759 c3ea120c 0 1 759 > > 0004002 [SLPQ ttyin 0xc3737410][SLP][SWAP] getty 758 c3ea1418 0 1 758 > > 0004002 [SLPQ ttyin 0xc3734c10][SLP][SWAP] getty 757 c3ea1624 0 1 757 > > 0004002 [SLPQ ttyin 0xc3734410][SLP][SWAP] getty 756 c3ea1830 0 1 756 > > 0004002 [SLPQ ttyin 0xc372f410][SLP][SWAP] getty 755 c3ea1a3c 0 1 755 > > 0004002 [SLPQ ttyin 0xc3736410][SLP][SWAP] getty 754 c3d18c48 0 1 754 > > 0004002 [SLPQ ttyin 0xc3730c10][SLP][SWAP] getty 753 c3d18624 0 1 753 > > 0004002 [SLPQ ttyin 0xc3735810][SLP][SWAP] getty 752 c38de000 0 1 752 > > 0004002 [SLPQ ttyin 0xc3734810][SLP][SWAP] getty 719 c3d91418 0 1 717 > > 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] snmpd 709 c3d18a3c 0 1 > > 709 0000100 [SLPQ pause 0xc3d18a70][SLP][SWAP] sendmail 706 c3d12418 > > 0 1 706 0000100 [SLPQ pause 0xc3d1244c][SLP][SWAP] sendmail 703 > > c3d91000 0 1 703 0000100 [SLPQ pause 0xc3d91034][SLP][SWAP] sendmail > > 700 c3d91624 0 1 700 0000100 [SLPQ pause 0xc3d91658][SLP][SWAP] > > sendmail 696 c3d12a3c 0 1 696 0000100 [SLPQ pause > > 0xc3d12a70][SLP][SWAP] sendmail 692 c3d94000 0 1 692 0000100 [SLPQ > > pause 0xc3d94034][SLP][SWAP] sendmail 689 c3d94624 0 1 689 0000100 > > [SLPQ pause 0xc3d94658][SLP][SWAP] sendmail 686 c3d1020c 0 1 686 > > 0000100 [SLPQ pause 0xc3d10240][SLP] sendmail 683 c3d10624 0 1 683 > > 0000100 [SLPQ pause 0xc3d10658][SLP][SWAP] sendmail 680 c3d12000 0 1 > > 680 0000100 [SLPQ pause 0xc3d12034][SLP] sendmail 678 c3d12624 0 1 > > 678 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail 664 c3d10000 0 1 > > 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 650 c3cba000 > > 0 1 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 636 > > c3d10418 0 1 636 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] > > sendmail 620 c38d8418 0 616 616 0000001 [SLPQ lockf > > 0xc41d3bc0][SLP][SWAP] saslauthd 619 c3b99830 0 616 616 0000001 > > [SLPQ lockf 0xc878a040][SLP][SWAP] saslauthd 618 c3cba418 0 616 616 > > 0000001 [SLPQ lockf 0xc44dc900][SLP][SWAP] saslauthd 617 c38d8624 0 > > 616 616 0000001 [SLPQ accept 0xc3cf0466][SLP][SWAP] saslauthd 616 > > c3d10c48 0 1 616 0000001 [SLPQ lockf 0xc44dc380][SLP][SWAP] saslauthd > > 605 c3cba20c 21802 1 605 0000000 [SLPQ select 0xc0941aa4][SLP] perl > > 604 c3d12830 0 577 604 0004003 [SLPQ ttyin 0xc3725810][SLP][SWAP] > > zsh 577 c3c33830 127 576 577 0004002 [SLPQ pause > > 0xc3c33864][SLP][SWAP] zsh 576 c3b9920c 127 574 574 0000100 [SLPQ > > select 0xc0941aa4][SLP][SWAP] sshd 574 c38dd624 0 491 574 0004100 > > [SLPQ sbwait 0xc3c30638][SLP][SWAP] sshd 551 c38a5a3c 21802 1 551 > > 0000001 [SLPQ select 0xc0941aa4][SLP] perl 550 c38ddc48 21802 1 550 > > 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 546 c3b99a3c 21802 1 > > 546 0000001 [SLPQ select 0xc0941aa4][SLP] perl 539 c38a5830 21802 534 > > 534 0000000 [SLPQ select 0xc0941aa4][SLP][SWAP] httpd 538 c38a8000 21802 > > 534 534 0000000 [SLPQ select 0xc0941aa4][SLP][SWAP] httpd 537 c38d820c > > 21802 534 534 0000000 [SLPQ select 0xc0941aa4][SLP][SWAP] httpd 536 > > c3ba120c 21802 534 534 0000000 [SLPQ select 0xc0941aa4][SLP][SWAP] > > httpd 535 c3ba1418 21802 534 534 0000000 [SLPQ select > > 0xc0941aa4][SLP][SWAP] httpd 534 c38d8000 21802 1 534 0000000 [SLPQ > > select 0xc0941aa4][SLP] httpd 527 c38a5624 0 1 527 0008180 (threaded) > > perl > > thread 0xc41c1a80 ksegrp 0xc34db660 [SLPQ kserel 0xc34db694][SLP] > > thread 0xc879b300 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc43a6a80 ksegrp 0xc34db660 [SLPQ kserel > > 0xc34db694][SLP] thread 0xc42eec00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a8480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59b5600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e8480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ade000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42ee480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e5180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a9300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a2480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc423a600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc44f2600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a0b780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a36600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e9780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42ee780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e9300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e7a80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a20000 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc87e9c00 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc42efa80 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc41c0780 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc4099900 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc43c9780 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc43ca180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c4180 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc41c1600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59e7c00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a2a80 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc5a20d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422d480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a20600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3ea2900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41a1180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a3780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4096480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59e7480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a6900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442dc00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f03000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a20a80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4adf180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc599d480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42ef600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442e600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ade480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e7180 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc879ba80 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc59b5180 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc4120d80 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc5a0b900 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc87e7600 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc599d900 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc423a900 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc3ea2600 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc87e8780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422d780 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc5a20900 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc5a36480 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc43ca300 ksegrp 0xc34db660 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc41a1600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3c35780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3d96900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87ead80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4067000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f28180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ade600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a96000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e5900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422d180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e9900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e7480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc44ef180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59b5900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e5d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc599d180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59b5000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ade780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a4480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4121300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f92900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422d600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4adf000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f8fa80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a8780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4adea80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3ea2480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc423aa80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4adf780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e5300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f03180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4099300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442d480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4068780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4121d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a4780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4095d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a2000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a4900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43c9d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e6d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a96600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43ca480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59b5d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59e7300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3d17600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e8c00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42f1600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a6780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987c00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422c600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442d300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a97000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3ea3c00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a8a80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc44efd80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87ea780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ade900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c4300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a20c00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e6900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3d17480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ae1c00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43cad80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87ea900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422dd80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e7d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e9180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4adf900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4aeb780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a20780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a2180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59e7d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43c9180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42f1480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c1780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a4d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c1180 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc44ef480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ae1a80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a3900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e9480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc44ef900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3d95d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc599d600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422cc00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc599d000 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3ea2300 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e6480 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42f1d80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e8600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4120780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc599d780 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41bfc00 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4aeb900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42ee900 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4067a80 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a8600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4121600 ksegrp 0xc34db660 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc38df900 ksegrp 0xc354d8a0 [SLPQ ksesigwait > > 0xc38a575c][SLP] 520 c38dd830 21803 1 69 000c083 (threaded) perl > > thread 0xc43a9600 ksegrp 0xc354dcc0 [SLPQ kserel 0xc354dcf4][SLP] > > thread 0xc8a97a80 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc41a0c00 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc5a0bc00 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc87e9d80 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc43a8900 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc59e7000 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc3f8f180 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc8a97d80 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc599dc00 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc4ae1480 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc42ee300 ksegrp 0xc354dcc0 [SLPQ kserel 0xc354dcf4][SLP] > > thread 0xc87e8a80 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP] > > thread 0xc87e5600 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc87e7300 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc5a0bd80 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc442da80 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc879b480 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc879b780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc44f2180 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc3f90780 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc5987d80 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc879bc00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e7c00 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc42ee180 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc87a2300 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc59b5480 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc442e900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987780 ksegrp 0xc354dcc0 [SLPQ filelist lock > > 0xc0938bd0][SLP] thread 0xc5a0b480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc599dd80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87ea180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3c34780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59b5c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c0900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a9900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f92c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a3480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e9000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f81d80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41a0d80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a36000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a9480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a96a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42f1900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a2d80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c0600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4120000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e8900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e7780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a3300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e9a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc423a480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4099d80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a20480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442d180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59e7180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4096d80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43ca600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5a36180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc423ad80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442ec00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c4c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e5c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4096a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3ea3600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c4000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a4c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43c9300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42eed80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a9a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4adfd80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ade180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41a1480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a4000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e6a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a9780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a96180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4aeb000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42ef480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4120a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4121180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc44ef300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a3a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a96780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59b5780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a9180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e8180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4adf600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc423a300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41bfd80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3ea3480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422c780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422c300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422d900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc44f2000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a97600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59e7900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f28780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4aeb180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e8d80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a8300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f28c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43c9600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43c9900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c0a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41bf600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422c900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43c9a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42f1a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3ea3900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43caa80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43ca900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43c9c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442d600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a96300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87ea480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc43a8180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c1c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3b9a000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41bf180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4121c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4aebc00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422da80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e5000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc422c180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87eaa80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42ef300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e6000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42f1c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59e7a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc41c0d80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4adfc00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442e780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4099000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87a4300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4121000 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a97180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4121480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc59b5300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc879b600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc87e9600 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f90c00 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a96d80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc8a97300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4122a80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc42eea80 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc5987300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc423a780 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4068300 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ae1180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc3f8f480 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc442e180 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc4ae1900 ksegrp 0xc354dcc0 [SLPQ select > > 0xc0941aa4][SLP] thread 0xc38e0180 ksegrp 0xc354d900 [SLPQ ksesigwait > > 0xc38dd968][SLP] 519 c38a5c48 0 1 69 0004002 [SLPQ sbwait > > 0xc39cabc8][SLP][SWAP] perl 504 c38a5418 0 1 504 0000000 [SLPQ nanslp > > 0xc093af0c][SLP] cron 491 c38a5000 0 1 491 0000100 [SLPQ filelist > > lock 0xc0938bd0][SLP] sshd 474 c38dd20c 0 1 474 0000000 [SLPQ select > > 0xc0941aa4][SLP] ntpd 381 c38a520c 0 1 381 0000000 [SLPQ select > > 0xc0941aa4][SLP] syslogd 352 c38d8830 0 1 352 0000000 [SLPQ select > > 0xc0941aa4][SLP][SWAP] devd 268 c38dda3c 64 262 262 0000100 [SLPQ > > bpf 0xc38dce00][SLP] pflogd 262 c38d8a3c 0 1 262 0000000 [SLPQ sbwait > > 0xc39ca370][SLP][SWAP] pflogd 68 c38a8418 0 0 0 0000204 [SLPQ - > > 0xef03ed08][SLP] schedcpu 67 c38a8624 0 0 0 0000204 [SLPQ - > > 0xc094a02c][SLP] nfsiod 3 66 c38a8830 0 0 0 0000204 [SLPQ - > > 0xc094a028][SLP] nfsiod 2 65 c38a8a3c 0 0 0 0000204 [SLPQ - > > 0xc094a024][SLP] nfsiod 1 64 c38a8c48 0 0 0 0000204 [SLPQ - > > 0xc094a020][SLP] nfsiod 0 63 c355a20c 0 0 0 0000204 [SLPQ vlruwt > > 0xc355a20c][SLP] vnlru 62 c355a418 0 0 0 0000204 [SLPQ syncer > > 0xc093ac7c][SLP] syncer 61 c355a624 0 0 0 0000204 [SLPQ psleep > > 0xc0941fec][SLP] bufdaemon 60 c355a830 0 0 0 000020c [SLPQ pgzero > > 0xc09505e4][SLP] pagezero 59 c355aa3c 0 0 0 0000204 [SLPQ psleep > > 0xc0950134][SLP] vmdaemon 58 c355ac48 0 0 0 0000204 [SLPQ psleep > > 0xc09500f0][SLP] pagedaemon 57 c3603000 0 0 0 0000204 [IWAIT] swi0: > > sio > > 56 c360320c 0 0 0 0000204 [SLPQ - 0xc372903c][SLP] fdc0 > > 55 c3603418 0 0 0 0000204 [SLPQ aifthd 0xc3603418][SLP] aac0aif > > 54 c3603624 0 0 0 0000204 [SLPQ idle 0xc3528c00][SLP] > > aic_recovery0 53 c3603830 0 0 0 0000204 [SLPQ idle 0xc3528c00][SLP] > > aic_recovery0 52 c3603a3c 0 0 0 0000204 [IWAIT] swi6:+ > > 51 c3603c48 0 0 0 0000204 [IWAIT] swi6: task queue > > 9 c354b624 0 0 0 0000204 [SLPQ - 0xc3563600][SLP] acpi_task2 > > 8 c354b830 0 0 0 0000204 [SLPQ - 0xc3563600][SLP] acpi_task1 > > 7 c354ba3c 0 0 0 0000204 [SLPQ - 0xc3563600][SLP] acpi_task0 > > 50 c354bc48 0 0 0 0000204 [IWAIT] swi2: cambio > > 6 c3559000 0 0 0 0000204 [SLPQ - 0xc3563780][SLP] kqueue taskq > > 49 c355920c 0 0 0 0000204 [IWAIT] swi5:+ > > 5 c3559418 0 0 0 0000204 [SLPQ - 0xc3563980][SLP] thread taskq > > 48 c3559624 0 0 0 0000204 [SLPQ - 0xc0935c60][SLP] yarrow > > 4 c3559830 0 0 0 0000204 [SLPQ - 0xc0938428][SLP] g_down > > 3 c3559a3c 0 0 0 0000204 [SLPQ - 0xc0938424][SLP] g_up > > 2 c3559c48 0 0 0 0000204 [SLPQ - 0xc093841c][SLP] g_event > > 47 c355a000 0 0 0 0000204 [IWAIT] swi1: net > > 46 c353bc48 0 0 0 0000204 [IWAIT] swi3: vm > > 45 c354a000 0 0 0 000020c [IWAIT] swi4: clock sio > > 44 c354a20c 0 0 0 0000204 [IWAIT] irq0: > > 43 c354a418 0 0 0 0000204 [IWAIT] irq31: aac0 > > 42 c354a624 0 0 0 0000204 [IWAIT] irq30: ahc0 > > 41 c354a830 0 0 0 0000204 [IWAIT] irq29: > > 40 c354aa3c 0 0 0 0000204 [IWAIT] irq28: > > 39 c354ac48 0 0 0 0000204 [IWAIT] irq27: > > 38 c354b000 0 0 0 0000204 [IWAIT] irq26: > > 37 c354b20c 0 0 0 0000204 [IWAIT] irq25: > > 36 c354b418 0 0 0 0000204 [IWAIT] irq24: > > 35 c3529624 0 0 0 0000204 [IWAIT] irq23: > > 34 c3529830 0 0 0 0000204 [IWAIT] irq22: fxp0 > > 33 c3529a3c 0 0 0 0000204 [IWAIT] irq21: > > 32 c3529c48 0 0 0 0000204 [IWAIT] irq20: > > 31 c353b000 0 0 0 0000204 [IWAIT] irq19: > > 30 c353b20c 0 0 0 0000204 [IWAIT] irq18: fxp1 > > 29 c353b418 0 0 0 0000204 [IWAIT] irq17: bge0 > > 28 c353b624 0 0 0 0000204 [IWAIT] irq16: fxp2 > > 27 c353b830 0 0 0 0000204 [IWAIT] irq15: ata1 > > 26 c353ba3c 0 0 0 0000204 [IWAIT] irq14: ata0 > > 25 c34dc20c 0 0 0 0000204 [IWAIT] irq13: > > 24 c34dc418 0 0 0 0000204 [IWAIT] irq12: > > 23 c34dc624 0 0 0 0000204 [IWAIT] irq11: > > 22 c34dc830 0 0 0 0000204 [IWAIT] irq10: > > 21 c34dca3c 0 0 0 0000204 [IWAIT] irq9: acpi0 > > 20 c34dcc48 0 0 0 0000204 [IWAIT] irq8: > > 19 c3529000 0 0 0 0000204 [IWAIT] irq7: ppc0 > > 18 c352920c 0 0 0 0000204 [IWAIT] irq6: fdc0 > > 17 c3529418 0 0 0 0000204 [IWAIT] irq5: > > 16 c34d7000 0 0 0 0000204 [IWAIT] irq4: sio0 > > 15 c34d720c 0 0 0 0000204 [IWAIT] irq3: sio1 > > 14 c34d7418 0 0 0 0000204 [IWAIT] irq2: > > 13 c34d7624 0 0 0 0000204 [IWAIT] irq1: atkbd0 > > 12 c34d7830 0 0 0 000020c [CPU 0] idle: cpu0 > > 11 c34d7a3c 0 0 0 000020c [Can run] idle: cpu1 > > 1 c34d7c48 0 0 1 0004200 [SLPQ wait 0xc34d7c48][SLP] init > > 10 c34dc000 0 0 0 0000204 [SLPQ ktrace 0xc0938e78][SLP] ktrace > > 0 c0938520 0 0 0 0000200 [IWAIT] swapper > > db> panic > > panic: from debugger > > cpuid = 1 > > KDB: stack backtrace: > > kdb_backtrace(100,c3c34c00,0,c089e89c,c089e8b8) at 0xc0659ab5 = > > kdb_backtrace+0x29 panic(c084a8b4,f1325a04,c0468c70,c093ace0,0) at > > 0xc06408b8 = panic+0x114 db_panic(c093ace0,0,ffffffff,f1325978,0) at > > 0xc0468cd9 = db_panic+0xd > > db_command(c09217e4,c08a7820,c089e89c,c089e8b8,c084a8c2) at 0xc0468c70 = > > db_command+0x264 db_command_loop(0,0,f1325a30,f1325a1c,f1325a64) at > > 0xc0468d38 = db_command_loop+0x5c db_trap(3,0,c3c34c00,3,c3c34c00) at > > 0xc046a949 = db_trap+0xdd > > kdb_trap(3,0,f1325abc) at 0xc0659dd0 = kdb_trap+0x88 > > trap(f1320008,c0650028,c0870028,f9,c372ec00) at 0xc0817c88 = trap+0x498 > > calltrap() at 0xc0804cba = calltrap+0x5 > > --- trap 0x3, eip = 0xc0659b37, esp = 0xf1325afc, ebp = 0xf1325afc --- > > kdb_enter(c0895a5e) at 0xc0659b37 = kdb_enter+0x2b > > siointr1(c372ec00) at 0xc07f09b2 = siointr1+0xce > > siointr(c372ec00) at 0xc07f0792 = siointr+0x5e > > intr_execute_handlers(c3528090,f1325b68,4,f1325bcc,c0805073) at > > 0xc0808985 = intr_execute_handlers+0x85 lapic_handle_intr(34) at > > 0xc080b042 = lapic_handle_intr+0x2e > > Xapic_isr1() at 0xc0805073 = Xapic_isr1+0x33 > > --- interrupt, eip = 0xc0685d59, esp = 0xf1325bac, ebp = 0xf1325bcc --- > > unp_gc(c43be2c8) at 0xc0685d59 = unp_gc+0xf1 > > unp_detach(c43b7000,f1325c08,c067a188,c43be2c8,0) at 0xc06840fa = > > unp_detach+0xae uipc_detach(c43be2c8) at 0xc0682f8e = uipc_detach+0x82 > > soclose(c43be2c8,c40fb6c0,0,f1325c34,c0620c10) at 0xc067a188 = > > soclose+0x278 soo_close(c40fb6c0,c3c34c00) at 0xc0669fb3 = soo_close+0x63 > > fdrop_locked(c40fb6c0,c3c34c00,c3a3d800,f1325cb4,c061f0fb) at 0xc0620c10 > > = fdrop_locked+0xac fdrop(c40fb6c0,c3c34c00,c066371f,c40fb6c0,c878ac00) > > at 0xc0620b5d = fdrop+0x41 closef(c40fb6c0,c3c34c00,0,c3c34c00,6) at > > 0xc061f0fb = closef+0x427 close(c3c34c00,f1325d04,1,0,292) at 0xc061c4bd > > = close+0x211 > > syscall(3b,1003b,bfbf003b,80d58e7,80e8780) at 0xc0818557 = syscall+0x2bf > > Xint0x80_syscall() at 0xc0804d0f = Xint0x80_syscall+0x1f > > --- syscall (6, FreeBSD ELF32, close), eip = 0x282cb7d3, esp = > > 0xbfbfae4c, ebp = 0xbfbfae68 --- KDB: stack backtrace: > > kdb_backtrace(28d5a61e,53755e05,ffc03014,c3c34c00,c084a8b4) at 0xc0659ab5 > > = kdb_backtrace+0x29 mi_switch(1,0,f1325924,c0640190,c3c34c00) at > > 0xc0647162 = mi_switch+0xf6 sched_bind(c3c34c00,0) at 0xc065287c = > > sched_bind+0x44 > > boot(104,104,c3c34c00,0,c089e89c) at 0xc0640190 = boot+0x64 > > panic(c084a8b4,f1325a04,c0468c70,c093ace0,0) at 0xc064095d = panic+0x1b9 > > db_panic(c093ace0,0,ffffffff,f1325978,0) at 0xc0468cd9 = db_panic+0xd > > db_command(c09217e4,c08a7820,c089e89c,c089e8b8,c084a8c2) at 0xc0468c70 = > > db_command+0x264 db_command_loop(0,0,f1325a30,f1325a1c,f1325a64) at > > 0xc0468d38 = db_command_loop+0x5c db_trap(3,0,c3c34c00,3,c3c34c00) at > > 0xc046a949 = db_trap+0xdd > > kdb_trap(3,0,f1325abc) at 0xc0659dd0 = kdb_trap+0x88 > > trap(f1320008,c0650028,c0870028,f9,c372ec00) at 0xc0817c88 = trap+0x498 > > calltrap() at 0xc0804cba = calltrap+0x5 > > --- trap 0x3, eip = 0xc0659b37, esp = 0xf1325afc, ebp = 0xf1325afc --- > > kdb_enter(c0895a5e) at 0xc0659b37 = kdb_enter+0x2b > > siointr1(c372ec00) at 0xc07f09b2 = siointr1+0xce > > siointr(c372ec00) at 0xc07f0792 = siointr+0x5e > > intr_execute_handlers(c3528090,f1325b68,4,f1325bcc,c0805073) at > > 0xc0808985 = intr_execute_handlers+0x85 lapic_handle_intr(34) at > > 0xc080b042 = lapic_handle_intr+0x2e > > Xapic_isr1() at 0xc0805073 = Xapic_isr1+0x33 > > --- interrupt, eip = 0xc0685d59, esp = 0xf1325bac, ebp = 0xf1325bcc --- > > unp_gc(c43be2c8) at 0xc0685d59 = unp_gc+0xf1 > > unp_detach(c43b7000,f1325c08,c067a188,c43be2c8,0) at 0xc06840fa = > > unp_detach+0xae uipc_detach(c43be2c8) at 0xc0682f8e = uipc_detach+0x82 > > soclose(c43be2c8,c40fb6c0,0,f1325c34,c0620c10) at 0xc067a188 = > > soclose+0x278 soo_close(c40fb6c0,c3c34c00) at 0xc0669fb3 = soo_close+0x63 > > fdrop_locked(c40fb6c0,c3c34c00,c3a3d800,f1325cb4,c061f0fb) at 0xc0620c10 > > = fdrop_locked+0xac fdrop(c40fb6c0,c3c34c00,c066371f,c40fb6c0,c878ac00) > > at 0xc0620b5d = fdrop+0x41 closef(c40fb6c0,c3c34c00,0,c3c34c00,6) at > > 0xc061f0fb = closef+0x427 close(c3c34c00,f1325d04,1,0,292) at 0xc061c4bd > > = close+0x211 > > syscall(3b,1003b,bfbf003b,80d58e7,80e8780) at 0xc0818557 = syscall+0x2bf > > Xint0x80_syscall() at 0xc0804d0f = Xint0x80_syscall+0x1f > > --- syscall (6, FreeBSD ELF32, close), eip = 0x282cb7d3, esp = > > 0xbfbfae4c, ebp = 0xbfbfae68 --- [Previous attempts at using "call > > boot(0)"/etc. produced a solid hang that required physical access to the > > Reset button / power switch.] > > db> call cpu_reset() > > cpu_reset: Restarting BSP > > cpu_reset_proxy: Stopped CPU 1 > > > > > > [Seeing other posts that request kernel config additions to assist with > > debugging, I then tried to install a kernel with the added config of: > > options WITNESS > > and booting failed as such:] > > > > /boot/kernel/acpi.ko text=0x40c0c data=0x2160+0x1090 > > syms=[0x4+0x7810+0x4+0xa292] KDB: debugger backends: ddb > > KDB: current backend: ddb > > Copyright (c) 1992-2005 The FreeBSD Project. > > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > > The Regents of the University of California. All rights reserved. > > FreeBSD 6.0-RC1 #1: Wed Oct 19 09:45:23 CDT 2005 > > root@[host-elided]:/usr/obj/usr/src/sys/SMP_DDB > > WARNING: WITNESS option enabled, expect reduced performance. > > Timecounter \"i8254\" frequency 1193182 Hz quality 0 > > CPU: Intel Pentium III (997.46-MHz 686-class CPU) > > Origin = \"GenuineIntel\" Id = 0x68a Stepping = 10 > > > > Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,M > >CA,CMOV,PAT,PSE36,MMX,FXSR,SSE> real memory = 2147418112 (2047 MB) > > avail memory = 2096447488 (1999 MB) > > ACPI APIC Table: <DELL PE2550 > > > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > > cpu0 (BSP): APIC ID: 1 > > cpu1 (AP): APIC ID: 0 > > ioapic0: Changing APIC ID to 2 > > ioapic1: Changing APIC ID to 3 > > MADT: Forcing active-low polarity and level trigger for SCI > > ioapic0 <Version 1.1> irqs 0-15 on motherboard > > ioapic1 <Version 1.1> irqs 16-31 on motherboard > > panic: spin lock hptlock not in order list > > cpuid = 0 > > KDB: stack backtrace: > > kdb_backtrace(100,c0903380,0,c085e142,c08a28ac) at 0xc0646395 = > > kdb_backtrace+0x29 panic(c084621f,c085e142,c090c6c0,0,c0845c3a) at > > 0xc062fd4c = panic+0x114 enroll(c085e142,c08a28ac,c096d720,1,c0c20d5c) at > > 0xc064f61b = enroll+0xef > > witness_init(c096d720,c34bfb74,c08dd510,c28000,c0c20d74) at 0xc064e6dd = > > witness_init+0x11d mtx_init(c096d720,c085e142,0,1,c0c20d88) at 0xc0628d63 > > = mtx_init+0x8f hpt_init(0,c1ec00,c1e000,0,c04457e5) at 0xc07ac352 = > > hpt_init+0x16 mi_startup() at 0xc060b1be = mi_startup+0x96 > > begin() at 0xc04457e5 = begin+0x2c > > KDB: enter: panic > > [thread pid 0 tid 0 ] > > Stopped at 0xc0646417 = kdb_enter+0x2b: nop > > db> > > db> where > > Tracing pid 0 tid 0 td 0xc0903380 > > kdb_enter(c0843318) at 0xc0646417 = kdb_enter+0x2b > > panic(c084621f,c085e142,c090c6c0,0,c0845c3a) at 0xc062fd5f = panic+0x127 > > enroll(c085e142,c08a28ac,c096d720,1,c0c20d5c) at 0xc064f61b = enroll+0xef > > witness_init(c096d720,c34bfb74,c08dd510,c28000,c0c20d74) at 0xc064e6dd = > > witness_init+0x11d mtx_init(c096d720,c085e142,0,1,c0c20d88) at 0xc0628d63 > > = mtx_init+0x8f hpt_init(0,c1ec00,c1e000,0,c04457e5) at 0xc07ac352 = > > hpt_init+0x16 mi_startup() at 0xc060b1be = mi_startup+0x96 > > begin() at 0xc04457e5 = begin+0x2c > > db> show pcpu > > cpuid = 0 > > curthread = 0xc0903380: pid 0 \"swapper\" > > curpcb = 0xc0c20d90 > > fpcurthread = none > > idlethread = 0xc34d8900: pid 12 \"idle: cpu0\" > > APIC ID = 1 > > currentldt = 0x50 > > spin locks held: > > db> show pcpu 0 > > cpuid = 0 > > curthread = 0xc0903380: pid 0 \"swapper\" > > curpcb = 0xc0c20d90 > > fpcurthread = none > > idlethread = 0xc34d8900: pid 12 \"idle: cpu0\" > > APIC ID = 1 > > currentldt = 0x50 > > spin locks held: > > db> show pcpu 1 > > cpuid = 1 > > curthread = none > > curpcb = 0 > > fpcurthread = none > > idlethread = 0xc34d8780: pid 11 \"idle: cpu1\" > > APIC ID = 0 > > currentldt = 0x50 > > spin locks held: > > db> show sysctllock > > No such command > > db> print sysctllock > > c0905920 > > db> show reg > > cs 0x20 > > ds 0xc0840028 > > es 0xc0640028 blist_create+0x4 > > fs 0xc0c20008 > > ss 0x28 > > eax 0x12 > > ecx 0xc1033000 > > edx 0 > > ebx 0xc0c20d0c > > esp 0xc0c20ce0 > > ebp 0xc0c20ce0 > > esi 0x1 > > edi 0xc084621f > > eip 0xc0646417 kdb_enter+0x2b > > efl 0x80292 > > dr0 0 > > dr1 0 > > dr2 0 > > dr3 0 > > dr4 0xffff0ff0 > > dr5 0x400 > > dr6 0xffff0ff0 > > dr7 0x400 > > 0xc0646417 = kdb_enter+0x2b: nop > > db> show witness > > Sleep locks: > > 0 module subsystem sx lock -- last acquired @ > > /usr/src/sys/kern/kern_module.c:115 0 kernel environment -- last acquired > > @ /usr/src/sys/kern/subr_hints.c:117 0 user map -- last acquired @ > > /usr/src/sys/vm/vm_map.c:2489 > > 1 system map -- last acquired @ /usr/src/sys/vm/vm_kern.c:295 > > 2 KMAP ENTRY -- last acquired @ /usr/src/sys/vm/uma_core.c:2631 > > 3 UMA zone -- last acquired @ /usr/src/sys/vm/uma_core.c:1898 > > 2 kmem object -- last acquired @ /usr/src/sys/vm/vm_kern.c:397 > > 3 SYSMAPS -- last acquired @ /usr/src/sys/i386/i386/pmap.c:2410 > > 3 vm page queue mutex -- last acquired @ /usr/src/sys/vm/vm_kern.c:404 > > 4 pmap -- last acquired @ /usr/src/sys/i386/i386/pmap.c:1878 > > 4 vnode interlock -- last acquired @ order list:0 > > 5 cdev -- last acquired @ /usr/src/sys/kern/kern_conf.c:62 > > 2 kernel object -- last acquired @ /usr/src/sys/vm/vm_object.c:454 > > 3 SYSMAPS -- (already displayed) > > 3 vm page queue mutex -- (already displayed) > > 2 UMA lock -- last acquired @ /usr/src/sys/vm/uma_core.c:914 > > 3 UMA zone -- (already displayed) > > 0 nfsd_mtx -- last acquired @ order list:0 > > 3 so_snd -- last acquired @ order list:0 > > 4 so_rcv -- last acquired @ order list:0 > > 5 radix node head -- last acquired @ order list:0 > > 6 rtentry -- last acquired @ order list:0 > > 7 ifaddr -- last acquired @ order list:0 > > 5 sellck -- last acquired @ order list:0 > > 0 bpf global lock -- last acquired @ order list:0 > > 1 bpf interface lock -- last acquired @ order list:0 > > 2 bpf cdev lock -- last acquired @ order list:0 > > 0 ddp_list_mtx -- last acquired @ order list:0 > > 1 ddp_mtx -- last acquired @ order list:0 > > 0 slip_mtx -- last acquired @ order list:0 > > 1 slip sc_mtx -- last acquired @ order list:0 > > 0 tcp -- last acquired @ order list:0 > > 1 tcpinp -- last acquired @ order list:0 > > 3 so_snd -- (already displayed) > > 0 udp -- last acquired @ order list:0 > > 1 udpinp -- last acquired @ order list:0 > > 3 so_snd -- (already displayed) > > 0 unp -- last acquired @ order list:0 > > 3 so_snd -- (already displayed) > > 0 in_multi_mtx -- last acquired @ order list:0 > > 1 igmp_mtx -- last acquired @ order list:0 > > 2 if_addr_mtx -- last acquired @ order list:0 > > 0 Giant -- last acquired @ /usr/src/sys/kern/subr_witness.c:513 > > 1 eventhandler list -- last acquired @ > > /usr/src/sys/kern/subr_eventhandler.c:132 1 vm object_list -- last > > acquired @ /usr/src/sys/vm/vm_object.c:229 1 system map -- (already > > displayed) > > 1 kobj -- last acquired @ /usr/src/sys/kern/subr_kobj.c:148 > > 1 eventhandler -- last acquired @ > > /usr/src/sys/kern/subr_eventhandler.c:82 1 filedesc structure -- last > > acquired @ /usr/src/sys/kern/kern_descrip.c:1512 2 accept -- last > > acquired @ order list:0 > > 3 so_snd -- (already displayed) > > 2 pipe mutex -- last acquired @ order list:0 > > 3 sigio lock -- last acquired @ order list:0 > > 4 process group -- last acquired @ /usr/src/sys/kern/kern_fork.c:562 > > 5 process lock -- last acquired @ > > /usr/src/sys/kern/kern_kthread.c:96 6 ktrace -- last acquired @ > > /usr/src/sys/kern/kern_fork.c:590 > > 6 sigacts -- last acquired @ /usr/src/sys/kern/kern_kthread.c:98 > > 6 session -- last acquired @ /usr/src/sys/kern/kern_fork.c:572 > > 7 uidinfo hash -- last acquired @ /usr/src/sys/kern/kern_resource.c:1032 > > 8 sleep mtxpool -- last acquired @ > > /usr/src/sys/kern/kern_resource.c:914 8 uidinfo struct -- last acquired > > @ order list:0 > > 9 allprison -- last acquired @ order list:0 > > 1 ithread -- last acquired @ /usr/src/sys/kern/kern_intr.c:275 > > 1 malloc -- last acquired @ /usr/src/sys/kern/kern_malloc.c:598 > > 1 kernel linker -- last acquired @ /usr/src/sys/kern/kern_linker.c:470 > > 1 ACPI semaphore -- last acquired @ > > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/Osd/OsdSynch.c:301 1 > > TID lock -- last acquired @ /usr/src/sys/kern/subr_unit.c:592 1 standard > > object -- last acquired @ /usr/src/sys/vm/vm_glue.c:290 1 ttylist -- > > last acquired @ /usr/src/sys/kern/tty.c:2865 > > 1 GEOM orphanage -- last acquired @ /usr/src/sys/geom/geom_event.c:300 > > 0 proctree -- last acquired @ /usr/src/sys/kern/kern_fork.c:561 > > 4 process group -- (already displayed) > > 1 allproc -- last acquired @ /usr/src/sys/kern/kern_fork.c:286 > > 5 process lock -- (already displayed) > > Spin locks: > > 0 ap boot -- last acquired @ order list:0 > > 1 rm.mutex_mtx -- last acquired @ order list:0 > > 2 sio -- last acquired @ order list:0 > > 3 cy -- last acquired @ order list:0 > > 4 uart_hwmtx -- last acquired @ order list:0 > > 5 sabtty -- last acquired @ order list:0 > > 6 zstty -- last acquired @ order list:0 > > 7 ng_node -- last acquired @ order list:0 > > 8 ng_worklist -- last acquired @ order list:0 > > 9 taskqueue_fast -- last acquired @ order list:0 > > 10 intr table -- last acquired @ > > /usr/src/sys/i386/i386/intr_machdep.c:89 11 ithread table lock -- > > last acquired @ order list:0 > > 12 sleepq chain -- last acquired @ > > /usr/src/sys/kern/subr_sleepqueue.c:218 13 sched lock -- last > > acquired @ /usr/src/sys/kern/kern_kthread.c:116 14 turnstile chain -- > > last acquired @ order list:0 > > 15 td_contested -- last acquired @ order list:0 > > 16 callout -- last acquired @ /usr/src/sys/kern/kern_timeout.c:439 > > 17 entropy harvest mutex -- last acquired @ order list:0 > > 18 allpmaps -- last acquired @ /usr/src/sys/i386/i386/pmap.c:1429 > > 19 vm page queue free mutex -- last acquired @ > > /usr/src/sys/vm/vm_page.c:787 20 icu -- last acquired @ order > > list:0 > > 21 smp rendezvous -- last acquired @ order list:0 > > 22 tlb -- last acquired @ order list:0 > > 23 clk -- last acquired @ /usr/src/sys/i386/isa/clock.c:260 > > 24 mutex profiling lock -- last acquired @ order list:0 > > 25 kse zombie lock -- last acquired @ order list:0 > > 26 ALD Queue -- last acquired @ order list:0 > > 27 pcicfg -- last acquired @ order list:0 > > 28 NDIS thread lock -- last acquired @ order list:0 > > 29 tw_osl_io_lock -- last acquired @ order list:0 > > 30 tw_osl_q_lock -- last acquired @ order list:0 > > 31 tw_cl_io_lock -- last acquired @ order list:0 > > 32 tw_cl_intr_lock -- last acquired @ order list:0 > > 33 tw_cl_gen_lock -- last acquired @ order list:0 > > Locks which were never acquired: > > LED sx > > LED mtx > > tty > > nfslock > > random reseed > > devd > > bio queue > > GEOM topology > > if_cloners lock > > ifnet > > acpica subsystem lock > > ACPI global lock > > sf_buf > > taskqueue list > > uma object > > p_peers > > struct pargs.ref > > pbuf mutex > > bdone lock > > buffer daemon lock > > needsbuffer lock > > runningbufspace lock > > buf queue lock > > callout_wait_lock > > ACPI embedded controller > > ACPI CPU > > ACPI PCI power methods > > ACPI cmbat > > ACPI generic battery > > ACPI AC adapter > > ACPI thermal zone > > ACPI PCI link > > ACPI root bus > > ACPI PCI bus methods > > ACPI lid > > ACPI power resources > > umtxq_lock > > bounce pages lock > > arc4_mtx > > securelevel mutex lock > > accounting > > knlist lock for lockless objects > > net80211 instances > > rtsock route_cb lock > > Softdep Lock > > domain list > > rawcb > > pfil_head_list lock > > mountlist > > pmc shared lock > > accept_filter_mtx > > protect sysfilt_ops > > kqueue order > > UUID generator mutex lock > > unit# allocation > > Name Cache > > fifo mutex > > so_glabel > > encapmtx > > DEVFS ruleset lock > > fdesc > > filelist lock > > sysctl lock > > phys_pager list > > dev_pager list > > dev_pager create > > swapdev > > swap_pager list > > vm map sleep mutex > > PMAP2 > > vm86 lock > > db> cont > > Uptime: 1s > > Automatic reboot in 15 seconds - press a key on the console to abort > > Rebooting... > > > > > > And, attempts at adding: > > options INVARIANTS > > > > failed to compile with the following: > > > > /usr/src/sys/dev/ata/atapi-cd.c: In function `acd_geom_attach': > > /usr/src/sys/dev/ata/atapi-cd.c:181: warning: implicit declaration of > > function `_sx_assert' /usr/src/sys/dev/ata/atapi-cd.c:181: warning: > > nested extern declaration of `_sx_assert' > > > > > > > > Since I am having a hang that feels the same to me in both RELENG_5 and > > RELENG_6, that's the main thing I would like to look at. I am not very > > conversant at what I will need for kernel debugging (knowing which > > addresses to dump or structures to inspect) to debug this myself, but > > if you can tell me any further commands to issue in DDB or which > > addresses would assist in diagnosing or config options to add, that will > > be no problem and I will be more than happy to assist in any way! > > > > > > Thank you, > > Philip > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to > > "freebsd-current-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" -- John Baldwin <jhb@FreeBSD.org> <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200510191304.28422.jhb>