From owner-freebsd-bugs@FreeBSD.ORG Fri Feb 6 11:33:21 2009 Return-Path: Delivered-To: freebsd-bugs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A707106567D; Fri, 6 Feb 2009 11:33:21 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.154]) by mx1.freebsd.org (Postfix) with ESMTP id D033A8FC12; Fri, 6 Feb 2009 11:33:19 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so406341fgb.35 for ; Fri, 06 Feb 2009 03:33:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:to:cc:subject:references :organization:from:date:in-reply-to:message-id:user-agent :mime-version:content-type; bh=ulbRKCj69qGdgxju7bsm6REDjR9FPBOsqIGeE8qF1Q8=; b=cQ3SS4KRrEZMoA0TT8xoO8akIMMkh/v0Ep2F45Ve0FbgpHEct+m6TCC9d/3J8jcvD/ e2XmJmCcdfwbzHQJb1aZ/8C0uBIdIJpwwC1sLUtti16hDKTu4S16hsCLo+c1wOWEM9TP BXPUNj8378FqPogok18wIkpWJVufOPc5xpLoI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=to:cc:subject:references:organization:from:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=V6wwWLoSa3ZXrQKeKmVdxy9ssa7ARQXBtzNccMFPrInmx/WZ0gY+3m2D/Gi66XZvLJ XYeupxEPR288K8agyecH5fP0hYCGJCV1SLA86IdmJVl9nKli3Lq/EP+lwKp+EWavCQLF peFzqhPgAxICy+vu4VslfbHbU3La3FeMqzEDQ= Received: by 10.86.4.2 with SMTP id 2mr891960fgd.49.1233919998783; Fri, 06 Feb 2009 03:33:18 -0800 (PST) Received: from localhost (ms.singlescrowd.net [80.85.90.67]) by mx.google.com with ESMTPS id 3sm714812fge.22.2009.02.06.03.33.16 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 06 Feb 2009 03:33:17 -0800 (PST) To: Mikolaj Golub References: <200902021314.n12DEwd3098744@freefall.freebsd.org> <817i47oibn.fsf@zhuzha.ua1> <86iqnrb8aa.fsf@kopusha.onet> <81tz78nc5k.fsf@zhuzha.ua1> <8663jo663f.fsf@kopusha.onet> <81myczuafa.fsf@zhuzha.ua1> Organization: TOA Ukraine From: Mikolaj Golub Date: Fri, 06 Feb 2009 13:33:14 +0200 In-Reply-To: <81myczuafa.fsf@zhuzha.ua1> (Mikolaj Golub's message of "Fri\, 06 Feb 2009 12\:28\:09 +0200") Message-ID: <81prhvpzph.fsf@zhuzha.ua1> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Sergey S , remko@freebsd.org, freebsd-bugs@freebsd.org Subject: Re: misc/131290: How to completely freeze FreeBSD 7.1 under a non-privileged user X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Feb 2009 11:33:22 -0000 --=-=-= On Fri, 06 Feb 2009 12:28:09 +0200 Mikolaj Golub wrote: > On Thu, 05 Feb 2009 21:23:16 +0200 Mikolaj Golub wrote: > >> On Thu, 5 Feb 2009 07:51:31 -0800 Sergey S wrote: >> >> SS> Hello. >> >> >> I can't hang the system running many cycles of loop.sh... >> >> SS> :( >> >> SS> I've no idea, what can be done in order to elucidate the problem... >> >> When I went home I left the machine with loop.sh run and now it is not >> pingable, so it looks like it hangs :-) Tomorrow, I will know this for sure. >> >> I noticed from your top output that your system was idle when you did your >> test while mine was rather loaded. That is why, I suppose, it required more >> time for me to hang it. > > Yes, it hanged. I added to the GENERIC kernel config the followin options: > > options INVARIANTS > options INVARIANT_SUPPORT > options WITNESS > options DEBUG_LOCKS > options DEBUG_VFS_LOCKS > options DIAGNOSTIC > > options KDB > options DDB > > and run script on idle system. The system froze on the second cycle run. It > did not panic and I was not able to enter the debugger, so no crush dump is > available. > > Then I changed the scheduler to 4BSD. It hangs with 4BSD too but it looks like > it requires more cycles. When it froze first time with 4BSD I managed to enter > debugger pressing Ctr-ALt-ESC. But I made silly thing, I decided to check if > the system would resume to normal state if I just return from the debugger. It > did not and I was not able to enter ddb again. After this I hanged the system > several times to try to enter ddb but without success :-( But I can make my system panic in another way. When I did tests, before running loop.sh I stopped some of my services to make my system idle. One time when I stopped ejabberd (it is written on erlang if somebody does not know) the system panicked. It is reproducible. Next time it required for me several times to do /usr/local/etc/rc.d/ejabberd start /usr/local/etc/rc.d/ejabberd stop cycles to make the system panic. I don't know if this is related to this thread initial problem, but below is some gdb info for this crash. GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: panic: Bad link elm 0xc4f0c1f0 next->prev != elm cpuid = 0 KDB: enter: panic exclusive sleep mutex sellck r = 0 (0xc0cc7204) locked @ /usr/src/sys/kern/sys_generic.c:1127 exclusive sleep mutex pipe mutex r = 0 (0xc4f0c2fc) locked @ /usr/src/sys/kern/sys_pipe.c:1132 panic: from debugger cpuid = 0 Uptime: 38m7s Physical memory: 1003 MB Dumping 116 MB: 101 85 69 53 37 21 5 Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done. done. Loaded symbols for /boot/kernel/acpi.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel/pf.ko.symbols...done. done. Loaded symbols for /boot/kernel/pf.ko Reading symbols from /boot/kernel/smbfs.ko...Reading symbols from /boot/kernel/smbfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/smbfs.ko Reading symbols from /boot/kernel/libiconv.ko...Reading symbols from /boot/kernel/libiconv.ko.symbols...done. done. Loaded symbols for /boot/kernel/libiconv.ko Reading symbols from /boot/kernel/libmchain.ko...Reading symbols from /boot/kernel/libmchain.ko.symbols...done. done. Loaded symbols for /boot/kernel/libmchain.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/logo_saver.ko...Reading symbols from /boot/kernel/logo_saver.ko.symbols...done. done. Loaded symbols for /boot/kernel/logo_saver.ko Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from /boot/kernel/ng_socket.ko.symbols...done. done. Loaded symbols for /boot/kernel/ng_socket.ko Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /boot/kernel/netgraph.ko.symbols...done. done. Loaded symbols for /boot/kernel/netgraph.ko Reading symbols from /boot/kernel/if_bridge.ko...Reading symbols from /boot/kernel/if_bridge.ko.symbols...done. done. Loaded symbols for /boot/kernel/if_bridge.ko Reading symbols from /boot/kernel/bridgestp.ko...Reading symbols from /boot/kernel/bridgestp.ko.symbols...done. done. Loaded symbols for /boot/kernel/bridgestp.ko #0 doadump () at pcpu.h:196 196 pcpu.h: No such file or directory. in pcpu.h (kgdb) backtrace #0 doadump () at pcpu.h:196 #1 0xc079a07e in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc079a352 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0493a07 in db_panic (addr=Could not find the frame base for "db_panic". ) at /usr/src/sys/ddb/db_command.c:446 #4 0xc049440c in db_command (last_cmdp=0xc0c48114, cmd_table=0x0, dopager=1) at /usr/src/sys/ddb/db_command.c:413 #5 0xc049451a in db_command_loop () at /usr/src/sys/ddb/db_command.c:466 #6 0xc0495d0d in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:228 #7 0xc07c3866 in kdb_trap (type=3, code=0, tf=0xe69ceac8) at /usr/src/sys/kern/subr_kdb.c:524 #8 0xc0a9fb5b in trap (frame=0xe69ceac8) at /usr/src/sys/i386/i386/trap.c:688 #9 0xc0a8541b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #10 0xc07c39ea in kdb_enter_why (why=0xc0b42038 "panic", msg=0xc0b42038 "panic") at cpufunc.h:60 #11 0xc079a336 in panic (fmt=0xc0af1d18 "Bad link elm %p next->prev != elm") at /usr/src/sys/kern/kern_shutdown.c:557 #12 0xc07d4b3b in doselwakeup (sip=0xc4f0c1f0, pri=88) at /usr/src/sys/kern/sys_generic.c:1138 #13 0xc07d4c1e in selwakeuppri (sip=0xc4f0c1f0, pri=88) at /usr/src/sys/kern/sys_generic.c:1114 #14 0xc07d9c8e in pipe_write (fp=0xc42b95f0, uio=0xe69cec60, active_cred=0xc4aa8800, flags=0, td=0xc4bc8aa0) at /usr/src/sys/kern/sys_pipe.c:528 ---Type to continue, or q to quit---#15 0xc07d6095 in dofilewrite (td=0xc4bc8aa0, fd=4, fp=0xc42b95f0, auio=0xe69cec60, offset=-1, flags=0) at file.h:256 #16 0xc07d6318 in kern_writev (td=0xc4bc8aa0, fd=4, auio=0xe69cec60) at /usr/src/sys/kern/sys_generic.c:401 #17 0xc07d638f in write (td=0xc4bc8aa0, uap=0xe69cecfc) at /usr/src/sys/kern/sys_generic.c:317 #18 0xc0a9f2d3 in syscall (frame=0xe69ced38) at /usr/src/sys/i386/i386/trap.c:1090 #19 0xc0a85480 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255 #20 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) list *0xc07d4b3b 0xc07d4b3b is in doselwakeup (/usr/src/sys/kern/sys_generic.c:1138). 1133 } 1134 if (td == NULL) { 1135 mtx_unlock(&sellock); 1136 return; 1137 } 1138 TAILQ_REMOVE(&td->td_selq, sip, si_thrlist); 1139 sip->si_thread = NULL; 1140 thread_lock(td); 1141 td->td_flags &= ~TDF_SELECT; 1142 thread_unlock(td); (kgdb) For next panic I configured ddb to produce textdump. Obtained textdump.tar is attached. Please, let me know if I can provide any other useful info. -- Mikolaj Golub --=-=-=--