Date: Fri, 2 Mar 2001 11:17:37 -0400 (AST) From: The Hermit Hacker <scrappy@hub.org> To: John Baldwin <jhb@FreeBSD.org> Cc: <freebsd-current@FreeBSD.org> Subject: RE: System hangs with -current ... Message-ID: <Pine.BSF.4.33.0103021116500.88613-100000@mobile.hub.org> In-Reply-To: <XFMail.010301121844.jhb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 1 Mar 2001, John Baldwin wrote: > > On 01-Mar-01 The Hermit Hacker wrote: > > > > any comments on this? any way of doing this without a serial console? > > > > thanks ... > > The data is too much to make a normal console feasible, although you > could try cranking up the console to hte highest res (80x60 or 132x60, > etc.) you can and let it freeze and then write down those 60 lines adn > maybe that will be enough to figure it out. However, if its looping > this won't work. :( I've no idea atm why the serial console isn't > working for you. Inability to actually find a NULL modem cable, actually :( Checked two local shops, and neither of them carry one ... just hijacked one from work for the weekend, so will hit this tonight and report anything I can come up with ... > > > On Wed, 28 Feb 2001, The Hermit Hacker wrote: > > > >> > >> Yup, definitely doesn't like me using the console ... just tried it again, > >> and its as if it can't scroll up the screen to send more data or > >> something? > >> > >> I just rebooted, and then ssh'd in from remote ... type'd the two sysctl > >> commands, and got: > >> > >> cpu1 ../../i386/i386/trap.c.181 GOT (spin) sched lock [0xc0320f20] r=0 at > >> ../../i386/i386/trap.c:181 > >> cpcsocp/../i386/i386/trap.c.217 REL (spin) sched l > >> > >> on my screen ... type'd exactly as seen ... and that's it ... console is > >> now locked again ... > >> > >> On Tue, 27 Feb 2001, The Hermit Hacker wrote: > >> > >> > > >> > Okay, can't seem to find a 9pin->9pin NULL modem cable in this 'pit of the > >> > earth' town, so figured I'd do the sysctl commands on my console and use > >> > an ssh connection into the machine to run the 'hanging sequence' ... the > >> > console flashed a bunch of 'debugging info' and then hung solid ... I > >> > could still login remotely and whatnot, type commands, just nothing was > >> > happening on the console, couldn't change vty's, nothing ... > >> > > >> > is it supposed to do that? *raised eyebrow* > >> > > >> > On Thu, 22 Feb 2001, John Baldwin wrote: > >> > > >> > > > >> > > On 23-Feb-01 The Hermit Hacker wrote: > >> > > > On Thu, 22 Feb 2001, John Baldwin wrote: > >> > > > > >> > > >> > >> > > >> On 22-Feb-01 The Hermit Hacker wrote: > >> > > >> > > >> > > >> > Okay, I have to pick up a NULL modem cable tomorrow and dive into > >> > > >> > this ... > >> > > >> > finally ... > >> > > >> > > >> > > >> > The various KTR_ that you mention below, these are kernel settings > >> > > >> > that I > >> > > >> > compile into the kernel? > >> > > >> > >> > > >> Yes. You want this: > >> > > >> > >> > > >> options KTR > >> > > >> options KTR_EXTEND > >> > > >> options KTR_COMPILE=0x1208 > >> > > > > >> > > > okay, just so that I understand ... I compile my kernel with these > >> > > > options, and then run the two sysctl commands you list below? the > >> > > > KTR_COMPILE arg looks similar to the ktr_mask one below, which is why > >> > > > I'm > >> > > > confirming ... > >> > > > >> > > Yes. KTR_COMPILE controls what KTR tracepoints are actually compiled > >> > > into > >> > > the kernel. The ktr_mask sysctl controls a runtime mask that lets you > >> > > choose > >> > > which of the compiled in masks you want to enable. I have manpages for > >> > > this > >> > > stuff, but they are waiting for doc guys to review them. > >> > > > >> > > >> The mtx_quiet.patch is old and won't apply to current now I'm afraid. > >> > > >> > >> > > >> > On Tue, 2 Jan 2001, John Baldwin wrote: > >> > > >> > > >> > > >> >> > >> > > >> >> On 02-Jan-01 The Hermit Hacker wrote: > >> > > >> >> > > >> > > >> >> > Over the past several months, as others have reported, I've been > >> > > >> >> > getting > >> > > >> >> > system hangs using 5.0-CURRENT w/ SMP ... I've got DDB enabled, > >> > > >> >> > but > >> > > >> >> > ctl-alt-esc doesn't break me to the debugger ... > >> > > >> >> > > >> > > >> >> > I'm not complaining about the hangs, if I was overly concerned, > >> > > >> >> > I'd run > >> > > >> >> > -STABLE, but I'm wondering how one goes about providing debug > >> > > >> >> > information > >> > > >> >> > on them other then through DDB? > >> > > >> >> > >> > > >> >> Not easily. :( If you can make the problem easily repeatable, > >> > > >> >> then you > >> > > >> >> can > >> > > >> >> try > >> > > >> >> turning on KTR in your kernel (see NOTES, you will need > >> > > >> >> KTR_EXTEND), > >> > > >> >> setting > >> > > >> >> up > >> > > >> >> a serial console that you log the output of, create a shell script > >> > > >> >> that > >> > > >> >> runs > >> > > >> >> the following commands: > >> > > >> >> > >> > > >> >> #!/bin/sh > >> > > >> >> > >> > > >> >> # Turn on KTR_INTR, KTR_PROC, and KTR_LOCK > >> > > >> >> sysctl -w debug.ktr_mask=0x1208 > >> > > >> >> sysctl -w debug.ktr_verbose=2 > >> > > >> >> > >> > > >> >> run_magic_command_that_hangs_my_machine > >> > > >> >> > >> > > >> >> and run the script. You probably want to run it over a tty or > >> > > >> >> remote > >> > > >> >> login > >> > > >> >> so > >> > > >> >> tthat the serial console output is just the logging (warning, it > >> > > >> >> will be > >> > > >> >> very > >> > > >> >> verbose!). Also, you probably want to use > >> > > >> >> http://www.FreeBSD.org/~jhb/patches/mtx_quiet.patch to shut up > >> > > >> >> most of > >> > > >> >> the > >> > > >> >> irrelevant and cluttery mutex trace messages. Note that having > >> > > >> >> this much > >> > > >> >> logging on will probably slow the machine to a crawl as well, so > >> > > >> >> you may > >> > > >> >> have > >> > > >> >> to just start this up and go off and do something else until it > >> > > >> >> hangs. > >> > > >> >> :-/ > >> > > >> >> Another alternative is to rig up a NMI debouncer and use it to > >> > > >> >> break into > >> > > >> >> the > >> > > >> >> debugger. Then you can start poking around to see who owns > >> > > >> >> sched_lock, > >> > > >> >> etc. > >> > > >> >> > >> > > >> >> > Thanks ... > >> > > > >> > > -- > >> > > > >> > > John Baldwin <jhb@FreeBSD.org> -- http://www.FreeBSD.org/~jhb/ > >> > > PGP Key: http://www.baldwin.cx/~john/pgpkey.asc > >> > > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > >> > > > >> > > >> > Marc G. Fournier ICQ#7615664 IRC Nick: > >> > Scrappy > >> > Systems Administrator @ hub.org > >> > primary: scrappy@hub.org secondary: > >> > scrappy@{freebsd|postgresql}.org > >> > > >> > > >> > To Unsubscribe: send mail to majordomo@FreeBSD.org > >> > with "unsubscribe freebsd-current" in the body of the message > >> > > >> > >> Marc G. Fournier ICQ#7615664 IRC Nick: > >> Scrappy > >> Systems Administrator @ hub.org > >> primary: scrappy@hub.org secondary: > >> scrappy@{freebsd|postgresql}.org > >> > >> > >> > >> To Unsubscribe: send mail to majordomo@FreeBSD.org > >> with "unsubscribe freebsd-current" in the body of the message > >> > > > > Marc G. Fournier ICQ#7615664 IRC Nick: > > Scrappy > > Systems Administrator @ hub.org > > primary: scrappy@hub.org secondary: > > scrappy@{freebsd|postgresql}.org > > > > -- > > John Baldwin <jhb@FreeBSD.org> -- http://www.FreeBSD.org/~jhb/ > PGP Key: http://www.baldwin.cx/~john/pgpkey.asc > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy Systems Administrator @ hub.org primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.33.0103021116500.88613-100000>