Date: Thu, 01 Mar 2001 12:18:44 -0800 (PST) From: John Baldwin <jhb@FreeBSD.org> To: The Hermit Hacker <scrappy@hub.org> Cc: freebsd-current@FreeBSD.org Subject: RE: System hangs with -current ... Message-ID: <XFMail.010301121844.jhb@FreeBSD.org> In-Reply-To: <Pine.BSF.4.33.0103011400340.88613-100000@mobile.hub.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 01-Mar-01 The Hermit Hacker wrote: > > any comments on this? any way of doing this without a serial console? > > thanks ... The data is too much to make a normal console feasible, although you could try cranking up the console to hte highest res (80x60 or 132x60, etc.) you can and let it freeze and then write down those 60 lines adn maybe that will be enough to figure it out. However, if its looping this won't work. :( I've no idea atm why the serial console isn't working for you. > On Wed, 28 Feb 2001, The Hermit Hacker wrote: > >> >> Yup, definitely doesn't like me using the console ... just tried it again, >> and its as if it can't scroll up the screen to send more data or >> something? >> >> I just rebooted, and then ssh'd in from remote ... type'd the two sysctl >> commands, and got: >> >> cpu1 ../../i386/i386/trap.c.181 GOT (spin) sched lock [0xc0320f20] r=0 at >> ../../i386/i386/trap.c:181 >> cpcsocp/../i386/i386/trap.c.217 REL (spin) sched l >> >> on my screen ... type'd exactly as seen ... and that's it ... console is >> now locked again ... >> >> On Tue, 27 Feb 2001, The Hermit Hacker wrote: >> >> > >> > Okay, can't seem to find a 9pin->9pin NULL modem cable in this 'pit of the >> > earth' town, so figured I'd do the sysctl commands on my console and use >> > an ssh connection into the machine to run the 'hanging sequence' ... the >> > console flashed a bunch of 'debugging info' and then hung solid ... I >> > could still login remotely and whatnot, type commands, just nothing was >> > happening on the console, couldn't change vty's, nothing ... >> > >> > is it supposed to do that? *raised eyebrow* >> > >> > On Thu, 22 Feb 2001, John Baldwin wrote: >> > >> > > >> > > On 23-Feb-01 The Hermit Hacker wrote: >> > > > On Thu, 22 Feb 2001, John Baldwin wrote: >> > > > >> > > >> >> > > >> On 22-Feb-01 The Hermit Hacker wrote: >> > > >> > >> > > >> > Okay, I have to pick up a NULL modem cable tomorrow and dive into >> > > >> > this ... >> > > >> > finally ... >> > > >> > >> > > >> > The various KTR_ that you mention below, these are kernel settings >> > > >> > that I >> > > >> > compile into the kernel? >> > > >> >> > > >> Yes. You want this: >> > > >> >> > > >> options KTR >> > > >> options KTR_EXTEND >> > > >> options KTR_COMPILE=0x1208 >> > > > >> > > > okay, just so that I understand ... I compile my kernel with these >> > > > options, and then run the two sysctl commands you list below? the >> > > > KTR_COMPILE arg looks similar to the ktr_mask one below, which is why >> > > > I'm >> > > > confirming ... >> > > >> > > Yes. KTR_COMPILE controls what KTR tracepoints are actually compiled >> > > into >> > > the kernel. The ktr_mask sysctl controls a runtime mask that lets you >> > > choose >> > > which of the compiled in masks you want to enable. I have manpages for >> > > this >> > > stuff, but they are waiting for doc guys to review them. >> > > >> > > >> The mtx_quiet.patch is old and won't apply to current now I'm afraid. >> > > >> >> > > >> > On Tue, 2 Jan 2001, John Baldwin wrote: >> > > >> > >> > > >> >> >> > > >> >> On 02-Jan-01 The Hermit Hacker wrote: >> > > >> >> > >> > > >> >> > Over the past several months, as others have reported, I've been >> > > >> >> > getting >> > > >> >> > system hangs using 5.0-CURRENT w/ SMP ... I've got DDB enabled, >> > > >> >> > but >> > > >> >> > ctl-alt-esc doesn't break me to the debugger ... >> > > >> >> > >> > > >> >> > I'm not complaining about the hangs, if I was overly concerned, >> > > >> >> > I'd run >> > > >> >> > -STABLE, but I'm wondering how one goes about providing debug >> > > >> >> > information >> > > >> >> > on them other then through DDB? >> > > >> >> >> > > >> >> Not easily. :( If you can make the problem easily repeatable, >> > > >> >> then you >> > > >> >> can >> > > >> >> try >> > > >> >> turning on KTR in your kernel (see NOTES, you will need >> > > >> >> KTR_EXTEND), >> > > >> >> setting >> > > >> >> up >> > > >> >> a serial console that you log the output of, create a shell script >> > > >> >> that >> > > >> >> runs >> > > >> >> the following commands: >> > > >> >> >> > > >> >> #!/bin/sh >> > > >> >> >> > > >> >> # Turn on KTR_INTR, KTR_PROC, and KTR_LOCK >> > > >> >> sysctl -w debug.ktr_mask=0x1208 >> > > >> >> sysctl -w debug.ktr_verbose=2 >> > > >> >> >> > > >> >> run_magic_command_that_hangs_my_machine >> > > >> >> >> > > >> >> and run the script. You probably want to run it over a tty or >> > > >> >> remote >> > > >> >> login >> > > >> >> so >> > > >> >> tthat the serial console output is just the logging (warning, it >> > > >> >> will be >> > > >> >> very >> > > >> >> verbose!). Also, you probably want to use >> > > >> >> http://www.FreeBSD.org/~jhb/patches/mtx_quiet.patch to shut up >> > > >> >> most of >> > > >> >> the >> > > >> >> irrelevant and cluttery mutex trace messages. Note that having >> > > >> >> this much >> > > >> >> logging on will probably slow the machine to a crawl as well, so >> > > >> >> you may >> > > >> >> have >> > > >> >> to just start this up and go off and do something else until it >> > > >> >> hangs. >> > > >> >> :-/ >> > > >> >> Another alternative is to rig up a NMI debouncer and use it to >> > > >> >> break into >> > > >> >> the >> > > >> >> debugger. Then you can start poking around to see who owns >> > > >> >> sched_lock, >> > > >> >> etc. >> > > >> >> >> > > >> >> > Thanks ... >> > > >> > > -- >> > > >> > > John Baldwin <jhb@FreeBSD.org> -- http://www.FreeBSD.org/~jhb/ >> > > PGP Key: http://www.baldwin.cx/~john/pgpkey.asc >> > > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ >> > > >> > >> > Marc G. Fournier ICQ#7615664 IRC Nick: >> > Scrappy >> > Systems Administrator @ hub.org >> > primary: scrappy@hub.org secondary: >> > scrappy@{freebsd|postgresql}.org >> > >> > >> > To Unsubscribe: send mail to majordomo@FreeBSD.org >> > with "unsubscribe freebsd-current" in the body of the message >> > >> >> Marc G. Fournier ICQ#7615664 IRC Nick: >> Scrappy >> Systems Administrator @ hub.org >> primary: scrappy@hub.org secondary: >> scrappy@{freebsd|postgresql}.org >> >> >> >> To Unsubscribe: send mail to majordomo@FreeBSD.org >> with "unsubscribe freebsd-current" in the body of the message >> > > Marc G. Fournier ICQ#7615664 IRC Nick: > Scrappy > Systems Administrator @ hub.org > primary: scrappy@hub.org secondary: > scrappy@{freebsd|postgresql}.org > -- John Baldwin <jhb@FreeBSD.org> -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?XFMail.010301121844.jhb>