Date: Fri, 1 Apr 2005 19:04:29 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: Poul-Henning Kamp <phk@phk.freebsd.dk> Cc: John Baldwin <jhb@FreeBSD.org> Subject: Re: 4BSD Scheduler Problem on 5.3 Message-ID: <Pine.NEB.3.96L.1050401185611.53976D-100000@fledge.watson.org> In-Reply-To: <89197.1112381558@critter.freebsd.dk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 1 Apr 2005, Poul-Henning Kamp wrote: > In message <200504011120.27285.jhb@FreeBSD.org>, John Baldwin writes: > > >> I think we have a race in -STABLE relating to tty wakeups and > >> open/close/device teardown. I've seen a panic relating to sio during a > >> tty close on RELENG_5 about 5-6 months ago, but was unable to get a dump. > >> Scott has since fixed dumps with twe, but I've not yet been able to get > >> the bug to recur. I'll give it another try. > > > >Sounds very plausible. Does Poul-Henning have any ideas? > > Is this before or after my tty changes ? > > There is a general nastyness about ttys/sessions/exit which I have never > really felt comfortable about. My hope is that I have solved it by > refcounting the tty structure. > > So if this is before my changes: "Yeah, known (but rare) issue)" > > If after my changes: "D**N!" The instance of the panic I saw was in RELENG_5 in January when using a serial console. Here's a copy of the e-mail I sent to stable@ when it occurred. It's a little weak on the debugging side because I couldn't get a dump and didn't have a kernel with symbols easily on hand, but my reading was that there was a race relating to open/close and device events. Robert N M Watson Date: Sun, 23 Jan 2005 16:51:14 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: stable@FreeBSD.org Cc: imp@FreeBSD.org Subject: NULL pointer deref in sioopen() suggests a close/open race on sio device? Ran into the following panic on a 5-STABLE box this morning, which occurred after hitting Ctrl-D to close a login session on a serial console (ttyd0 at 9600 bps): login: Jan 23 10:43:27 fledge login: 2 LOGIN FAILURES ON ttyd0 Fatal trap 12: page fault while in kernel mode fault virtual address = 0x1c fault code = supervisor write, page not present instruction pointer = 0x8:0xc051537b stack pointer = 0x10:0xe7345988 frame pointer = 0x10:0xe7345994 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 45092 (getty) [thread pid 45092 tid 100201 ] Stopped at knote+0x27: cmpxchgl %ecx,0x1c(%edx) db> show pcpu cpuid = 0 curthread = 0xc290d190: pid 45092 "getty" curpcb = 0xe7345da0 fpcurthread = 0xc290d190: pid 45092 "getty" idlethread = 0xc22644b0: pid 11 "idle" APIC ID = 0 currentldt = 0x30 db> trace Tracing pid 45092 tid 100201 td 0xc290d190 knote(c264e098,0,0,c290d190,e73459c4) at knote+0x27 ttwwakeup(c264e000) at ttwwakeup+0xc8 comstart(c264e000) at comstart+0x385 comparam(c264e000,c264e0a4,c264e000,3,0) at comparam+0x253 sioopen(c079f060,3,2000,c290d190,c078e6a0) at sioopen+0x1df spec_open(e7345a84,e7345b40,c058d585,e7345a84,180) at spec_open+0x2b6 spec_vnoperate(e7345a84) at spec_vnoperate+0x13 vn_open_cred(e7345be4,e7345ce4,c08,c2261d80,0) at vn_open_cred+0x419 vn_open(e7345be4,e7345ce4,c08,0,c4289b58) at vn_open+0x1e kern_open(c290d190,804f8e0,0,3,bfbfee18) at kern_open+0xe3 open(c290d190,e7345d14,3,0,292) at open+0x18 syscall(2f,2f,2f,804f8e0,0) at syscall+0x27b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (5, FreeBSD ELF32, open), eip = 0x280d155b, esp = 0xbfbfedec, ebp = 0xbfbfee18 --- The ps list is a bit boring, but the primary interesting thing is that it looks like the close was going on in one thread just about when the sio swi was scheduled to run also: db> ps pid proc uarea uid ppid pgrp flag stat wmesg wchan cmd 45092 c6762388 e7387000 0 1 1 0004000 [CPU 0] getty ... 132 c235954c e4fbf000 0 0 0 000020c [RUNQ] swi5: clock sio I didn't have a kernel with debugging symbols on-hand, but the above address in knote() is a cmpxchg early in the function, which means it's likely the conditional call to mtx_lock() hitting a NULL mutex pointer for kl_lock. This in turn suggests that something has called ttyrel/tty_close on the TTY in a race with the open, or otherwise NULL'd that pointer via knlist_destroy(). Anyone have any pointers on this one? The TTY code is not my forte... Robert N M Watson
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.96L.1050401185611.53976D-100000>