From owner-freebsd-bugs@FreeBSD.ORG Fri Aug 1 20:20:15 2003 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6128337B401 for ; Fri, 1 Aug 2003 20:20:15 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D36443FCB for ; Fri, 1 Aug 2003 20:20:14 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h723KEUp021979 for ; Fri, 1 Aug 2003 20:20:14 -0700 (PDT) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h723KE2Q021978; Fri, 1 Aug 2003 20:20:14 -0700 (PDT) Resent-Date: Fri, 1 Aug 2003 20:20:14 -0700 (PDT) Resent-Message-Id: <200308020320.h723KE2Q021978@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Kris Kennaway Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4927F37B401 for ; Fri, 1 Aug 2003 20:16:05 -0700 (PDT) Received: from obsecurity.dyndns.org (adsl-63-207-60-135.dsl.lsan03.pacbell.net [63.207.60.135]) by mx1.FreeBSD.org (Postfix) with ESMTP id C47BE43F93 for ; Fri, 1 Aug 2003 20:15:59 -0700 (PDT) (envelope-from kris@obsecurity.org) Received: from enigma.obsecurity.org (enigma [10.0.0.4]) by obsecurity.dyndns.org (Postfix) with ESMTP id BE75466BE5 for ; Fri, 1 Aug 2003 20:15:52 -0700 (PDT) Received: by enigma.obsecurity.org (Postfix, from userid 1000) id 63DD21A69; Fri, 1 Aug 2003 14:50:51 -0700 (PDT) Message-Id: <20030801215051.63DD21A69@enigma.obsecurity.org> Date: Fri, 1 Aug 2003 14:50:51 -0700 (PDT) From: Kris Kennaway To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Subject: kern/55175: LOR in select and poll X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Kris Kennaway List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 03:20:15 -0000 >Number: 55175 >Category: kern >Synopsis: LOR in select and poll >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Aug 01 20:20:13 PDT 2003 >Closed-Date: >Last-Modified: >Originator: Kris Kennaway >Release: FreeBSD 5.1-CURRENT >Organization: FreeBSD >Environment: System: FreeBSD enigma.obsecurity.org 5.1-CURRENT FreeBSD 5.1-CURRENT #0: Sat Jul 19 18:37:43 PDT 2003 kkenn@enigma.obsecurity.org:/usr/src/sys/sparc64/compile/ENIGMA sparc64 >Description: Since upgrading the bento i386 package build machines to 5.1-CURRENT, the following two lock order reversals have been seen. lock order reversal 1st 0xc6c1c334 filedesc structure (filedesc structure) @ +/a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:902 2nd 0xc04aa120 Giant (Giant) @ /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372 Stack backtrace: backtrace(c043d4af,c04aa120,c0439aa4,c0439aa4,c0434e3d) at backtrace+0x17 witness_lock(c04aa120,8,c0434e3d,174,1bc) at witness_lock+0x672 _mtx_lock_flags(c04aa120,0,c0434e3d,174,c043daba) at _mtx_lock_flags+0xba spec_poll(d8dddaf8,d8dddb18,c02d119c,d8dddaf8,c04939a0) at spec_poll+0x134 spec_vnoperate(d8dddaf8,c04939a0,c520b124,40,c675e300) at spec_vnoperate+0x18 vn_poll(c44c5e14,40,c675e300,c6222d10,c675e300) at vn_poll+0x3c selscan(c6222d10,d8dddb98,d8dddb88,6,4) at selscan+0x13e kern_select(c6222d10,6,bfbff5c0,0,0) at kern_select+0x36f select(c6222d10,d8dddd10,c0455899,3ee,5) at select+0x66 syscall(2f,2f,2f,8055050,bfbff5b8) at syscall+0x273 Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (93), eip = 0x280ccacc, esp = 0x2832eb68, ebp = 0x2832ebc0 --- Debugger("witness_lock") Stopped at Debugger+0x54: xchgl %ebx,in_Debugger.0 #8 0xc0290ed7 in witness_lock (lock=0xc04aa120, flags=8, file=0xc0434e3d "/a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c", line=372) at /a/asami/portbuild/i386/src-client/sys/kern/subr_witness.c:838 #9 0xc0261f4a in _mtx_lock_flags (m=0x0, opts=0, file=0xc04d17a8 "", line=-1068850912) at /a/asami/portbuild/i386/src-client/sys/kern/kern_mutex.c:334 #10 0xc0231154 in spec_poll (ap=0xd8dddaf8) at /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372 #11 0xc0230648 in spec_vnoperate (ap=0x0) at /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:122 #12 0xc02d119c in vn_poll (fp=0x0, events=0, active_cred=0xc675e300, td=0x0) at vnode_if.h:537 #13 0xc02945ae in selscan (td=0xc6222d10, ibits=0xd8dddb98, obits=0xd8dddb88, nfd=6) at /a/asami/portbuild/i386/src-client/sys/sys/file.h:272 #14 0xc029412f in kern_select (td=0xc6222d10, nd=6, fd_in=0xbfbff5c0, fd_ou=0x0, fd_ex=0x0, tvp=0xd8dddcd4) at /a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:822 #15 0xc0293da6 in select (td=0x0, uap=0xd8dddd10) at /a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:726 #16 0xc03ef9b3 in syscall (frame= {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 134565968, tf_esi = -1077938760, tf_ebp = 674425792, +tf_isp = -656548492, tf_ebx = 0, tf_edx = -1077938752, tf_ecx = 0, tf_eax = 93, tf_trapno = 12, tf_err = 2, +tf_eip = 671926988, tf_cs = 31, tf_eflags = 534, tf_esp = 674425704, tf_ss = 47}) at /a/asami/portbuild/i386/src-client/sys/i386/i386/trap.c:1008 #17 0xc03dfbed in Xint0x80_syscall () at {standard input}:144 ---Can't read userspace from dump, or kernel process--- lock order reversal 1st 0xc6a69634 filedesc structure (filedesc structure) @ +/a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:1071 2nd 0xc04aa120 Giant (Giant) @ /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372 Stack backtrace: backtrace(c043d4af,c04aa120,c0439aa4,c0439aa4,c0434e3d) at backtrace+0x17 witness_lock(c04aa120,8,c0434e3d,174,246) at witness_lock+0x672 _mtx_lock_flags(c04aa120,0,c0434e3d,174,c043daba) at _mtx_lock_flags+0xba spec_poll(d8dfcb44,d8dfcb64,c02d119c,d8dfcb44,c04939a0) at spec_poll+0x134 spec_vnoperate(d8dfcb44,c04939a0,c52cfa44,41,c6cfd280) at spec_vnoperate+0x18 vn_poll(c45dc880,41,c6cfd280,c5f7a4c0,c6cfd280) at vn_poll+0x3c pollscan(c5f7a4c0,d8dfcbd4,2,3e7,10) at pollscan+0xb0 poll(c5f7a4c0,d8dfcd10,c0455899,3ee,3) at poll+0x252 syscall(2f,2f,2f,0,2) at syscall+0x273 Xint0x80_syscall() at Xint0x80_syscall+0x1d #8 0xc0290ed7 in witness_lock (lock=0xc04aa120, flags=8, file=0xc0434e3d "/a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c", line=372) at /a/asami/portbuild/i386/src-client/sys/kern/subr_witness.c:838 #9 0xc0261f4a in _mtx_lock_flags (m=0x0, opts=0, file=0xc04d1818 "", line=-1068850912) at /a/asami/portbuild/i386/src-client/sys/kern/kern_mutex.c:334 #10 0xc0231154 in spec_poll (ap=0xd8dfcb44) at /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372 #11 0xc0230648 in spec_vnoperate (ap=0x0) at /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:122 #12 0xc02d119c in vn_poll (fp=0x0, events=0, active_cred=0xc6cfd280, td=0x0) at vnode_if.h:537 #13 0xc0294c50 in pollscan (td=0xc5f7a4c0, fds=0xd8dfcbdc, nfd=2) at /a/asami/portbuild/i386/src-client/sys/sys/file.h:272 #14 0xc02948a2 in poll (td=0xc5f7a4c0, uap=0xd8dfcd10) at /a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:1001 #15 0xc03ef9b3 in syscall (frame= {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 0, tf_esi = 2, tf_ebp = -1077940448, tf_isp = -656421516, +tf_ebx = 673224876, tf_edx = 139153408, tf_ecx = 137314336, tf_eax = 209, tf_trapno = 0, tf_err = 2, tf_eip = +672942388, tf_cs = 31, tf_eflags = 659, tf_esp = -1077940508, tf_ss = 47}) at /a/asami/portbuild/i386/src-client/sys/i386/i386/trap.c:1008 #16 0xc03dfbed in Xint0x80_syscall () at {standard input}:144 ---Can't read userspace from dump, or kernel process--- The original mail may be found here: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=73697+0+current/freebsd-current Message-ID: <20030727233351.GB80934@rot13.obsecurity.org> rwatson replied with: > I've bumped into some similar problems -- it's a property of how we > current lock select(). We hold the file descriptor lock for the duration > of polling each object being "selected", and if any of those objects has > to grab a lock for any reason, it has to implicitly fall after the file > descriptor lock. I actually run into this in some of our MAC code, > because I need to grab a vnode lock to authorize polling the vnode using > VOP_POLL(), and since the vnode lock is a sleep lock, this generates a > WITNESS warning. Unfortunately, it's not immediately clear what a better > locking scheme would look like without going overboard on the fine-grained > side. We probably need to grab Giant before entering the select code > since it's highly likely something in there will require Giant -- it > reaches down into VFS, the device stuff, socket code, tc. >How-To-Repeat: Unknown. >Fix: >Release-Note: >Audit-Trail: >Unformatted: