From owner-freebsd-current@FreeBSD.ORG Mon Jul 28 08:10:48 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D47A837B405; Mon, 28 Jul 2003 08:10:48 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 903C543FE0; Mon, 28 Jul 2003 08:10:47 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h6SF9uai057764; Mon, 28 Jul 2003 11:09:56 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h6SF9ul5057761; Mon, 28 Jul 2003 11:09:56 -0400 (EDT) Date: Mon, 28 Jul 2003 11:09:55 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Kris Kennaway In-Reply-To: <20030727233351.GB80934@rot13.obsecurity.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: alc@FreeBSD.org cc: current@FreeBSD.org Subject: Re: LOR with filedesc structure and Giant X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2003 15:10:49 -0000 On Sun, 27 Jul 2003, Kris Kennaway wrote: > After upgrading last night, one of the package machines found this: I've bumped into some similar problems -- it's a property of how we current lock select(). We hold the file descriptor lock for the duration of polling each object being "selected", and if any of those objects has to grab a lock for any reason, it has to implicitly fall after the file descriptor lock. I actually run into this in some of our MAC code, because I need to grab a vnode lock to authorize polling the vnode using VOP_POLL(), and since the vnode lock is a sleep lock, this generates a WITNESS warning. Unfortunately, it's not immediately clear what a better locking scheme would look like without going overboard on the fine-grained side. We probably need to grab Giant before entering the select code since it's highly likely something in there will require Giant -- it reaches down into VFS, the device stuff, socket code, tc. > lock order reversal > 1st 0xc6c1c334 filedesc structure (filedesc structure) @ > /a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:902 > 2nd 0xc04aa120 Giant (Giant) @ > /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372 > Stack backtrace: > backtrace(c043d4af,c04aa120,c0439aa4,c0439aa4,c0434e3d) at backtrace+0x17 > witness_lock(c04aa120,8,c0434e3d,174,1bc) at witness_lock+0x672 > _mtx_lock_flags(c04aa120,0,c0434e3d,174,c043daba) at _mtx_lock_flags+0xba > spec_poll(d8dddaf8,d8dddb18,c02d119c,d8dddaf8,c04939a0) at spec_poll+0x134 > spec_vnoperate(d8dddaf8,c04939a0,c520b124,40,c675e300) at spec_vnoperate+0x18 > vn_poll(c44c5e14,40,c675e300,c6222d10,c675e300) at vn_poll+0x3c > selscan(c6222d10,d8dddb98,d8dddb88,6,4) at selscan+0x13e > kern_select(c6222d10,6,bfbff5c0,0,0) at kern_select+0x36f > select(c6222d10,d8dddd10,c0455899,3ee,5) at select+0x66 > syscall(2f,2f,2f,8055050,bfbff5b8) at syscall+0x273 > Xint0x80_syscall() at Xint0x80_syscall+0x1d > --- syscall (93), eip = 0x280ccacc, esp = 0x2832eb68, ebp = 0x2832ebc0 --- > Debugger("witness_lock") > Stopped at Debugger+0x54: xchgl %ebx,in_Debugger.0 > > Kris