From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 19 14:53:42 2006 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D7D4D16A412 for ; Thu, 19 Oct 2006 14:53:42 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id C481943D46 for ; Thu, 19 Oct 2006 14:53:41 +0000 (GMT) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id k9JErWN6066690; Thu, 19 Oct 2006 10:53:33 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 19 Oct 2006 10:09:52 -0400 User-Agent: KMail/1.9.1 References: <5e4707340608181226u131be51ak547c5912a35cfcec@mail.gmail.com> In-Reply-To: <5e4707340608181226u131be51ak547c5912a35cfcec@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200610191009.53148.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Thu, 19 Oct 2006 10:53:33 -0400 (EDT) X-Virus-Scanned: ClamAV 0.88.3/2050/Thu Oct 19 03:58:33 2006 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Alex Unleashed Subject: Re: devfs deadlock X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Oct 2006 14:53:43 -0000 On Friday 18 August 2006 15:26, Alex Unleashed wrote: > Hello, > > Before anything else I'd like to say I'm working on a SoC project for > Gentoo for which I have to port a sandbox-like application wrapper (in > userspace) to FreeBSD, which deals with the building and installing of > software so that nothing gets screwed in the filesystem. It's > finished, but unfortunately it gets frozen at random points, > preventing the system from launching new programs or saving to disk, > which suggested a kernel bug, as was later confirmed looking at the > waitchannels and debugging. I know there is someone working out issues > in devfs code in 6.x, so this might also be interesting to him. > > I've been able to reproduce both in 6.1-RELEASE-p3 and 6-STABLE > (snapshot from August 16th ~01:00 GMT) a deadlock in devfs code which > leaves the system unable to access the disk. I've come up with some > interesting debugging info, and it looks to me like there are vnode > problems while a sx lock is being held. > > My take at it is that the deadlock occurs when a process gets a lock > on a vnode (tagged "devfs") and another process xlocks an sx lock > ("devfsmount"). For some reason the one holding the sx lock wants to > get the lock on the vnode through devfs_allocv(), and the other > process wants to get the sx lock through devfs_lookup(). From this > point on, pretty much anything wanting to touch the filesystem waits > forever on devfs_root() for another vnode flagged as VV_ROOT and > locked by the process holding the sx lock. > > Patching the devfs code with fixes from -CURRENT didn't work out. This deadlock is fixed in current as of this: kib 2006-09-18 13:23:08 UTC FreeBSD src repository Modified files: sys/fs/devfs devfs.h devfs_devs.c devfs_vfsops.c devfs_vnops.c Log: Resolve the devfs deadlock caused by LOR between devfs_mount->dm_lock and vnode lock in devfs_allocv. Do this by temporary dropping dm_lock around vnode locking. For safe operation, add hold counters for both devfs_mount and devfs_dirent, and DE_DOOMED flag for devfs_dirent. The facilities allow to continue after dropping of the dm_lock, by making sure that referenced memory does not disappear. Reviewed by: tegge Tested by: kris Approved by: kan (mentor) PR: kern/102335 Revision Changes Path 1.30 +11 -0 src/sys/fs/devfs/devfs.h 1.47 +12 -1 src/sys/fs/devfs/devfs_devs.c 1.51 +20 -4 src/sys/fs/devfs/devfs_vfsops.c 1.134 +70 -11 src/sys/fs/devfs/devfs_vnops.c and kib 2006-09-19 14:03:02 UTC FreeBSD src repository Modified files: sys/fs/devfs devfs_vnops.c Log: Fix the bug in rev. 1.134. In devfs_allocv_drop_refs(), when not_found == 2 and drop_dm_lock is true, no unlocking shall be attempted. The lock is already dropped and memory is freed. Found with: Coverity Prevent(tm) CID: 1536 Approved by: pjd (mentor) Revision Changes Path 1.135 +1 -1 src/sys/fs/devfs/devfs_vnops.c -- John Baldwin