From owner-freebsd-arch@FreeBSD.ORG Fri Nov 23 15:57:02 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2356716A41A for ; Fri, 23 Nov 2007 15:57:02 +0000 (UTC) (envelope-from ups@freebsd.org) Received: from smtpauth13.prod.mesa1.secureserver.net (smtpauth13.prod.mesa1.secureserver.net [64.202.165.37]) by mx1.freebsd.org (Postfix) with SMTP id 8B85613C458 for ; Fri, 23 Nov 2007 15:57:01 +0000 (UTC) (envelope-from ups@freebsd.org) Received: (qmail 24905 invoked from network); 23 Nov 2007 15:57:00 -0000 Received: from unknown (66.23.216.53) by smtpauth13.prod.mesa1.secureserver.net (64.202.165.37) with ESMTP; 23 Nov 2007 15:57:00 -0000 Message-ID: <4746F858.4070301@freebsd.org> Date: Fri, 23 Nov 2007 10:57:12 -0500 From: Stephan Uphoff User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Alfred Perlstein References: <20071121222319.GX44563@elvis.mu.org> <200711221641.02484.max@love2party.net> <3bbf2fe10711220753u435ff4cbxa94d5b682292b970@mail.gmail.com> <200711221726.27108.max@love2party.net> <20071123082339.GN44563@elvis.mu.org> <47469328.8020404@freebsd.org> <20071123092415.GP44563@elvis.mu.org> In-Reply-To: <20071123092415.GP44563@elvis.mu.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Attilio Rao , Max Laier , freebsd-arch@freebsd.org Subject: Re: rwlocks, correctness over speed. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Nov 2007 15:57:02 -0000 Alfred Perlstein wrote: > * Stephan Uphoff [071123 00:46] wrote: > >> Alfred Perlstein wrote: >> >>> * Max Laier [071122 14:40] wrote: >>> >>> >>>> On Thursday 22 November 2007, Attilio Rao wrote: >>>> >>>> >>>>> 2007/11/22, Max Laier : >>>>> >>>>> >>>>>> rwlocks are already used in places that do recursive reads. The one >>>>>> place I'm certain about is pfil(9) and we need a proper sollution for >>>>>> this. Before rwlocks were used, I had a handrolled locking that >>>>>> supported both read/write semantics and starvation avoidance - at the >>>>>> cost of failing to allow futher read access when a writer asked for >>>>>> access. This however, was quite application specific and not the >>>>>> most efficient implementation either. >>>>>> >>>>>> >>>>> I'm not a pfil(9) expert, but for what I've heard, rmlocks should be >>>>> really good for it, shouldn't them? >>>>> >>>>> The concept is that if we want to maintain fast paths for rwlock we >>>>> cannot do too much tricks there. And you can really deadlock if you >>>>> allow recursion on readers... >>>>> >>>>> >>>> How about adding rwlock_try_rlock() which would do the following: >>>> 1) Only variant to allow[1] read recursion and ... >>>> 2) ... only if no outstanding write requests >>>> 3) Let the caller deal with failure >>>> >>>> This can be implemented statically, so no overhead in the fast path. The >>>> caller is in the best position to decide if it is recursing or not - >>>> could keep that info on the stack - and can either fail[2] or do a normal >>>> rwlock_rlock() which would wait for the writer to enter and exit. >>>> >>>> [2] In most situation where you use read locks you can fail or roll back >>>> carefully as you didn't change anything - obviously. In pfil - for >>>> instance - we just dropped the packet if there was a writer waiting. >>>> >>>> [1] "allow" in terms of WITNESS - if that can be done. >>>> >>>> >>> The problem is that there is no tracking in the common case (without >>> additional overhead), so you can't know if you're recursing, unless >>> you track it yourself. >>> >>> -Alfred >>> >>> >>> >>> >> I talked with Attilio about that on IRC. >> Most common cases of writer starvation (but not all) could be solved by >> keeping a per thread count of shared acquired rwlocks. >> If a rwlock is currently locked as shared/read AND a thread is blocked >> on it to lock it exclusively/write - then new shared/read locks >> will only be granted to thread that already has a shared lock. (per >> thread shared counter is non zero) >> >> To be honest I am a bit twitchy about a lock without priority >> propagation - especially since in FreeBSD threads run with user priority >> in kernel >> space and can get preempted. >> >> Stephan >> > > That's an interesting hack, I guess it could be done. > > I would still like to disallow recursion. > Oh - I am all for disallowing recursion. In my opinion the only valid place for a thread to acquire the same lock multiple times is inside a transaction system with full deadlock detection. The question is if we can do that this late in the game? Maybe we could make non recursive the default and add a call rw_allow_recurse or rw_init_recurse to allow recursion on a lock if we can't get away with the straight out removal of the option? (Or less desirable - keep the current default and add functions to disallow recursion) > Can we come to a concensus on that? > > +1 for disallowing recursion Stephan