From owner-freebsd-arch@FreeBSD.ORG Wed May 22 13:27:24 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 902DAA7B; Wed, 22 May 2013 13:27:24 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 808A96F; Wed, 22 May 2013 13:27:23 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (OTWAON23-1176242366.sdsl.bell.ca [70.28.8.190]) by elvis.mu.org (Postfix) with ESMTPSA id 166461A3C23; Wed, 22 May 2013 06:27:20 -0700 (PDT) Message-ID: <519CC7B4.2030208@mu.org> Date: Wed, 22 May 2013 09:27:16 -0400 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: John Baldwin Subject: Re: FreeBSD spinlock - compatibility layer References: <981733489AB3BD4DB24B48340F53E0A55B0CFD79@MTLDAG01.mtl.com> <201305200950.26834.jhb@freebsd.org> <519BDBB0.2070302@mu.org> <201305220905.57939.jhb@freebsd.org> In-Reply-To: <201305220905.57939.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Orit Moskovich , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 13:27:24 -0000 On 5/22/13 9:05 AM, John Baldwin wrote: > On Tuesday, May 21, 2013 4:40:16 pm Alfred Perlstein wrote: >> On 5/20/13 9:50 AM, John Baldwin wrote: >>> On Tuesday, May 14, 2013 6:04:21 am Orit Moskovich wrote: >>>> Hi, >>>> >>>> I read about the FreeBSD mutex implementation for spinlock in the >>> compatibility layer. >>>> I might be wrong, but I noticed a code section that might be problematic: >>>> >>>> Taken from > http://svn.freebsd.org/base/release/9.1.0/sys/ofed/include/linux/spinlock.h: >>>> static inline void >>>> spin_lock_init(spinlock_t *lock) >>>> { >>>> >>>> memset(&lock->m, 0, sizeof(lock->m)); >>>> mtx_init(&lock->m, "lnxspin", NULL, MTX_DEF | MTX_NOWITNESS); >>>> } >>>> >>>> But MTX_DEF initializes mutex as a sleep mutex: >>>> >>>> By default, MTX_DEF mutexes will context switch when they are already >>>> >>>> held. >>>> >>>> >>>> There is a flag MTX_SPIN Which I think is the right one in this case . >>>> >>>> >>>> >>>> I'd appreciate your take on this issue. >>> Since FreeBSD uses a different approach to interrupt handlers (they run in >>> threads, not in the bottom half), a regular mutex may in fact give the > closest >>> match to the same semantics. Regular mutexes are also cheaper and in > general >>> preferable to spin mutexes whenever possible. >>> >> Sure, but is it possible that someone might want some of the other >> guarantees of MTX_SPIN spinlocks such as: >> >> critical section/non-pre-emptable/non-migrating on cpu/latency versus >> throughput ? > Probably not. For example, on FreeBSD you want your driver lock to be > preempted by an interrupt to avoid higher interrupt latency for filter > handlers. Most drivers should not need temporary pinning. If they want to > pin work to threads they should bind threads or IRQs to specific CPUs, not > rely on temporary pinning. > I know how it works in FreeBSD. I think that a compatibility layer should first and foremost aim for compatibility, not speed at expense of expected semantics. That said, I'm hopeful that the ofed stack doesn't use any of the other guarantees you'd expect from real spinlocks other than mutual exclusion so it's not that big of a deal. Unless it does... which will be interesting track down. -Alfred