From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 9 09:52:18 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AB0722E3 for ; Fri, 9 Nov 2012 09:52:18 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id DD2B88FC14 for ; Fri, 9 Nov 2012 09:52:16 +0000 (UTC) Received: (qmail 72107 invoked from network); 9 Nov 2012 11:27:12 -0000 Received: from unknown (HELO [62.48.0.94]) ([62.48.0.94]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 9 Nov 2012 11:27:12 -0000 Message-ID: <509CD2A8.3070404@freebsd.org> Date: Fri, 09 Nov 2012 10:53:44 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121026 Thunderbird/16.0.2 MIME-Version: 1.0 To: David Xu Subject: Re: ule+smp: small optimization for turnstile priority lending References: <50587F8D.9060102@FreeBSD.org> <505AD2A5.6060008@freebsd.org> <5099C5C9.4040703@freebsd.org> <509A11B4.7030605@freebsd.org> In-Reply-To: <509A11B4.7030605@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: attilio@freebsd.org, freebsd-hackers , Jeff Roberson , Jeff Roberson , Andriy Gapon X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Nov 2012 09:52:18 -0000 On 07.11.2012 08:45, David Xu wrote: > On 2012/11/07 14:17, Jeff Roberson wrote: >> On Wed, 7 Nov 2012, David Xu wrote: >> >>> On 2012/11/06 19:03, Attilio Rao wrote: >>>> On 9/20/12, David Xu wrote: >>>>> I found another scenario in taskqueue, in the function >>>>> taskqueue_terminate, current thread tries to wake >>>>> another thread up and sleep immediately, the tq_mutex sometimes >>>>> is a spinlock. So if you remove one thread load from current cpu >>>>> before wakeup, the resumed thread may be put on same cpu, >>>>> so it will optimize the cpu scheduling too. >>>> >>>> I think that in order to fit with sched_add() modifies I have in mind >>>> (see other patches within this thread) wakeup() should grow a new >>>> argument, or maybe we can use wakeup_flags() new KPI. >>>> If the latter is the case, I would also propose to let wakeup_one() to >>>> be absorbed into wakeup_flags() with its own flag. >>>> >>> >>> Yes, I like the idea. >> >>> From ~2007 >> >> http://people.freebsd.org/~jeff/wakeupflags.diff >> >> This has some different optimizations that can be done when you have a >> hinted wakeup. >> > > wakeup_flags is a good start point. > But what flags should be supported? I think WAKEUP_WILLSLEEP may be > needed if the current thread will switch out as soon as possible. WAKEUP_ONE is important for accept sockets. We don't want all idle threads to compete and contend on a new connection. -- Andre > I see you have added WAKEUP_LOCAL and WAKEUP_TAIL. Are they for cache > optimization ? both of them are arguable. > > WAKEUP_LOCAL increases cpu migration, not good, since one benefit of > ULE is keeping thread on its old cpu, the WAKEUP_LOCAL violates the > design. > WAKEUP_LOCAL used in pipe code may not be correct if the pipe only > has few of bytes to be transfered or receiver only eats a few bytes > each time, so why bother to move other threads to local cpu ? > If the other threads have many bytes cached in their old cpu, this > migration is expensive. > > WAKEUP_TAIL is a good idea, I guess that you want to wake up hot-thread > its code and data are still in its old cpu's cache. But this will lead > to unfair. I'd like the sleep queue to implement a policy like I did > in libthr, it picks a thread at tail of queue in a fixed percentage, it > does have some level of unfair but not fatal at all. > >> Thanks, >> Jeff > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > >