From owner-freebsd-stable@FreeBSD.ORG Mon Dec 19 21:33:45 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 6B4AF106566B; Mon, 19 Dec 2011 21:33:45 +0000 (UTC) Date: Mon, 19 Dec 2011 21:33:45 +0000 From: Alexander Best To: Nathan Whitehorn Message-ID: <20111219213345.GA64578@freebsd.org> References: <4EE69C5A.3090005@FreeBSD.org> <20111213104048.40f3e3de@nonamehost> <20111213090051.GA3339@vniz.net> <4EED5200.20302@cran.org.uk> <20111218164924.L64681@sola.nimnet.asn.au> <20111218075241.GA45367@vniz.net> <20111218102401.GA42627@freebsd.org> <20111218102600.GA44118@freebsd.org> <4EEF5CC1.7020709@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EEF5CC1.7020709@freebsd.org> X-Mailman-Approved-At: Mon, 19 Dec 2011 21:51:44 +0000 Cc: Bruce Cran , Ivan Klymenko , Adrian Chadd , Doug Barton , freebsd-stable@freebsd.org, Ian Smith , "O. Hartmann" , Current FreeBSD , freebsd-performance@freebsd.org Subject: Re: SCHED_ULE should not be the default X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 21:33:45 -0000 On Mon Dec 19 11, Nathan Whitehorn wrote: > On 12/18/11 04:34, Adrian Chadd wrote: > >The trouble is that there's lots of anecdotal evidence, but noone's > >really gone digging deep into _their_ example of why it's broken. The > >developers who know this stuff don't see anything wrong. That hints to > >me it may be something a little more creepy - as an example, the > >interplay between netisr/swi/taskqueue/callbacks and such. It may be > >that something is being starved that isn't obviously obvious. It's > >just a stab in the dark, but it sounds somewhat plausible based on > >what I've seen ULE do in my network throughput hacking. > > > >I applaud reppie for trying to make it as easy as possible for people > >to use KTR to provide scheduler traces for him to go digging with, so > >please, if you have these issues and you can absolutely reproduce > >them, please follow his instructions and work with him to get him what > >he needs. > > The thing I've seen is that ULE is substantially more enthusiastic about > migrating processes between cores than 4BSD. Often, this is a good > thing, but can increase the rate of cache misses, hurting performance > for cache-bound processes (I see this particularly in HPC-type > scientific workloads). It might be interesting to add some kind of > tunable here. does r228718 have any impact regarding this behaviour? cheers. alex > > Another more interesting and slightly longer-term possibility if someone > wants a project would be to integrate scheduling decisions with hwpmc > counters, to accumulate statistics on cache hits at each context switch > and preferentially keep processes with a high hits/misses ratio on the > same thread/cache domain relative to processes with a low one. > -Nathan > > P.S. The other thing that could be very interesting from a research and > scheduling standpoint would be to integrate heterogeneous SMP support > into the operating system, with a FreeBSD-4 "Application Processor" > syscall model. We seem to be going down the road where GPGPU computing > has MMUs, timer interrupts, IPIs, etc. (the next AMD Fusions, IBM Cell). > This is something that no operating system currently supports well, and > would be a place for BSD to shine. If anyone has a free graduate student...