Date: Mon, 19 Dec 2011 21:33:45 +0000 From: Alexander Best <arundel@freebsd.org> To: Nathan Whitehorn <nwhitehorn@freebsd.org> Cc: Bruce Cran <bruce@cran.org.uk>, Ivan Klymenko <fidaj@ukr.net>, Adrian Chadd <adrian@freebsd.org>, Doug Barton <dougb@freebsd.org>, freebsd-stable@freebsd.org, Ian Smith <smithi@nimnet.asn.au>, "O. Hartmann" <ohartman@mail.zedat.fu-berlin.de>, Current FreeBSD <freebsd-current@freebsd.org>, freebsd-performance@freebsd.org Subject: Re: SCHED_ULE should not be the default Message-ID: <20111219213345.GA64578@freebsd.org> In-Reply-To: <4EEF5CC1.7020709@freebsd.org> References: <4EE69C5A.3090005@FreeBSD.org> <20111213104048.40f3e3de@nonamehost> <20111213090051.GA3339@vniz.net> <4EED5200.20302@cran.org.uk> <20111218164924.L64681@sola.nimnet.asn.au> <20111218075241.GA45367@vniz.net> <20111218102401.GA42627@freebsd.org> <20111218102600.GA44118@freebsd.org> <CAJ-VmomJ9D63xYyg4udKu1FnihVworgp8MrxzXwpG2XVZww8DA@mail.gmail.com> <4EEF5CC1.7020709@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon Dec 19 11, Nathan Whitehorn wrote: > On 12/18/11 04:34, Adrian Chadd wrote: > >The trouble is that there's lots of anecdotal evidence, but noone's > >really gone digging deep into _their_ example of why it's broken. The > >developers who know this stuff don't see anything wrong. That hints to > >me it may be something a little more creepy - as an example, the > >interplay between netisr/swi/taskqueue/callbacks and such. It may be > >that something is being starved that isn't obviously obvious. It's > >just a stab in the dark, but it sounds somewhat plausible based on > >what I've seen ULE do in my network throughput hacking. > > > >I applaud reppie for trying to make it as easy as possible for people > >to use KTR to provide scheduler traces for him to go digging with, so > >please, if you have these issues and you can absolutely reproduce > >them, please follow his instructions and work with him to get him what > >he needs. > > The thing I've seen is that ULE is substantially more enthusiastic about > migrating processes between cores than 4BSD. Often, this is a good > thing, but can increase the rate of cache misses, hurting performance > for cache-bound processes (I see this particularly in HPC-type > scientific workloads). It might be interesting to add some kind of > tunable here. does r228718 have any impact regarding this behaviour? cheers. alex > > Another more interesting and slightly longer-term possibility if someone > wants a project would be to integrate scheduling decisions with hwpmc > counters, to accumulate statistics on cache hits at each context switch > and preferentially keep processes with a high hits/misses ratio on the > same thread/cache domain relative to processes with a low one. > -Nathan > > P.S. The other thing that could be very interesting from a research and > scheduling standpoint would be to integrate heterogeneous SMP support > into the operating system, with a FreeBSD-4 "Application Processor" > syscall model. We seem to be going down the road where GPGPU computing > has MMUs, timer interrupts, IPIs, etc. (the next AMD Fusions, IBM Cell). > This is something that no operating system currently supports well, and > would be a place for BSD to shine. If anyone has a free graduate student...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111219213345.GA64578>