Date: Tue, 13 Jun 2006 12:34:44 -0700 From: "Kip Macy" <kip.macy@gmail.com> To: "Robert Watson" <rwatson@freebsd.org> Cc: Scott Long <scottl@samsco.org>, kmacy@freebsd.org, David Xu <davidxu@freebsd.org>, Kris Kennaway <kris@obsecurity.org>, freebsd-performance@freebsd.org, danial_thom@yahoo.com Subject: Re: Initial 6.1 questions Message-ID: <b1fa29170606131234h35631b27md45969b83081d6c5@mail.gmail.com> In-Reply-To: <20060613105930.N34121@fledge.watson.org> References: <20060612195754.72452.qmail@web33306.mail.mud.yahoo.com> <20060612210723.K26068@fledge.watson.org> <20060612203248.GA72885@xor.obsecurity.org> <200606130715.52425.davidxu@freebsd.org> <20060613105930.N34121@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
I have a number of issues with our current locking regime and our
propensity for disabling interrupts. I have in mind some ideas for
reducing interrupt disabling and eliminating scheduling contention
except in the case of one cpu stealing a thread from another cpu's
runqueue. I'll try to dash that off early this evening. This should
also greatly reduce the overhead of timer interrupts.
-Kip
On 6/13/06, Robert Watson <rwatson@freebsd.org> wrote:
>
> On Tue, 13 Jun 2006, David Xu wrote:
>
> > On Tuesday 13 June 2006 04:32, Kris Kennaway wrote:
> >> On Mon, Jun 12, 2006 at 09:08:12PM +0100, Robert Watson wrote:
> >>> On Mon, 12 Jun 2006, Scott Long wrote:
> >>>> I run a number of high-load production systems that do a lot of network
> >>>> and filesystem activity, all with HZ set to 100. It has also been shown
> >>>> in the past that certain things in the network area where not fixed to
> >>>> deal with a high HZ value, so it's possible that it's even more
> >>>> stable/reliable with an HZ value of 100.
> >>>>
> >>>> My personal opinion is that HZ should gop back down to 100 in 7-CURRENT
> >>>> immediately, and only be incremented back up when/if it's proven to be
> >>>> the right thing to do. And, I say that as someone who (errantly) pushed
> >>>> for the increase to 1000 several years ago.
> >>>
> >>> I think it's probably a good idea to do it sooner rather than later. It
> >>> may slightly negatively impact some services that rely on frequent timers
> >>> to do things like retransmit timing and the like. But I haven't done any
> >>> measurements.
> >>
> >> As you know, but for the benefit of the list, restoring HZ=100 is often an
> >> important performance tweak on SMP systems with many CPUs because of all
> >> the sched_lock activity from statclock/hardclock, which scales with HZ and
> >> NCPUS.
> >
> > sched_lock is another big bottleneck, since if you 32 CPUs, in theory you
> > have 32X context switch speed, but now it still has only 1X speed, and there
> > are code abusing sched_lock, the M:N bits dynamically inserts a thread into
> > thread list at context switch time, this is a bug, this causes thread list
> > in a proc has to be protected by scheduler lock, and delivering a signal to
> > process has to hold scheduler lock and find a thread, if the proc has many
> > threads, this will introduce long scheduler latency, a proc lock is not
> > enough to find a thread, this is a bug, there are other code abusing
> > scheduler lock which really can use its own lock.
>
> I've added Kip Macy to the CC, who is working with a patch for Sun4v that
> eliminates sched_lock. Maybe he can comment some more on this thread?
>
> Robert N M Watson
> Computer Laboratory
> Universty of Cambridge
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?b1fa29170606131234h35631b27md45969b83081d6c5>