From owner-freebsd-performance@FreeBSD.ORG Tue Jun 13 10:01:11 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6AC2A16A41B; Tue, 13 Jun 2006 10:01:11 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 03E8F43D46; Tue, 13 Jun 2006 10:01:10 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 993A046C43; Tue, 13 Jun 2006 06:01:10 -0400 (EDT) Date: Tue, 13 Jun 2006 11:01:10 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: David Xu In-Reply-To: <200606130715.52425.davidxu@freebsd.org> Message-ID: <20060613105930.N34121@fledge.watson.org> References: <20060612195754.72452.qmail@web33306.mail.mud.yahoo.com> <20060612210723.K26068@fledge.watson.org> <20060612203248.GA72885@xor.obsecurity.org> <200606130715.52425.davidxu@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-performance@freebsd.org, kmacy@FreeBSD.org, danial_thom@yahoo.com, Scott Long , Kris Kennaway Subject: Re: Initial 6.1 questions X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jun 2006 10:01:11 -0000 On Tue, 13 Jun 2006, David Xu wrote: > On Tuesday 13 June 2006 04:32, Kris Kennaway wrote: >> On Mon, Jun 12, 2006 at 09:08:12PM +0100, Robert Watson wrote: >>> On Mon, 12 Jun 2006, Scott Long wrote: >>>> I run a number of high-load production systems that do a lot of network >>>> and filesystem activity, all with HZ set to 100. It has also been shown >>>> in the past that certain things in the network area where not fixed to >>>> deal with a high HZ value, so it's possible that it's even more >>>> stable/reliable with an HZ value of 100. >>>> >>>> My personal opinion is that HZ should gop back down to 100 in 7-CURRENT >>>> immediately, and only be incremented back up when/if it's proven to be >>>> the right thing to do. And, I say that as someone who (errantly) pushed >>>> for the increase to 1000 several years ago. >>> >>> I think it's probably a good idea to do it sooner rather than later. It >>> may slightly negatively impact some services that rely on frequent timers >>> to do things like retransmit timing and the like. But I haven't done any >>> measurements. >> >> As you know, but for the benefit of the list, restoring HZ=100 is often an >> important performance tweak on SMP systems with many CPUs because of all >> the sched_lock activity from statclock/hardclock, which scales with HZ and >> NCPUS. > > sched_lock is another big bottleneck, since if you 32 CPUs, in theory you > have 32X context switch speed, but now it still has only 1X speed, and there > are code abusing sched_lock, the M:N bits dynamically inserts a thread into > thread list at context switch time, this is a bug, this causes thread list > in a proc has to be protected by scheduler lock, and delivering a signal to > process has to hold scheduler lock and find a thread, if the proc has many > threads, this will introduce long scheduler latency, a proc lock is not > enough to find a thread, this is a bug, there are other code abusing > scheduler lock which really can use its own lock. I've added Kip Macy to the CC, who is working with a patch for Sun4v that eliminates sched_lock. Maybe he can comment some more on this thread? Robert N M Watson Computer Laboratory Universty of Cambridge