From owner-freebsd-current@FreeBSD.ORG Mon Feb 6 22:37:27 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 521A616A420; Mon, 6 Feb 2006 22:37:27 +0000 (GMT) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 83AEC43D48; Mon, 6 Feb 2006 22:37:26 +0000 (GMT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.13.4/8.13.4) with ESMTP id k16MbNNT025793 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 6 Feb 2006 17:37:23 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id k16MbISo000188; Mon, 6 Feb 2006 17:37:18 -0500 (EST) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17383.53150.324978.91528@grasshopper.cs.duke.edu> Date: Mon, 6 Feb 2006 17:37:18 -0500 (EST) To: John Baldwin In-Reply-To: <200602061532.02223.jhb@freebsd.org> References: <17379.56708.421007.613310@grasshopper.cs.duke.edu> <43E74872.7000002@freebsd.org> <17383.42908.349070.31155@grasshopper.cs.duke.edu> <200602061532.02223.jhb@freebsd.org> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Cc: freebsd-current@freebsd.org, Andre Oppermann Subject: Re: machdep.cpu_idle_hlt and SMP perf? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2006 22:37:27 -0000 John Baldwin writes: > On Monday 06 February 2006 14:46, Andrew Gallatin wrote: > > Andre Oppermann writes: > > > Andrew Gallatin wrote: > > > > Why dooes machdep.cpu_idle_hlt=1 drop my 10GbE network rx > > > > performance by a considerable amount (7.5Gbs -> 5.5Gbs)? > > > > <...> > > > > > This may be the same problem OpenBSD has fixed last year in the handling > > > of the idle loop. From the kerneltrap posting: > > > > <....> > > > > > First commit message: > > > http://marc.theaimsgroup.com/?l=openbsd-cvs&m=111692513727274&w=2 > > > > > > The MFC with all changes in one commit message: > > > http://marc.theaimsgroup.com/?l=openbsd-cvs&m=111859519015510&w=2 > > > > The bug they fixes was missing interrupts by both calling APM's idle > > routine, which may hlt, and hlt'ing in the idle loop itself. Since I > > have no idea what acpi is doing, I got excited about this. > > > > Alas, it seems like this isn't it. I pointed cpu_idle_hook back to > > cpu_idle_default and away from acpi_cpu_idle, but that made no > > difference. > > You may be seeing problems because it might simply take a while for the CPU to > wake up from HLT when an interrupt comes in. The 4BSD scheduler tries to do > IPIs to wakeup any sleeping CPUs when it schedules a new thread, but that > would add higher latency for ithreads than just preempting directly to the > ithread. Oh, you have to turn that on, it's off by default > (kern.sched.ipiwakeup.enabled=1). Hmm.. It seems to be on by default. Unfortunately, it does not seem to help. Would you expect ULE to do better? I've noticed that if I screw up the time state of the machine by switching between ACPI-fast and TSC timecounters, performance for TCP ping-pongs goes all over the map... Drew