From owner-freebsd-threads@FreeBSD.ORG Tue Jul 6 15:51:39 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1374B16A4CE for ; Tue, 6 Jul 2004 15:51:39 +0000 (GMT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id C683943D41 for ; Tue, 6 Jul 2004 15:51:38 +0000 (GMT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.12.10/8.12.10) with ESMTP id i66FpZqM013468 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 6 Jul 2004 11:51:35 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id i66FpUk6017516; Tue, 6 Jul 2004 11:51:30 -0400 (EDT) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16618.51842.291266.840333@grasshopper.cs.duke.edu> Date: Tue, 6 Jul 2004 11:51:30 -0400 (EDT) To: Daniel Eischen In-Reply-To: References: X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid cc: Julian Elischer cc: freebsd-threads@freebsd.org Subject: Re: pthread switch (was Odd KSE panic) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2004 15:51:39 -0000 Daniel Eischen writes: > > With a small number of threads, it probably doesn't make sense > to have more than one KSE (unless they are mostly CPU-bound). > In Drew's example, there are 2 KSEs (HTT enabled) and only 2 threads. Actually, there are 2 procs, each with 2 threads. To elaborate, there are two processes, each running one instance of a pingpong test for each of 2 NICs in the box. > Each time a thread sleeps, the KSE enters the kernel to sleep > (kse_release()) because there are no other threads to run. > > Drew, can you try lowering the concurrency? You can > either try using pthread_setconcurrency(1) or setting > kern.threads.virtual_cpu=1. Here is the raw data for a ping-pong between 2 nics on the same box with various configurations of cpu_idle_hlt and virtual_cpu. Polling 5.0us Linux 20.4us linulator 82.5us thr 75.1us kse 123.6us machdep.cpu_idle_hlt=0 linulator 74.0us thr 64.6us kse 111.9 kern.threads.virtual_cpu=1 machdep.cpu_idle_hlt=0 kse 100.8 kern.threads.virtual_cpu=1 machdep.cpu_idle_hlt=0 kse 93.9 Key: Linux: kernel 2.66, Debian libc6 2.2.5-11.5 (no sysenter, or NPTL) linulator: static binary used above, run under COMPAT_LINUX thr: libthr (via libmap.conf) kse: libpthread BTW, there's a little more fat in the interrupt/wakeup path in the FreeBSD MD driver code (condvars, MPSAFE intr, rather than an INTR_FAST and some homebrew technique). Based on other measurements, I don't think its 54us worth of fat (more like 15 or 20usec). Drew