From owner-freebsd-threads@FreeBSD.ORG Tue Jul 6 17:42:24 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 83FB116A4CE for ; Tue, 6 Jul 2004 17:42:24 +0000 (GMT) Received: from sccrmhc11.comcast.net (sccrmhc11.comcast.net [204.127.202.55]) by mx1.FreeBSD.org (Postfix) with ESMTP id D88A843D4C for ; Tue, 6 Jul 2004 17:42:23 +0000 (GMT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([24.7.73.28]) by comcast.net (sccrmhc11) with ESMTP id <20040706174222011009pojde>; Tue, 6 Jul 2004 17:42:23 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id KAA64674; Tue, 6 Jul 2004 10:42:20 -0700 (PDT) Date: Tue, 6 Jul 2004 10:42:18 -0700 (PDT) From: Julian Elischer To: Andrew Gallatin In-Reply-To: <16618.51842.291266.840333@grasshopper.cs.duke.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: pthread switch (was Odd KSE panic) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2004 17:42:24 -0000 On Tue, 6 Jul 2004, Andrew Gallatin wrote: > > Daniel Eischen writes: > > > > With a small number of threads, it probably doesn't make sense > > to have more than one KSE (unless they are mostly CPU-bound). > > In Drew's example, there are 2 KSEs (HTT enabled) and only 2 threads. > > Actually, there are 2 procs, each with 2 threads. > > To elaborate, there are two processes, each running one instance of a > pingpong test for each of 2 NICs in the box. > > > Each time a thread sleeps, the KSE enters the kernel to sleep > > (kse_release()) because there are no other threads to run. > > > > Drew, can you try lowering the concurrency? You can > > either try using pthread_setconcurrency(1) or setting > > kern.threads.virtual_cpu=1. > > > Here is the raw data for a ping-pong between 2 nics on the same box > with various configurations of cpu_idle_hlt and virtual_cpu. tried libc_r? > > > > Polling 5.0us > Linux 20.4us > linulator 82.5us > thr 75.1us > kse 123.6us > > machdep.cpu_idle_hlt=0 > linulator 74.0us > thr 64.6us > kse 111.9 > > kern.threads.virtual_cpu=1 > machdep.cpu_idle_hlt=0 > kse 100.8 > > kern.threads.virtual_cpu=1 > machdep.cpu_idle_hlt=0 > kse 93.9 > > > Key: > Linux: kernel 2.66, Debian libc6 2.2.5-11.5 (no sysenter, or NPTL) > linulator: static binary used above, run under COMPAT_LINUX > thr: libthr (via libmap.conf) > kse: libpthread > > > BTW, there's a little more fat in the interrupt/wakeup path in the > FreeBSD MD driver code (condvars, MPSAFE intr, rather than an > INTR_FAST and some homebrew technique). Based on other measurements, I > don't think its 54us worth of fat (more like 15 or 20usec). > > Drew >