From owner-freebsd-arch@FreeBSD.ORG Wed Feb 19 21:04:59 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F5C8417; Wed, 19 Feb 2014 21:04:59 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 452531447; Wed, 19 Feb 2014 21:04:59 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5DE0BB972; Wed, 19 Feb 2014 16:04:58 -0500 (EST) From: John Baldwin To: Adrian Chadd Subject: Re: [rfc] bind per-cpu timeout threads to each CPU Date: Wed, 19 Feb 2014 16:02:54 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <530508B7.7060102@FreeBSD.org> <53050D24.3020505@FreeBSD.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201402191602.54465.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 19 Feb 2014 16:04:58 -0500 (EST) Cc: Alexander Motin , freebsd-current , Jeffrey Faden , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 21:04:59 -0000 On Wednesday, February 19, 2014 3:04:51 pm Adrian Chadd wrote: > On 19 February 2014 11:59, Alexander Motin wrote: > > >> So if we're moving towards supporting (among others) a pcbgroup / RSS > >> hash style work load distribution across CPUs to minimise > >> per-connection lock contention, we really don't want the scheduler to > >> decide it can schedule things on other CPUs under enough pressure. > >> That'll just make things worse. > > > True, though it is also not obvious that putting second thread on CPU run > > queue is better then executing it right now on another core. > > Well, it depends if you're trying to optimise for "run all runnable > tasks as quickly as possible" or "run all runnable tasks in contexts > that minimise lock contention." > > The former sounds great as long as there's no real lock contention > going on. But as you add more chances for contention (something like > "100,000 concurrent TCP flows") then you may end up having your TCP > timer firing stuff interfere with more TXing or RXing on the same > connection. > > Chasing this stuff down is a pain, because it only really shows up > when you're doing lots of concurrency. > > I'm happy to make this a boot-time option and leave it off for the > time being. How's that? I think having it be a tunable would be good. OTOH, I could also see another option which would be to pin all clock threads except for the "default" one by default and only have the option control whether or not the default thread is pinned to CPU 0 as callers who use callout_on() are explicitly asking to run the callout on a specific CPU. -- John Baldwin