From owner-freebsd-arch@FreeBSD.ORG Wed Feb 19 21:04:55 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3CE9231B; Wed, 19 Feb 2014 21:04:55 +0000 (UTC) Received: from mail-ee0-x229.google.com (mail-ee0-x229.google.com [IPv6:2a00:1450:4013:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5115E1445; Wed, 19 Feb 2014 21:04:54 +0000 (UTC) Received: by mail-ee0-f41.google.com with SMTP id e49so477480eek.0 for ; Wed, 19 Feb 2014 13:04:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=FB0GMNpZOBmSHQf7WepADuZcQqRlVrLrhRN+FxFJx1k=; b=x2w2ZW/RrLTiaquWqQ88uFyAedhD67Vn2LjpOFtZRb0/DRuRT3Bbzsm1P41C4EYKQr ZJRwLSJ5fsbRiOcpNfX9cbHObyfHiVC9ZW9vbgFymU94pfsLTUaRzkKjjHJ2N+/CPROA XF/UMha5q6vlQN0k9ctW6qBvAeS2LhHGN3qtvYwmDuYzNf48ltq6sKZ+7shNoFX86yhW nw/+9aY7oxmjiTTTMxk2KehuUpdkEK0c1e1XOXoApD/A0ChwsWSyyhALLmPOBz+9hsMC Jg3sFyUI2XtCr7in2wH+PmfPd2bRmVRhaW3K9uYc2TrXN9aiDlT3vJRFwgJkf3Cc9vwW u4kw== X-Received: by 10.15.98.68 with SMTP id bi44mr16976364eeb.67.1392843892638; Wed, 19 Feb 2014 13:04:52 -0800 (PST) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id u6sm5116099eep.11.2014.02.19.13.04.50 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 19 Feb 2014 13:04:51 -0800 (PST) Sender: Alexander Motin Message-ID: <53051C71.3050705@FreeBSD.org> Date: Wed, 19 Feb 2014 23:04:49 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: [rfc] bind per-cpu timeout threads to each CPU References: <530508B7.7060102@FreeBSD.org> <53050D24.3020505@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Jeffrey Faden , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2014 21:04:55 -0000 On 19.02.2014 22:04, Adrian Chadd wrote: > On 19 February 2014 11:59, Alexander Motin wrote: > >>> So if we're moving towards supporting (among others) a pcbgroup / RSS >>> hash style work load distribution across CPUs to minimise >>> per-connection lock contention, we really don't want the scheduler to >>> decide it can schedule things on other CPUs under enough pressure. >>> That'll just make things worse. > >> True, though it is also not obvious that putting second thread on CPU run >> queue is better then executing it right now on another core. > > Well, it depends if you're trying to optimise for "run all runnable > tasks as quickly as possible" or "run all runnable tasks in contexts > that minimise lock contention." > > The former sounds great as long as there's no real lock contention > going on. But as you add more chances for contention (something like > "100,000 concurrent TCP flows") then you may end up having your TCP > timer firing stuff interfere with more TXing or RXing on the same > connection. 100K TCP flows probably means 100K locks. That means that chance of lock collision on each of them is effectively zero. More realistic it could be to speak about cache coherency traffic, etc, but I still think that number of expired timeouts should be much lower then number of other flow data accesses. > Chasing this stuff down is a pain, because it only really shows up > when you're doing lots of concurrency. > > I'm happy to make this a boot-time option and leave it off for the > time being. How's that? I generally hate tunables like that. There are too few people who may even try to make grounded decision in that question. If you think it right -- just do it, otherwise -- don't do it. I am not really objecting, more like sounding concerns. -- Alexander Motin