From owner-svn-src-head@FreeBSD.ORG Wed Jan 21 16:09:17 2015 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34303C8D; Wed, 21 Jan 2015 16:09:17 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB2996C6; Wed, 21 Jan 2015 16:09:16 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id y20so16596247ier.1; Wed, 21 Jan 2015 08:09:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=bFJBZfQSywHnL1WtZHiyOUVxBuZplov8XDyrOieWhGQ=; b=UT+UKnXlcN0appnO9uaD+Wd6/U0aH8vMjXV0v5UqzLZhJvCh33ZYwcIxCrM4Lk3/qb 4hkEYUrv7iw3BKnwk6W0l1RYavry57TQJatEoVqIWCSTzGZlVkIYm7R2iTHiZydVtg/A C9ZfJXllMi4Gkd6E2+N8eREsazdrJxvXXfhniFY/al4uWeWtCy7P2q7bHSLtLqCfhGr4 axbQmtOS6oZkPI8axRYrQMi2Pbc5vJc5Vhvbs7/stE5T4IwHdbZ3PUWGQGPjH3i2DA9J cQGieKayfKN/3Vtey7rgObWW5f4GI14RkANGxi8Uw1yMAJiEIHivkqouP96UE2akI16B PqDw== MIME-Version: 1.0 X-Received: by 10.50.93.70 with SMTP id cs6mr5348158igb.6.1421856556360; Wed, 21 Jan 2015 08:09:16 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.78.14 with HTTP; Wed, 21 Jan 2015 08:09:16 -0800 (PST) In-Reply-To: <54BFB10C.9080004@freebsd.org> References: <201501151532.t0FFWV2Y037455@svn.freebsd.org> <54BFB10C.9080004@freebsd.org> Date: Wed, 21 Jan 2015 08:09:16 -0800 X-Google-Sender-Auth: ZPygylGb3oQI4pCDTJd48UsJkLk Message-ID: Subject: Re: svn commit: r277213 - in head: share/man/man9 sys/kern sys/ofed/include/linux sys/sys From: Adrian Chadd To: Lawrence Stewart Content-Type: text/plain; charset=UTF-8 Cc: "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , Hans Petter Selasky X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jan 2015 16:09:17 -0000 On 21 January 2015 at 06:00, Lawrence Stewart wrote: > On 01/20/15 09:22, Adrian Chadd wrote: >> Yeah, it looks like you set c_cpu to timeout_cpu in >> _callout_init_locked(), but then you only handle the case of the CPU >> being changed in certain circumstances. You aren't handling the CPU >> being initialised when the callout is first added. >> >> And, callout_restart_async() calls callout_lock(), which calls >> CC_LOCK() on the callout CPU, which initially is zero. Then, it never >> seems to be 'moved' into the correct CPU, even though it's being >> called with a CPU id. >> >> So, if I am reading this all correctly, you aren't really handling >> multi CPU callwheels at all. ;) >> >> In my instance, I'm seeing quite a lot of lock contention between the >> userland threads, the network RX threads and the clock thread, all >> contending on a single callout wheel being used for TCP timers. One of >> the early goals for fixing up the RSS stuff in -HEAD was to make >> per-CPU (Well, per-RSS-bucket really) TCP timers not contend with the >> NIC RX processing path, and now it does. :( > > To clarify, you're seeing this with net.inet.tcp.per_cpu_timers=1? Yup, RSS enables that by default. -adrian