From owner-svn-src-all@FreeBSD.ORG Tue Jan 20 05:22:44 2015 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADE01CE4; Tue, 20 Jan 2015 05:22:44 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 520F2304; Tue, 20 Jan 2015 05:22:44 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id l15so11247798wiw.0; Mon, 19 Jan 2015 21:22:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EqLmoYzCG7hgRy3y1Rdnst7MlwIYzBzeZB/yf18XSz8=; b=DOBg0I0F2cnuZ1shIyPVWgo9rrySXqSm1H9WRUyDNMTN9olyYqjREfi/B+xEMqSiZg g+wjCf1TuEquepJx5rQpPL4WNtAuNgV/LKjJKvrSc4RdQk3RoXJX78wuBErP2Ccahsjq 7hsX8FUwP745U8lOGbBFE8FztB0QRWW6Ou9QJZn7UFu5ftsMUmdPrV3mo4doTMDmREo0 S/K3BYvcvJA515XxCLP9fnyo24piV2/V5n0GbhlA24vKWXRF6q5XEeHa/0yZD7F5xnyk oKuKBMhsj5eC82iz4YHNaHiO45uiUYBwzB2Jlt/RoSrdCrCtDwL2tjJfFgj8WZykNax8 0bPA== MIME-Version: 1.0 X-Received: by 10.194.108.9 with SMTP id hg9mr66751006wjb.68.1421731362795; Mon, 19 Jan 2015 21:22:42 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.41.136 with HTTP; Mon, 19 Jan 2015 21:22:42 -0800 (PST) In-Reply-To: <54BDE58F.3070909@selasky.org> References: <201501151532.t0FFWV2Y037455@svn.freebsd.org> <54BDD9E1.6090505@selasky.org> <54BDE58F.3070909@selasky.org> Date: Mon, 19 Jan 2015 21:22:42 -0800 X-Google-Sender-Auth: t5EneEEpnWlTRAvgbYf0SIPqb8Y Message-ID: Subject: Re: svn commit: r277213 - in head: share/man/man9 sys/kern sys/ofed/include/linux sys/sys From: Adrian Chadd To: Hans Petter Selasky Content-Type: text/plain; charset=UTF-8 Cc: "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jan 2015 05:22:44 -0000 On 19 January 2015 at 21:20, Hans Petter Selasky wrote: > On 01/20/15 06:04, Adrian Chadd wrote: >> >> On 19 January 2015 at 20:30, Hans Petter Selasky wrote: >>> >>> On 01/19/15 22:59, Adrian Chadd wrote: >>>> >>>> >>>> Hi, >>>> >>>> Would you please check what the results of this are with CPU specific >>>> callwheels? >>>> >>>> I'm doing some 10+ gig traffic testing on -HEAD with RSS enabled (on >>>> ixgbe) and with this setup, the per-CPU TCP callwheel stuff is >>>> enabled. But all the callwheels are now back on clock(0) and so is the >>>> lock contention. :( >>>> >>>> Thanks, >>>> >>> >>> Hi, >>> >>> Like stated in the manual page, callout_reset_curcpu/on() does not work >>> with >>> MPSAFE callouts any more! >> >> >> Hm! >> > > Hi Adrian, > >> How many places in the kernel did you leave like this? :P > > > :-) > >> >> I mean, I'm glad to have stuff be forced to be cleaned up, but you >> didn't even leave a KASSERT or a debug warning that something >> unsupported is being done. I'm sure I'm not going to be the first >> person to be caught out like this. > > > MPSAFE is still valid and fully useable and can be used with > callout_reset_curcpu/on(), but the callout CPU will remain at zero. > There is no need for a KASSERT() yet. Right, but people won't know that their callout won't be scheduled on the CPU they've asked for and there's no way to get notified of that. So something debug-y may be useful, even to make it easy to track down situations where sometimes it succeeds and sometimes it fails (because well, there's lots of places in the kernel where locking is .. suboptimal.) >> >>> You need to use callout_init_{mtx,rm,rw} and remove the custom locking >>> inside the callback in the TCP stack to get it working like before! >> >> >> Would you please give me a hand with this? I've sunk a lot of (unpaid, >> personal) spare time into getting the RSS stuff into shape and now a >> lot of it just plainly doesn't do anything. :( > > > I'll send you a patch in an hours time from now for 11-current. This should > be fairly trivial and then you can test and review it! Sweet, thanks. I'l test it, but anything that changes the locking to TCP is going to need a more thorough review. The "there be dragons" disclaimer is appropriate. :) -adrian