From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 28 23:23:55 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9575EBE6 for ; Sat, 28 Mar 2015 23:23:55 +0000 (UTC) Received: from mail-ig0-x236.google.com (mail-ig0-x236.google.com [IPv6:2607:f8b0:4001:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53EEAA25 for ; Sat, 28 Mar 2015 23:23:55 +0000 (UTC) Received: by igbud6 with SMTP id ud6so47731976igb.1 for ; Sat, 28 Mar 2015 16:23:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+ecSGIKrf2+x8teauMOrKE5qA6r6O2NYzeLt/m0YyKQ=; b=tSWEaNkHYIep5lE7fA+YcmQr0aXpONIamwwDdKT6/U+hSGNbzJD0uL4DoPv6HoB9zY pQYMDKunyit6efk8J1D6EPg0x2Sc7Z1RK0MPVZSEqDj2lAcQ+Lvfkv6xQ09S3jR+dqpn dxfPWVlnz6nOEpynRwgADpdru0pR4VYMnaqP6/sUome4zzOvdUo1DmeazUBbsnKysjpI S4pou+TPMeClxqAio9ULHlEGrCxhnX47qTMNn+dkP2Ylpi4eBeIAJxvU4FI2MQFkxQGl 2q4ADsy3QE4Z3ieXA9VgOJnEG/NL6NcHa0jCQyR2iKWbA5gwTucpnOrwdRWs/QYR2ux+ xQJA== MIME-Version: 1.0 X-Received: by 10.42.93.83 with SMTP id w19mr52896138icm.37.1427585034627; Sat, 28 Mar 2015 16:23:54 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sat, 28 Mar 2015 16:23:54 -0700 (PDT) In-Reply-To: <20150328230533.GI23643@zxy.spb.ru> References: <20150328192505.GD23643@zxy.spb.ru> <20150328194959.GE23643@zxy.spb.ru> <20150328201219.GF23643@zxy.spb.ru> <20150328221621.GG23643@zxy.spb.ru> <20150328224634.GH23643@zxy.spb.ru> <20150328230533.GI23643@zxy.spb.ru> Date: Sat, 28 Mar 2015 16:23:54 -0700 X-Google-Sender-Auth: 7pRGK4dG78tToBhyNTgYSlu5t90 Message-ID: Subject: Re: irq cpu binding From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2015 23:23:55 -0000 On 28 March 2015 at 16:05, Slawa Olhovchenkov wrote: > On Sat, Mar 28, 2015 at 03:49:48PM -0700, Adrian Chadd wrote: > >> You should totally join #bsdcode on efnet and ask me about it. :) > > I am totaly don't use IRC (last 20 years). > May be skype? Heh, IRC is better. There are more FreeBSD people in the channel. :) >> on RSS, this is what would happen: >> >> * ALL NICs RSS BUCKET 0 -> core 0 >> * ... >> * ALL NICs RSS BUCKET 7 -> core 7 > > My expirens: this is worse vs dedicated core (one core handeled only > one bucket of one NIC). The only reason(s) this becomes problematic is if things preempt other things on that CPU. Hopefully enough work gets done in each interrupt run - but, maybe the scheduler is doing something odd and interleaving all the supposedly-equivalent-ithreads based on what's blocking in locks and what isn't. It's worth digging into. Not only that, but I also do handle the case of fragments going to the "wrong" queue - then getting reassembled and reinjected back into the right RSS CPU. That way things are correctly in-order. > >> Now, that's not really 100% optimal for NUMA and multiple PCIe >> controllers, but we're not there yet. >> >> Hopefully I can twist/cajole navdeep @ chelsio to continue doing a >> little more RSS work so I can teach cxgbe/cxl about RSS configuration, >> but ixgbe, igb and ixl all do the above when RSS is enabled. > > Most part of my setup use cxgbe. Ok. Well, that (and other stuff) will happen at the speed of "adrian's doing this for fun as his home project", so if you/others would like to help out then please do. I'd like to get this stuff very much done and in -11 before it's released next year. -adrian