From owner-freebsd-net@FreeBSD.ORG Fri Sep 23 14:30:00 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9927F1065670; Fri, 23 Sep 2011 14:30:00 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 31B938FC08; Fri, 23 Sep 2011 14:29:59 +0000 (UTC) Received: by iadk27 with SMTP id k27so6319093iad.13 for ; Fri, 23 Sep 2011 07:29:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LGEDN+YZk877HD9MglGjgFHamDf/Q08U4XpQUt8x1Rw=; b=PPsq0PeJSqbILqBmnl/VDFT1z+vqJsPwlWIgN+7ASmj13qoCt2EKH00txbF9ueqWFn odPdgslSLQL26z2T/vHA00yJFYbY3DmvN/qdq0gUP4OUwaAFsIprNucavseB9W3x/W+f 4BwOH3gO6UlihUDfuKEA23UcmpysKxKogqwtE= MIME-Version: 1.0 Received: by 10.231.24.96 with SMTP id u32mr5595051ibb.61.1316786736255; Fri, 23 Sep 2011 07:05:36 -0700 (PDT) Received: by 10.231.12.138 with HTTP; Fri, 23 Sep 2011 07:05:36 -0700 (PDT) In-Reply-To: <8C8C6061-2BD8-4B38-843E-A0BA1218B773@dokukino.com> References: <1315221674.3092.282.camel@deadeye> <201109080834.11607.jhb@freebsd.org> <20110908184928.GA87872@hub.freebsd.org> <37419C45-4436-4738-851B-2B765BC2C60F@neville-neil.com> <1315529074.2804.63.camel@bwh-desktop> <8C8C6061-2BD8-4B38-843E-A0BA1218B773@dokukino.com> Date: Fri, 23 Sep 2011 17:05:36 +0300 Message-ID: From: Sami Halabi To: Takuya ASADA Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "support@pvd.citizen.co.jp" , "jfv@freebsd.org" , "owner-freebsd-net@freebsd.org" , John Baldwin , "freebsd-net@freebsd.org" Subject: Re: Adding Flow Director sysctls to ixgbe(4) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2011 14:30:00 -0000 Hi, i don't know what version of the driver currently on head, but on 8.1-R it isn't the latest driver on INTEL's site. my dual 10g card 82599EB didn't work well until i upgraded the driver(2.3.8) and recompiled the kernel. after that the card worked fine. Sami On Fri, Sep 23, 2011 at 4:59 PM, Takuya ASADA wrote: > Hi, > > On Sep 9, 2011, at 10:56 AM, owner-freebsd-net@freebsd.org wrote: > > > On Fri, Sep 09, 2011 at 01:44:34AM +0100, Ben Hutchings wrote: > >> On Thu, 2011-09-08 at 20:13 -0400, George Neville-Neil wrote: > >>> On Sep 8, 2011, at 14:49 , Navdeep Parhar wrote: > >>> > >>>> On Thu, Sep 08, 2011 at 08:34:11AM -0400, John Baldwin wrote: > >>>>> On Monday, September 05, 2011 7:21:12 am Ben Hutchings wrote: > >>>>>> On Mon, 2011-09-05 at 15:51 +0900, Takuya ASADA wrote: > >>>>>>> Hi, > >>>>>>> > >>>>>>> I implemented Ethernet Flow Director sysctls to ixgbe(4), here's a > detail: > >>>>>>> > >>>>>>> - Adding removing signature filter > >>>>>>> On linux version of ixgbe driver, it has ability to set/remove > perfect > >>>>>>> filter from userland using ethtool command. > >>>>>>> I implemented similar feature, but on sysctl, and not perfect > filter > >>>>>>> but signature filter(which means hash collision may occurs). > >>>>>> [...] > >>>>>> > >>>>>> Linux also has a generic interface to RX filtering and hashing > >>>>>> (ethtool_rxnfc) which ixgbe supports; wouldn't it be better for > FreeBSD > >>>>>> to support something like that? > >>>>> > >>>>> Some sort of shared interface might be nice. The cxgb(4) and > cxgbe(4) drivers > >>>>> both provide their own tools to manipulate filters, though they do > not > >>>>> provide explicit steering IIRC. > >>>> > >>>> Both of them can filter as well as steer (and the tools let you do > that). > >>>> cxgbe(4) can do a lot more (rewrite + switch, replicate, etc.) but > those > >>>> features are perhaps too specialized to be configurable via a general > >>>> purpose tool. > >>>> > >>>>> > >>>>> We would need to come up with some sort of standard interface > (ioctls?) for > >>>>> adding filters however. > >>>> > >>>> +1 for a standard interface. > >>>> > >>>> imho the kernel needs to be aware of the rx and tx queues of a NIC, > and > >>>> not just for steering. But that's a separate discussion. > >>>> > >>> > >>> Well I do think this is actually all of a part. Most of us realize by > now that > >>> high speed (e.g. 10G and higher) NICs only make sense if you can steer > traffic and > >>> pin queues to cores etc. > >> > >> Well, you can get way better than 1G performance without that. And for > >> routers, flow hashing may be fine. But for a host, of course, steering > >> packets properly can provide a major performance win. > >> > >> [...] > >>> What this means is that we have > >>> a failure of abstraction. Abstraction has a cost, and some of the > people who want > >>> access to low level queues are not interested in paying an extra > abstraction cost. > >> > >> Abstraction has a cost, but it's not necessarily that high compared to > >> rewriting a whole chunk of sockets code (especially if you don't > >> actually have the source code). > >> > >>> I think that some of the abstractions we need are tied up in the work > that Takuya did > >>> for SoC and some of it is in the work done by Luigi on netmap. I'd go > so far as to say > >>> that what we should do is try to combine those two pieces of code into > a set of > >>> low level APIs for programs to interact with high speed NICs. The one > thing most > >>> people do not talk about is extending our socket API to do two things > that I think would > >>> be a win for 80% of our users. If a socket, and also a kqueue, could > be pinned > >>> to a CPU as well as a NIC queue that should improve overall bandwidth > for a large > >>> number of our users. The API there is definitely an ioctl() and the > hard part is > >>> doing the tying together. To do this we need to also work out our low > level story. > >> > >> But it would be a lot nicer if this could be done automatically. Which > >> I believe it can - see the RFS and XPS features in Linux. > > > > rwatson@ has been working on "connection groups" (not sure what he calls > > his project) with a goal to improve the placement of work in the FreeBSD > > network stack. Some of the code is in the kernel but the parts that > > require closer cooperation with a NIC are not. > > It looks like reducing lock contention on inpcb lookup, does it even > effects the other part? (ex: CPU affinity > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > -- Sami Halabi Information Systems Engineer NMS Projects Expert