From owner-freebsd-current@FreeBSD.ORG Fri Sep 28 16:26:14 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 29E3D16A469; Fri, 28 Sep 2007 16:26:14 +0000 (UTC) (envelope-from Muhammad.Shafiq@neterion.com) Received: from owa.neterion.com (mx.neterion.com [72.1.205.142]) by mx1.freebsd.org (Postfix) with ESMTP id B5AA713C4BF; Fri, 28 Sep 2007 16:26:13 +0000 (UTC) (envelope-from Muhammad.Shafiq@neterion.com) X-MIMEOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Fri, 28 Sep 2007 12:14:07 -0400 Message-ID: <78C9135A3D2ECE4B8162EBDCE82CAD770245002D@nekter> In-Reply-To: <2a41acea0709241028q2503f12fp86eea2d32ac10aa7@mail.gmail.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: TX Multiqueue? Thread-Index: Acf+0Hb5oO1MMOqvQp+6JbpC3kH3PQDFbzCA References: <2a41acea0709221656n4aa62776y488c7f2da262c9f6@mail.gmail.com><46F614F2.4050402@freebsd.org> <2a41acea0709241028q2503f12fp86eea2d32ac10aa7@mail.gmail.com> From: "Muhammad Shafiq" To: "Jack Vogel" , "Kip Macy" Cc: Darren Reed , freebsd-net@freebsd.org, FreeBSD Current Subject: RE: TX Multiqueue? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Sep 2007 16:26:14 -0000 -----Original Message----- From: owner-freebsd-current@freebsd.org [mailto:owner-freebsd-current@freebsd.org] On Behalf Of Jack Vogel Sent: Monday, September 24, 2007 10:28 AM To: Kip Macy Cc: Darren Reed; freebsd-net@freebsd.org; FreeBSD Current Subject: Re: TX Multiqueue? On 9/23/07, Kip Macy wrote: > On 9/23/07, Darren Reed wrote: > > Kip Macy wrote: > > > My ethng branch supports multiple rx and tx queues. > > > > > > -Kip > > > > > > > What are your plans for how we use/manage/interact with the mutiple > > rx/tx queues? > > The rx hardware queue is determined by the hardware. Different > hardware allows for different policies. I just use the stock rss_hash > of a crc32 of the 4-tuple in cxgb. I've added a field to the pkthdr > which cxgb uses the least significant bits of to determine which > outbound queue to use. Its up to the upper layers to determine how to > set those bits. One of the changes that is required to take advantaged > of this is moving the queues into the driver. I've added a new > if_start function to ifnet to take advantage of this. I also have a > normal if_start function for backward compatibility. Yes, the queues in Oplin and Zoar are also a hardware feature, not just some software infrastructure. There are a number of different ways that it can be configured. I did not have some settled notion of how things should be managed but I would rather not have it be something done under the covers in the driver, a configurable stack option seems better to me. This needs to be done right or it will just be a hack, I don't know who the right parties are, it should not be just a one-person decision, those with a stake in this sort of thing should all be involved. [Muhammad Shafiq]=20 Based upon our experience, form other OS(s), the HW TX/RX queues are quite helpful in reducing CPU utilization (in MP machines). For example, XFRAME-I/II TX/RX queues, originally intended for "I/O Virtualization", can be affiliated to specific CPU(s); the resulting locality of reference improves cache hits and reduces lock contention, if any. The HW details of this mechanism can be found at the following link:=20 http://www.neterion.com/support/xframe_developer.html We would prefer to have a configurable, generic, mechanism to exploit the multiple TX/RX queues, now supported by NIC vendors. If possible, the mechanism should be extendable to fit into specific capability/requirement of HW vendors.=20 =20 Cheers, Jack _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"