From owner-freebsd-drivers@FreeBSD.ORG Tue Sep 20 16:28:51 2011 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 179AF106564A; Tue, 20 Sep 2011 16:28:51 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-qw0-f44.google.com (mail-qw0-f44.google.com [209.85.216.44]) by mx1.freebsd.org (Postfix) with ESMTP id 9D74B8FC14; Tue, 20 Sep 2011 16:28:50 +0000 (UTC) Received: by qwb8 with SMTP id 8so1865092qwb.3 for ; Tue, 20 Sep 2011 09:28:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Sqx60PAa/QkmWBOvji3NtghHdDLO8ARf6F6j8nv9VZs=; b=nSBM5/CSSUQN6LCKOvTMyOfCpzSsvVODNCsOMtImIZ9cKiWFNaCJZf9wgZUVHPP1jh 8o6u14al6jZOLMhsKYBLvpJ4US0X0ImPx2Dh15rWkM4zikdAEKg4yi3oI4a6cY0g+dyN 99LfOOzNtUh6uDrATmgHm0aMlYYEbwFQ6/HIQ= MIME-Version: 1.0 Received: by 10.52.89.36 with SMTP id bl4mr1031544vdb.10.1316536129994; Tue, 20 Sep 2011 09:28:49 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.52.113.202 with HTTP; Tue, 20 Sep 2011 09:28:49 -0700 (PDT) In-Reply-To: <6C998B08-9053-49F4-B918-B88331AE4D17@gmail.com> References: <201109190813.37817.jhb@freebsd.org> <6C998B08-9053-49F4-B918-B88331AE4D17@gmail.com> Date: Tue, 20 Sep 2011 18:28:49 +0200 X-Google-Sender-Auth: ttwbndhl2uPTTKHCAZgfh89pBBw Message-ID: From: "K. Macy" To: Naresh Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Robert Watson , freebsd-drivers@freebsd.org Subject: Re: Multiple TX queue support in nic driver X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Sep 2011 16:28:51 -0000 (added -drivers back to the CC for your mail's educational value) On Tue, Sep 20, 2011 at 5:02 PM, Naresh wrote: > Hi Macy, > > Thanks a lot for the detailed explanation, Now I understand why the ring = buffer is used. > > In Linux all these things are maintained in the stack, is there any advan= tage of FreeBSD moving these things in to drivers? I doubt it. When I ported Chelsio's T3 10G Linux driver to FreeBSD (cxgb) FreeBSD had no support for multiple transmit queues. At that time all packets were relayed to the driver by a per-driver instance linked list (IFQ) and then calling ifp->if_start(ifp). After some discussion with others I extended the API with if_transmit to allow the caller to do direct transmit if possible or defer transmission in a driver specific fashion. By adding buf_ring I provided drivers with a much faster queueing mechanism without enforcing any sort of policy. At the time I hadn't done much work in the network stack beyond updating some of the wireless drivers so my only real objective was getting the network stack out of the way of the driver. At this point we've established a sufficiently consistent driver programming model with if_transmit that it would make sense to move a lot of that out of the driver and in to the stack. > During module attach, Linux drivers report number of TX queues they suppo= rt and based on that value stack creates that many software queues. These a= llocated queues reference is stored in netdev structure, so that they can b= e accessible by both driver and stack. In XMIT routine, skbuff indicates wh= ich queue to use and based on that value packet is transmitted on appropria= te hardware TX queue. > > Both the approaches of the OS's are similar, Only difference is in Linux = these are taken care by stack. I think that is the right approach, it simply hasn't reached the front of anyone's priority queue. For my non-work hours I'm busy with fleshing out my user level network stack, and the others who might do the work are themselves busier still. > Once again thanks a lot for you help. > Cheers