From owner-freebsd-net@FreeBSD.ORG  Mon Nov 12 19:16:03 2012
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
 by hub.freebsd.org (Postfix) with ESMTP id 8FCBC473
 for <freebsd-net@freebsd.org>; Mon, 12 Nov 2012 19:16:03 +0000 (UTC)
 (envelope-from fodillemlinkarim@gmail.com)
Received: from mail-ia0-f182.google.com (mail-ia0-f182.google.com
 [209.85.210.182])
 by mx1.freebsd.org (Postfix) with ESMTP id 512688FC08
 for <freebsd-net@freebsd.org>; Mon, 12 Nov 2012 19:16:02 +0000 (UTC)
Received: by mail-ia0-f182.google.com with SMTP id x2so85832iad.13
 for <freebsd-net@freebsd.org>; Mon, 12 Nov 2012 11:16:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=message-id:date:from:user-agent:mime-version:to:subject
 :content-type:content-transfer-encoding;
 bh=/L2GN+b19cWYplZHr/34e8VOaKqJfMh4TPJrW3NZYVM=;
 b=z8asEgO7x6WkqjNWwY9Gni2PJo5aBZnderBWDPGHlQLAfRP27eKPhKl1MEsCBl8rNU
 ird4YRN6WPNzIVMgbrAMdZH8jlj+gICBL5lTQ3DxsoIfOTbCmvi5qJsVGWa08HJTmw3q
 FbOjLW/8ptY3x4O6r/0GhxjHxcpGeP0cH+2TycpvelxzR42I9UWOscOXozxF4z2eYRXA
 PH+1ZSL6qTEohIuCyyeFWatr8F+Cx6ZBYuescIqrAZUc1r0lOqViE5plW0RDA3qBf437
 2rwxssNBU/zZT7SuObVyktu3bnL8xkoT0lIRSINlQ2sHs42/AWNs9FcEjc5Qyod98/l6
 y0kw==
Received: by 10.50.220.199 with SMTP id py7mr8814416igc.34.1352747762324;
 Mon, 12 Nov 2012 11:16:02 -0800 (PST)
Received: from [192.168.1.71] ([208.85.112.101])
 by mx.google.com with ESMTPS id uz1sm6917845igb.16.2012.11.12.11.16.01
 (version=SSLv3 cipher=OTHER); Mon, 12 Nov 2012 11:16:02 -0800 (PST)
Message-ID: <50A14AEC.2040303@gmail.com>
Date: Mon, 12 Nov 2012 14:15:56 -0500
From: Karim <fodillemlinkarim@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:11.0) Gecko/20120410 Thunderbird/11.0.1
MIME-Version: 1.0
To: freebsd-net@freebsd.org
Subject: em/igb if_transmit (drbr) and ALTQ
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Nov 2012 19:16:03 -0000

Hi all,

I have been following the current discussions on igb tx/rx locking with 
great interest and I though I would point out (as it was before pointed 
out in kern/138392) that any driver setting if_start to NULL and using 
if_transmit (multi-queue or not) will effectively break ALTQ.

The reason for this can be found in tbr_timeout() in altq_subr.c where 
the token bucket regulator goes through all ALTQ registered interface 
and tries to 'kick' them into sending through if_start().

tbr_timeout() :

...

         for (ifp = TAILQ_FIRST(&V_ifnet); ifp;
             ifp = TAILQ_NEXT(ifp, if_list)) {
             /* read from if_snd unlocked */
             if (!TBR_IS_ENABLED(&ifp->if_snd))
                 continue;
             active++;
             if (!IFQ_IS_EMPTY(&ifp->if_snd) &&
                 ifp->if_start != NULL)                     /* if_start 
is NULL if if_transmit is used in em/igb driver */
                 (*ifp->if_start)(ifp);
         }

...

As you can see if_start is NULL on those new multi-queue enabled drivers 
which has for net effect to 'break' ALTQ's token bucket regulator.

I am writing this because I am interested in your comments on how this 
can be fixed properly looking forward. The whole range of suggestions; 
from 'don't compile with EM_MULTIQUEUE defined' to 'here is how you can 
make ALTQ use drbr' will help.

Thanks you,

Karim.