From owner-freebsd-net@FreeBSD.ORG  Wed Apr  9 19:16:43 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id C5F4615D
 for <freebsd-net@freebsd.org>; Wed,  9 Apr 2014 19:16:43 +0000 (UTC)
Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com
 [IPv6:2607:f8b0:4001:c03::236])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 941B3134E
 for <freebsd-net@freebsd.org>; Wed,  9 Apr 2014 19:16:43 +0000 (UTC)
Received: by mail-ie0-f182.google.com with SMTP id y20so2921334ier.13
 for <freebsd-net@freebsd.org>; Wed, 09 Apr 2014 12:16:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=message-id:date:from:user-agent:mime-version:to:subject
 :content-type:content-transfer-encoding;
 bh=AasCX+VtaaxIZnebvRJR+HeS9PXIrGk7nwnFT/1Lgak=;
 b=PxyqdtCUmA0lVDigCtCGyEyCPKbOEwf/XBua3uhv6jDOhb1QKtz1zaw+j5y+1cnpJG
 +Ri8e7F48psuB45fmj9DdrwmzkgcAwq/nenfr3OC1kfcMUix3o3yVZGEjrUvrtiXqx0k
 bMEq+cINv02Ss0PnRoOsiIfKRiYLcNJbYjmjx7Sg56aXWfSJtC8TfahH+IxJv0Zoi2cj
 P+wuN2XGL3P/WPheDPwTMqZK0wxafFUsOFpXy3z8VrzR2qBL12GFDlWihrOIVydSiosU
 qWnizl63mI/zEAWGifJgp9GmV0NtAGoT/x5YO+kERZf3J41cLplfnmBrtT8isPCLBH6O
 +iWw==
X-Received: by 10.50.36.66 with SMTP id o2mr11906269igj.24.1397071002890;
 Wed, 09 Apr 2014 12:16:42 -0700 (PDT)
Received: from [10.10.1.35] ([192.252.130.194])
 by mx.google.com with ESMTPSA id vu3sm7201462igc.6.2014.04.09.12.16.41
 for <freebsd-net@freebsd.org>
 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
 Wed, 09 Apr 2014 12:16:42 -0700 (PDT)
Message-ID: <53459C96.5040304@gmail.com>
Date: Wed, 09 Apr 2014 15:16:38 -0400
From: Karim Fodil-Lemelin <fodillemlinkarim@gmail.com>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:24.0) Gecko/20100101 Thunderbird/24.4.0
MIME-Version: 1.0
To: freebsd-net@FreeBSD.org
Subject: Preventing ng_callout() timeouts to trigger packet queuing
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Apr 2014 19:16:43 -0000

Hi List,

I'm calling out to the general wisdom ... I have seen an issue in 
netgraph where, if called, a callout routine registered by ng_callout() 
will trigger packet queuing inside the worklist of netgraph since 
ng_callout() makes my node suddenly a WRITER node (therefore non 
reentrant) for the duration of the call.

So as soon as the callout function returns, all following packets will 
get directly passed to the node again and when the ngintr thread gets 
executed then only then will I get the queued packets. This introduces 
out of order packets in the flow. I am using the current patch below to 
solve the issue and I am wondering if there is anything wrong with it 
(and maybe contribute back :):


@@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, hook_p 
hook, int ticks,
         if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL)
                 return (ENOMEM);

-       item->el_flags |= NGQF_WRITER;
+       item->el_flags = NGQF_READER;
         NG_NODE_REF(node);              /* and one for the item */
         NGI_SET_NODE(item, node);
         if (hook) {


Best regards,

Karim.