From owner-freebsd-net@FreeBSD.ORG Wed Apr 16 21:59:31 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 221FD3C7 for ; Wed, 16 Apr 2014 21:59:31 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E27F7163B for ; Wed, 16 Apr 2014 21:59:30 +0000 (UTC) Received: by mail-ig0-f179.google.com with SMTP id hl10so1598668igb.12 for ; Wed, 16 Apr 2014 14:59:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=X/K6L4IFDsUEREQLWcNGlAgcV8/T7hfxruu12m5zdi4=; b=aE/+cpY/Wd0bT2NjxkC4LWquy+8ZEW757HtN0ZJPfqP7FIT5kRYWJVQriTXhn/RMt0 gv+ldWesVbHcbILlHNbVGFmkIGg9hJ01co9ckMvlwxXGyJ802KFuuMV8VCeMZ98eNTDC kF9sWlPpk47EjrqyPGHrGibbmGVqRJzR7Lvg8+hCsC+6QlGlNPH7TxjipzRnugp/mO44 fcLVoMGynXPeyDfXBOe77pDHxcd2XoMm6Kz/pXjrewyT16j9BhpGJ5cl/KkO7sB4Q1NO IjPZF3j4JBTUPs+Hnm9sAWdXqeTSEPSzDZEstgA+mVz4Zjr+tYvdHv9EyiRNtfK2C4YT gtXA== X-Received: by 10.50.111.135 with SMTP id ii7mr7430804igb.35.1397685570369; Wed, 16 Apr 2014 14:59:30 -0700 (PDT) Received: from [10.10.1.35] ([192.252.130.194]) by mx.google.com with ESMTPSA id r4sm1170602igh.1.2014.04.16.14.59.29 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Apr 2014 14:59:29 -0700 (PDT) Message-ID: <534EFD3E.3010006@gmail.com> Date: Wed, 16 Apr 2014 17:59:26 -0400 From: Karim Fodil-Lemelin User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> <53483B8B.7030409@freebsd.org> In-Reply-To: <53483B8B.7030409@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 21:59:31 -0000 Hi Julian, The code behaves as you described. The intent is clearly there and I can confirm the behavior is correct for incoming packets. At this point it looks like turning off net.isr.direct makes a difference too so I'm trying to understand the relationship between the system's queuing behavior and netgraph's. The only common ground I can find at this point is they both use the "swi1: net" thread. There may be a higher level design concern with forcing WRITER on callout items. It seems that, in some cases, it is desirable to prevent queuing packets if a timer event (ng_callout) can execute asynchronously, but I digress. Regards, Karim. On 11/04/2014 2:59 PM, Julian Elischer wrote: > disclaimer: I'm not looking at the code now.. I want to go to bed: :-) > > When I wrote that code, the idea was that even a direct node execution > should become a queuing operation if there was already something else > on the queue. so in that model packets were not supposed to get > re-ordered. does that not still work? > > Either that, or you need to explain the problem to me a bit better.. > > > On 4/10/14, 5:25 AM, Karim Fodil-Lemelin wrote: >> Hi, >> >> Below is a revised patch for this issue. It accounts for nodes or >> hooks that explicitly need to be queuing: >> >> @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, >> hook_p hook, int ticks, >> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >> return (ENOMEM); >> >> - item->el_flags |= NGQF_WRITER; >> + if ((node->nd_flags & NGF_FORCE_WRITER) || >> + (hook && (hook->hk_flags & HK_FORCE_WRITER))) >> + item->el_flags |= NGQF_WRITER; >> + else >> + item->el_flags |= NGQF_READER; >> + >> NG_NODE_REF(node); /* and one for the item */ >> NGI_SET_NODE(item, node); >> if (hook) { >> >> Regards, >> >> Karim. >> >> On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >>> Hi List, >>> >>> I'm calling out to the general wisdom ... I have seen an issue in >>> netgraph where, if called, a callout routine registered by >>> ng_callout() will trigger packet queuing inside the worklist of >>> netgraph since ng_callout() makes my node suddenly a WRITER node >>> (therefore non reentrant) for the duration of the call. >>> >>> So as soon as the callout function returns, all following packets >>> will get directly passed to the node again and when the ngintr >>> thread gets executed then only then will I get the queued packets. >>> This introduces out of order packets in the flow. I am using the >>> current patch below to solve the issue and I am wondering if there >>> is anything wrong with it (and maybe contribute back :): >>> >>> >>> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, >>> hook_p hook, int ticks, >>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>> return (ENOMEM); >>> >>> - item->el_flags |= NGQF_WRITER; >>> + item->el_flags = NGQF_READER; >>> NG_NODE_REF(node); /* and one for the item */ >>> NGI_SET_NODE(item, node); >>> if (hook) { >>> >>> >>> Best regards, >>> >>> Karim. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"