From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 15:18:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6FE7BF8 for ; Thu, 10 Apr 2014 15:18:26 +0000 (UTC) Received: from mail-ie0-x22c.google.com (mail-ie0-x22c.google.com [IPv6:2607:f8b0:4001:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 916CA1E4F for ; Thu, 10 Apr 2014 15:18:26 +0000 (UTC) Received: by mail-ie0-f172.google.com with SMTP id as1so4098262iec.17 for ; Thu, 10 Apr 2014 08:18:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=OWS1+6Eu/xpEtv00R0v8PyUWCrVvzGf1mpGGOJ4dILw=; b=QZ3jCcBus/b5qaiyJbM49RB8uIBY2RF/tHWxClKy9ZQdIiuffZZiSdp26OTmGrKCIH 3qPDIOImhW0v/M9h4Wg9PgkxlfCVMQ/s9T5WtrSo4Ri1dWvwWyML1sCdin8VwV/Cl03N +x4EUKX5NE036pVRpQccDIzuI9PrZNPlNqIpgAIhXsGEo9Ju6LoOFsAqWnXAzbtMNDEV APC7FL+a7HwMNq4qGAwHsLuMX4f61/TXJL8bTfBS/GiPanChC/aDAyHdK8mC+546XtQu f03vDW4/q9RFZwlXGRCCYyEkuONr3R1gbeDpYYdbzbUnTDEOl0Ctpcg5EEvaD5y3h6NL zYyw== X-Received: by 10.50.49.44 with SMTP id r12mr11225810ign.41.1397143105899; Thu, 10 Apr 2014 08:18:25 -0700 (PDT) Received: from [10.10.1.35] ([192.252.130.194]) by mx.google.com with ESMTPSA id ix1sm20273631igc.15.2014.04.10.08.18.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 08:18:25 -0700 (PDT) Message-ID: <5346B63D.4080707@gmail.com> Date: Thu, 10 Apr 2014 11:18:21 -0400 From: Karim Fodil-Lemelin User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 15:18:26 -0000 Hi, By the way this change has opened the gates to greater performance for us when using ng_callout() inside nodes. In some cases we see twice as much pps since packets are direct dispatched instead of being queued in software interrupts threads (swi*). Thanks, Karim PS: I did file a PR : kern/188421 On 09/04/2014 9:09 PM, Adrian Chadd wrote: > Hi! > > Would you mind filing a PR for this? > > www.freebsd.org/send-pr.html > > That way it won't (hopefully!) get lost. > > Thanks! > > > -a > > > On 9 April 2014 14:25, Karim Fodil-Lemelin wrote: >> Hi, >> >> Below is a revised patch for this issue. It accounts for nodes or hooks that >> explicitly need to be queuing: >> >> @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p >> hook, int ticks, >> >> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >> return (ENOMEM); >> >> - item->el_flags |= NGQF_WRITER; >> + if ((node->nd_flags & NGF_FORCE_WRITER) || >> + (hook && (hook->hk_flags & HK_FORCE_WRITER))) >> >> + item->el_flags |= NGQF_WRITER; >> + else >> >> + item->el_flags |= NGQF_READER; >> + >> NG_NODE_REF(node); /* and one for the item */ >> NGI_SET_NODE(item, node); >> if (hook) { >> >> Regards, >> >> Karim. >> >> >> On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >>> Hi List, >>> >>> I'm calling out to the general wisdom ... I have seen an issue in netgraph >>> where, if called, a callout routine registered by ng_callout() will trigger >>> packet queuing inside the worklist of netgraph since ng_callout() makes my >>> node suddenly a WRITER node (therefore non reentrant) for the duration of >>> the call. >>> >>> So as soon as the callout function returns, all following packets will get >>> directly passed to the node again and when the ngintr thread gets executed >>> then only then will I get the queued packets. This introduces out of order >>> packets in the flow. I am using the current patch below to solve the issue >>> and I am wondering if there is anything wrong with it (and maybe contribute >>> back :): >>> >>> >>> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, hook_p >>> hook, int ticks, >>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>> return (ENOMEM); >>> >>> - item->el_flags |= NGQF_WRITER; >>> + item->el_flags = NGQF_READER; >>> NG_NODE_REF(node); /* and one for the item */ >>> NGI_SET_NODE(item, node); >>> if (hook) { >>> >>> >>> Best regards, >>> >>> Karim. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"