From owner-freebsd-net@FreeBSD.ORG Thu Apr 10 16:41:43 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2548F7C9 for ; Thu, 10 Apr 2014 16:41:43 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9D981849 for ; Thu, 10 Apr 2014 16:41:42 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id i50so4121625qgf.7 for ; Thu, 10 Apr 2014 09:41:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=gm51Cath61NqHXbQHfXhQ4G9BVRjDBMPUcGZ83ARmtc=; b=NrY4M4WReciNxsfCK/R3Pv/orpW09hSIIZtjjyubqibvxn7sFSUOKIwkBGGx2u6MqW wArsLRslNnP523EuG58NASwUZrXjT5l7hCcC5SxUU3K7v0Qp+/uumZbBzZn2yv7CIatf 5KzxUQrEDGT3bA7qnm2d245pJ2Qc1YzUiCMDxhyfTIzgg/2fTIXLH76dVwVB7xPgeUvP QtwegZc/u6AnvCacj+oD/6yTw+m1kEe4eevgyBzL/5weRQzfItgNlod32jhPaNuiUj4s aEHvjL1GVYcVhfu1n8Nvo2tIKywQneoiI03yfMlUia15b3ieRoIPx/zvrDQvAsdguHSV TYKw== MIME-Version: 1.0 X-Received: by 10.224.13.142 with SMTP id c14mr22439771qaa.76.1397148101966; Thu, 10 Apr 2014 09:41:41 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Thu, 10 Apr 2014 09:41:41 -0700 (PDT) In-Reply-To: <5346B63D.4080707@gmail.com> References: <53459C96.5040304@gmail.com> <5345BAE7.4010501@gmail.com> <5346B63D.4080707@gmail.com> Date: Thu, 10 Apr 2014 09:41:41 -0700 X-Google-Sender-Auth: G1ZuEJbYlPP__kwB-JFZNW79VH0 Message-ID: Subject: Re: Preventing ng_callout() timeouts to trigger packet queuing From: Adrian Chadd To: Karim Fodil-Lemelin Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 16:41:43 -0000 Sweet! I'll ask around and see if anyone netmap clued can review. :) -a On 10 April 2014 08:18, Karim Fodil-Lemelin wrote: > Hi, > > By the way this change has opened the gates to greater performance for us > when using ng_callout() inside nodes. In some cases we see twice as much pps > since packets are direct dispatched instead of being queued in software > interrupts threads (swi*). > > Thanks, > > Karim > > PS: I did file a PR : kern/188421 > > > On 09/04/2014 9:09 PM, Adrian Chadd wrote: >> >> Hi! >> >> Would you mind filing a PR for this? >> >> www.freebsd.org/send-pr.html >> >> That way it won't (hopefully!) get lost. >> >> Thanks! >> >> >> -a >> >> >> On 9 April 2014 14:25, Karim Fodil-Lemelin >> wrote: >>> >>> Hi, >>> >>> Below is a revised patch for this issue. It accounts for nodes or hooks >>> that >>> explicitly need to be queuing: >>> >>> @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p >>> hook, int ticks, >>> >>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>> return (ENOMEM); >>> >>> - item->el_flags |= NGQF_WRITER; >>> + if ((node->nd_flags & NGF_FORCE_WRITER) || >>> + (hook && (hook->hk_flags & HK_FORCE_WRITER))) >>> >>> + item->el_flags |= NGQF_WRITER; >>> + else >>> >>> + item->el_flags |= NGQF_READER; >>> + >>> NG_NODE_REF(node); /* and one for the item */ >>> NGI_SET_NODE(item, node); >>> if (hook) { >>> >>> Regards, >>> >>> Karim. >>> >>> >>> On 09/04/2014 3:16 PM, Karim Fodil-Lemelin wrote: >>>> >>>> Hi List, >>>> >>>> I'm calling out to the general wisdom ... I have seen an issue in >>>> netgraph >>>> where, if called, a callout routine registered by ng_callout() will >>>> trigger >>>> packet queuing inside the worklist of netgraph since ng_callout() makes >>>> my >>>> node suddenly a WRITER node (therefore non reentrant) for the duration >>>> of >>>> the call. >>>> >>>> So as soon as the callout function returns, all following packets will >>>> get >>>> directly passed to the node again and when the ngintr thread gets >>>> executed >>>> then only then will I get the queued packets. This introduces out of >>>> order >>>> packets in the flow. I am using the current patch below to solve the >>>> issue >>>> and I am wondering if there is anything wrong with it (and maybe >>>> contribute >>>> back :): >>>> >>>> >>>> @@ -3632,7 +3632,7 @@ ng_callout(struct callout *c, node_p node, hook_p >>>> hook, int ticks, >>>> if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) >>>> return (ENOMEM); >>>> >>>> - item->el_flags |= NGQF_WRITER; >>>> + item->el_flags = NGQF_READER; >>>> NG_NODE_REF(node); /* and one for the item */ >>>> NGI_SET_NODE(item, node); >>>> if (hook) { >>>> >>>> >>>> Best regards, >>>> >>>> Karim. >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >>> >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"