From owner-freebsd-bugs@FreeBSD.ORG Thu Apr 10 13:30:00 2014 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B01CC11F for ; Thu, 10 Apr 2014 13:30:00 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C54C12ED for ; Thu, 10 Apr 2014 13:30:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s3ADU0ma085505 for ; Thu, 10 Apr 2014 13:30:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s3ADU06W085504; Thu, 10 Apr 2014 13:30:00 GMT (envelope-from gnats) Resent-Date: Thu, 10 Apr 2014 13:30:00 GMT Resent-Message-Id: <201404101330.s3ADU06W085504@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Karim Fodil-Lemelin Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4995DFD7 for ; Thu, 10 Apr 2014 13:25:03 +0000 (UTC) Received: from cgiserv.freebsd.org (cgiserv.freebsd.org [IPv6:2001:1900:2254:206a::50:4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1C00112AF for ; Thu, 10 Apr 2014 13:25:03 +0000 (UTC) Received: from cgiserv.freebsd.org ([127.0.1.6]) by cgiserv.freebsd.org (8.14.8/8.14.8) with ESMTP id s3ADP2Yk055141 for ; Thu, 10 Apr 2014 13:25:02 GMT (envelope-from nobody@cgiserv.freebsd.org) Received: (from nobody@localhost) by cgiserv.freebsd.org (8.14.8/8.14.8/Submit) id s3ADP2Ll055136; Thu, 10 Apr 2014 13:25:02 GMT (envelope-from nobody) Message-Id: <201404101325.s3ADP2Ll055136@cgiserv.freebsd.org> Date: Thu, 10 Apr 2014 13:25:02 GMT From: Karim Fodil-Lemelin To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Subject: kern/188421: ng_callout() timeouts trigger packets queuing and out of order packets X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 13:30:00 -0000 >Number: 188421 >Category: kern >Synopsis: ng_callout() timeouts trigger packets queuing and out of order packets >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Apr 10 13:30:00 UTC 2014 >Closed-Date: >Last-Modified: >Originator: Karim Fodil-Lemelin >Release: FreeBSD 7, FreeBSD 9 >Organization: XipLink Inc. >Environment: FreeBSD pluto 7.4-RELEASE FreeBSD 7.4-RELEASE #0: Mon Apr 15 20:48:02 UTC 2013 kfl@pluto:/usr/src/sys/amd64/compile/SIMULATOR amd64 >Description: I have seen an issue in netgraph where, if called, a callout routine registered by ng_callout() will trigger packet queuing inside the worklist of netgraph since ng_callout() makes my node suddenly a WRITER node (therefore non reentrant) for the duration of the call. So as soon as the callout function returns, all following packets will get directly passed to the node again and when the ngintr thread gets executed then only then will I get the queued packets. This introduces out of order packets in the flow. I am using the current patch below to solve the issue and I am wondering if there is anything wrong with it (and maybe contribute back :). >How-To-Repeat: Have a constant stream of packets sent to a netgraph node. The node must at some point have registered a timeout function. Lets have it exceute every 10 miliseconds. Track the order (looking at ip id) of the packets as they come out. You should see out of order packets whenever the callout function is called. >Fix: Use the provided patch. Patch attached with submission follows: @@ -3632,7 +3632,12 @@ ng_callout(struct callout *c, node_p node, hook_p hook, int ticks, if ((item = ng_alloc_item(NGQF_FN, NG_NOFLAGS)) == NULL) return (ENOMEM); - item->el_flags |= NGQF_WRITER; + if ((node->nd_flags & NGF_FORCE_WRITER) || + (hook && (hook->hk_flags & HK_FORCE_WRITER))) + item->el_flags |= NGQF_WRITER; + else + item->el_flags |= NGQF_READER; + NG_NODE_REF(node); /* and one for the item */ NGI_SET_NODE(item, node); if (hook) { >Release-Note: >Audit-Trail: >Unformatted: