From owner-freebsd-net@freebsd.org Thu Nov 5 09:51:28 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AEC23A26BF3 for ; Thu, 5 Nov 2015 09:51:28 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 66A41188B for ; Thu, 5 Nov 2015 09:51:28 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 135931FE023; Thu, 5 Nov 2015 10:51:23 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Rasool Al-Saadi , "freebsd-net@freebsd.org" References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> From: Hans Petter Selasky Message-ID: <563B2703.5080402@selasky.org> Date: Thu, 5 Nov 2015 10:53:07 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> Content-Type: multipart/mixed; boundary="------------010409060107000901010706" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Nov 2015 09:51:28 -0000 This is a multi-part message in MIME format. --------------010409060107000901010706 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 11/05/15 00:44, Rasool Al-Saadi wrote: > > On Wednesday, 4 November 2015 12:34 AM, Hans Petter Selasky wrote: >> On 11/03/15 14:14, Rasool Al-Saadi wrote: >>> Does anyone have thoughts on what we can test next to narrow down the >> root-cause of these unusual timing jumps? >> >> You might also want to test the "projects/hps_head" branch, which uses a bit >> different callout implementation. > > Thanks Hans for your suggestion. > I have tried "projects/hps_head" branch and the result is better (number of spikes is less than in the master branch). However, the problem still exists on the same timer interrupt frequencies (>3000 in my case). You can see in this graph https://goo.gl/photos/C2Mqx4xhMQuzxWnz6 the RTT spikes still there. > > Do you have any further suggestions? > Hi, If the jitter is in the xx milliseconds range, like your graph shows, my guess is that td_owepreemt is not set when we return from the dummynet() callback. See attached patch. Else you might want to try to remove the C_HARDCLOCK flag from callout_reset_sbt() in ip_dummynet.c. --HPS --------------010409060107000901010706 Content-Type: text/x-patch; name="ip_dummynet.c.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ip_dummynet.c.diff" Index: sys/netpfil/ipfw/ip_dummynet.c =================================================================== --- sys/netpfil/ipfw/ip_dummynet.c (revision 290134) +++ sys/netpfil/ipfw/ip_dummynet.c (working copy) @@ -78,6 +78,8 @@ static struct task dn_task; static struct taskqueue *dn_tq = NULL; +#include + static void dummynet(void *arg) { @@ -84,6 +86,9 @@ (void)arg; /* UNUSED */ taskqueue_enqueue_fast(dn_tq, &dn_task); + + /* ensure taskqueue gets running */ + sched_preempt(curthread); } void --------------010409060107000901010706--