Date: Fri, 6 Nov 2015 09:44:20 +0100 From: Hans Petter Selasky <hps@selasky.org> To: Rasool Al-Saadi <ralsaadi@swin.edu.au>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Timing issue with Dummynet on high kernel timer interrupt Message-ID: <563C6864.2090907@selasky.org> In-Reply-To: <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------010202050408090608050403 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit On 11/06/15 01:08, Rasool Al-Saadi wrote: > > On Thursday, 5 November 2015 8:53 PM, Hans Petter Selasky wrote: >> >> On 11/05/15 00:44, Rasool Al-Saadi wrote: >>> >>> On Wednesday, 4 November 2015 12:34 AM, Hans Petter Selasky wrote: >>>> On 11/03/15 14:14, Rasool Al-Saadi wrote: >>>>> Does anyone have thoughts on what we can test next to narrow down >>>>> the >>>> root-cause of these unusual timing jumps? >>>> >>>> You might also want to test the "projects/hps_head" branch, which >>>> uses a bit different callout implementation. >>> >>> Thanks Hans for your suggestion. >>> I have tried "projects/hps_head" branch and the result is better (number >> of spikes is less than in the master branch). However, the problem still exists >> on the same timer interrupt frequencies (>3000 in my case). You can see in >> this graph https://goo.gl/photos/C2Mqx4xhMQuzxWnz6 the RTT spikes still >> there. >>> >>> Do you have any further suggestions? >>> >> >> Hi, >> >> If the jitter is in the xx milliseconds range, like your graph shows, my guess is >> that td_owepreemt is not set when we return from the dummynet() >> callback. See attached patch. > > The patch doesn't solve the problem. I tried it on the master and hps branches. > >> Else you might want to try to remove the C_HARDCLOCK flag from >> callout_reset_sbt() in ip_dummynet.c. > > Removing C_HARDCLOCK reduces the problem but doesn't solve it completely. However, removing C_DIRECT_EXEC instead solves the problem (but occasionally very small spike(s) appears in high hz values). > I mentioned in my first email that removing these flags makes the issue to disappear. But what the effects of removing these flags? If it cause timing issue to Dummynet, why we should use them? > Hi, The C_DIRECT_EXEC flag reduces task switching overhead, that you don't have to wakeup a thread to wakeup the dummynet worker thread. It affects timing. Here is one more patch you can try. See attachment. It restarts the timeout from within the timer callback, instead of when the worker thread is executing. --HPS --------------010202050408090608050403 Content-Type: text/x-patch; name="ipfw_reschedule.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ipfw_reschedule.diff" Index: sys/netpfil/ipfw/ip_dn_io.c =================================================================== --- sys/netpfil/ipfw/ip_dn_io.c (revision 290134) +++ sys/netpfil/ipfw/ip_dn_io.c (working copy) @@ -712,7 +712,7 @@ } DN_BH_WUNLOCK(); - dn_reschedule(); + //dn_reschedule(); if (q.head != NULL) dummynet_send(q.head); CURVNET_RESTORE(); Index: sys/netpfil/ipfw/ip_dummynet.c =================================================================== --- sys/netpfil/ipfw/ip_dummynet.c (revision 290134) +++ sys/netpfil/ipfw/ip_dummynet.c (working copy) @@ -84,6 +84,7 @@ (void)arg; /* UNUSED */ taskqueue_enqueue_fast(dn_tq, &dn_task); + dn_reschedule(); } void --------------010202050408090608050403--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?563C6864.2090907>