From owner-freebsd-net@freebsd.org Fri Nov 6 15:02:59 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78284A28F7F for ; Fri, 6 Nov 2015 15:02:59 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 39CBA14DC for ; Fri, 6 Nov 2015 15:02:58 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 4410A1FE023; Fri, 6 Nov 2015 16:02:55 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Luigi Rizzo References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> <563C786C.1050305@selasky.org> Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" From: Hans Petter Selasky Message-ID: <563CC186.9000807@selasky.org> Date: Fri, 6 Nov 2015 16:04:38 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 15:02:59 -0000 On 11/06/15 11:08, Luigi Rizzo wrote: > On Fri, Nov 6, 2015 at 10:52 AM, Hans Petter Selasky wrote: >> On 11/06/15 09:50, Luigi Rizzo wrote: >>> >>> On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky >>> wrote: > ... >>>> Hi, >>>> >>>> The C_DIRECT_EXEC flag reduces task switching overhead, that you don't >>>> have >>>> to wakeup a thread to wakeup the dummynet worker thread. It affects >>>> timing. >>> >>> >>> Hans, >>> thanks for the explanation. >>> >>> Can you clarify the behaviour of C_DIRECT_EXEC ? >>> Does this mean that the task is run within some common >>> thread instead of a dedicated one ? >> >> >> Hi Luigi, >> >> C_DIRECT_EXEC means that the timer callback is executed directly from the >> fast interrupt filter of the timer or IPI. >> >>> >>> If so, for this type of task (dummynet may run at high rate >>> and use a significant amount of cpu time) it may be a good >>> idea to remove C_DIRECT_EXEC altogether. >> >> >> The ipfw dummynet code is not run from the timer callback. It is run from a >> taskqueue. I suspect there is likely a bug somewhere. At the moment it is >> not clear to me if there is a bug in the callout subsystem, that the when >> the timer is started with 1 tick delay it doesn't kick in until after 50ms >> or so at HZ=4000. Or if the dummynet's task is doing a lot of work for 50ms. >> I think we need some more information to nail this one. > > It certainly does not run for 50ms, but it might occasionally keep the > thread busy for some 10-50us (I doubt it is longer than that) > and possibly cause the reschedule request to > fall into the interval where it should actually run. > > So if your theory is correct, it may well be that the callout system > sees the request "in the past" (possibly as a result as some incorrect > wraparound, or undefined behaviour on integer wraps) and then the > event is only recovered when the callout wheel (or whatever is the > underlying implementation) happens to go again through the entry. > > What is so magic in the values we see (400 or 600 or 40ms) i have no idea. > Rasool: It might be worth trying to set: kern.eventtimer.periodic=1 In /boot/loader.conf . Can you test that too? You need to reboot before the setting takes into effect. Luigi: I'm wondering if there is a problem with: cpu_new_callout(a,b,c); --HPS