From owner-freebsd-net@freebsd.org Fri Nov 6 09:51:09 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF90EA26C9A for ; Fri, 6 Nov 2015 09:51:09 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 919391DB4 for ; Fri, 6 Nov 2015 09:51:09 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id A45E71FE023; Fri, 6 Nov 2015 10:51:00 +0100 (CET) Subject: Re: Timing issue with Dummynet on high kernel timer interrupt To: Luigi Rizzo References: <6545444AE21C2749939E637E56594CEA3C0DCCC4@gsp-ex02.ds.swin.edu.au> <5638B7B5.3030802@selasky.org> <6545444AE21C2749939E637E56594CEA3C0DE7FF@gsp-ex02.ds.swin.edu.au> <563B2703.5080402@selasky.org> <6545444AE21C2749939E637E56594CEA3C0E0BD9@gsp-ex02.ds.swin.edu.au> <563C6864.2090907@selasky.org> Cc: Rasool Al-Saadi , "freebsd-net@freebsd.org" From: Hans Petter Selasky Message-ID: <563C786C.1050305@selasky.org> Date: Fri, 6 Nov 2015 10:52:44 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2015 09:51:09 -0000 On 11/06/15 09:50, Luigi Rizzo wrote: > On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky wrote: >> On 11/06/15 01:08, Rasool Al-Saadi wrote: >>> >>> >>> On Thursday, 5 November 2015 8:53 PM, Hans Petter Selasky wrote: >>>> >>>> >>>> On 11/05/15 00:44, Rasool Al-Saadi wrote: > ... >>> Removing C_HARDCLOCK reduces the problem but doesn't solve it completely. >>> However, removing C_DIRECT_EXEC instead solves the problem (but >>> occasionally very small spike(s) appears in high hz values). >>> I mentioned in my first email that removing these flags makes the issue to >>> disappear. But what the effects of removing these flags? If it cause timing >>> issue to Dummynet, why we should use them? >>> >> >> Hi, >> >> The C_DIRECT_EXEC flag reduces task switching overhead, that you don't have >> to wakeup a thread to wakeup the dummynet worker thread. It affects timing. > > Hans, > thanks for the explanation. > > Can you clarify the behaviour of C_DIRECT_EXEC ? > Does this mean that the task is run within some common > thread instead of a dedicated one ? Hi Luigi, C_DIRECT_EXEC means that the timer callback is executed directly from the fast interrupt filter of the timer or IPI. > > If so, for this type of task (dummynet may run at high rate > and use a significant amount of cpu time) it may be a good > idea to remove C_DIRECT_EXEC altogether. The ipfw dummynet code is not run from the timer callback. It is run from a taskqueue. I suspect there is likely a bug somewhere. At the moment it is not clear to me if there is a bug in the callout subsystem, that the when the timer is started with 1 tick delay it doesn't kick in until after 50ms or so at HZ=4000. Or if the dummynet's task is doing a lot of work for 50ms. I think we need some more information to nail this one. --HPS > > cheers > luigi >