From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 17 00:53:54 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 66177880; Wed, 17 Sep 2014 00:53:54 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A690759; Wed, 17 Sep 2014 00:53:53 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id q108so926600qgd.7 for ; Tue, 16 Sep 2014 17:53:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=R5PkOEB2IlJh6ht60AQ3hxlghSSKe2jaO1QayBdCbW0=; b=n3nYL4kHyKsmd2qK86ri51Y/b3lQJPANp8qXYjoCqwxx9b9gy435su5h3sVxKm/jH7 hlD+2C4XDcFIlqWB28GqCP7JIt/+tdKtUR7TbnaHKoi5zA8bIQajGHmc8iOJyiX/i+sz 3BpTVMpDNvARbLUzv2HoNnWj3rJyy67e7FICE3yn/MM7smW5OwVUa/nvgAg8tNCrlmF9 CZT3KGFPbFPsjHPsIkIRQ4GjNmjLVWPcVe8R8EbI2j7ZclsMfOR+reQOzdp2Zm8VCM7/ HimFnUTurfPWKg20Ofm2JpSYaXTAhBYSuTuXjM6uekOiHoLy+MSdcVNUXlaXzWJB7BLn cSCA== MIME-Version: 1.0 X-Received: by 10.229.136.133 with SMTP id r5mr18611785qct.31.1410915233141; Tue, 16 Sep 2014 17:53:53 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.39.139 with HTTP; Tue, 16 Sep 2014 17:53:52 -0700 (PDT) In-Reply-To: References: <5295A261.2060403@FreeBSD.org> <529F4409.9080403@FreeBSD.org> <52A1B869.6080407@FreeBSD.org> <52A21AE9.5020803@FreeBSD.org> <52A731FD.8060307@FreeBSD.org> Date: Tue, 16 Sep 2014 17:53:52 -0700 X-Google-Sender-Auth: d3ljxE12s7WXI1MI0HQgJBYKTiU Message-ID: Subject: Re: 9.1 callout behavior From: Adrian Chadd To: Alexander Motin Content-Type: text/plain; charset=UTF-8 Cc: Bret Ketchum , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2014 00:53:54 -0000 On 16 September 2014 17:38, Adrian Chadd wrote: > So, it did show up even if I say to run the callout on cpu #0 - any > kind of load on that CPU and that callwheel still gets run on another > CPU. > > What I next did was compile up KTR: > > include GENERIC > ident TESTING > options KTR > options ALQ > options KTR_ALQ > options KTR_COMPILE=0xffffffff > > .. then enable KTR_CALLOUT, and I see: > > 4175 0 884780740167480 precision set for 0xffffffff818194f8: 0.01999997 > 4174 0 884780667893634 callout lock 0xffffffff818194f8 func > 0xffffffff818191d0 arg 0xffffffff818194f8 > 4149 0 884780455441940 callout 0xffffffff818194f8 finished > 4148 0 884780455441414 scheduled 0xffffffff818194f8 func > 0xffffffff818191d0 arg 0xffffffff818194f8 in 561.4a5c984a > 4147 0 884780455440600 precision set for 0xffffffff818194f8: 0.01999997 > 4132 0 884780373374060 callout lock 0xffffffff818194f8 func > 0xffffffff818191d0 arg 0xffffffff818194f8 > 4103 0 884780170731868 callout 0xffffffff818194f8 finished > 4102 0 884780170731468 scheduled 0xffffffff818194f8 func > 0xffffffff818191d0 arg 0xffffffff818194f8 in 561.25c0f291 > 4101 0 884780170730948 precision set for 0xffffffff818194f8: 0.01999997 > 4086 0 884780089391348 callout lock 0xffffffff818194f8 func > 0xffffffff818191d0 arg 0xffffffff818194f8 > 4058 0 884779886019688 callout 0xffffffff818194f8 finished > > .. and those 561.xxxx values are sbintime values. > > The delta looks like it's ~ 142mS, which is in line with what your > callout routine reports. So the math to calculate the "next" event is > bumping it along to that value instead of 100mS. > > I'm going to update one of these boxes to -HEAD and see if it's still > a problem there. So the test case uses callout_reset(), which is the "older" way of doing it. What callout_reset() does is: * convert to sbintime_t; * set the callout precision to 0; * set C_HARDCLOCK. Now, C_HARDCLOCK doesn't read the last clock - it reads the last per-cpu clock value. I have no idea if PCPU_GET(hardclocktime) is _actually_ going to be equal across all CPUs - and boy if it isn't I'm not going to fix it - but the point here is it's not going to be updated very often when the system is idle and not receiving many interrupts. When I change the code to use callout_reset_sbt(): callout_reset_sbt(&cp->co, (SBT_1S * 100) / 1000, 0, ticktock_callback, cp, 0); .. and callout_reset_sbt(&c.co, (SBT_1S * 100) / 1000, 0, ticktock_callback, &c, 0); To fire at the same interval that you did (hz / 10; so 100mS) then it worked out perfectly fine. -a