From owner-freebsd-current@freebsd.org Sun Nov 29 14:20:19 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CAB834A356E for ; Sun, 29 Nov 2020 14:20:19 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4CkVrM2Dmlz4gx2 for ; Sun, 29 Nov 2020 14:20:19 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 4AC7C4A356D; Sun, 29 Nov 2020 14:20:19 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4A8824A3B9C for ; Sun, 29 Nov 2020 14:20:19 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CkVrM1RvGz4gns for ; Sun, 29 Nov 2020 14:20:18 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-wr1-x432.google.com with SMTP id s8so11590055wrw.10 for ; Sun, 29 Nov 2020 06:20:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=m2sQmgwDPhfZLwc2xeoLtCBKIaZuvKNb1GNTIJorFb0=; b=WUjm19m/15eaLZDbNt31enSJ/0N9wtrnaQrAdd/sWDH0zMLIw7Ftpwaz6J/mNt3dy7 3I+xpSB2YsNvXjOSxkLRCkcohVtGxXvrD3kvnRzbj3zUVLdr6R00G4MjOF/h1Xo49OGe dVTaNvsBvP4TbHkSv3X73HlBD3SppCTO2UkU9720JSRQtIMHPknfu889Pi2WD/Guq9Bm JUDLh5vOTF3/D1SDJD9l349sTCANcv+VMsN0A9qFsga9PiKScJtjixVdDpfTRmzp2NAi EwYnkR5tp52ZP0sbMJJViIK2ZglMHeo5Lu5MD/XuJNQnC3MmyAhsEVxgYQQifEI1Ahq1 QgWQ== X-Gm-Message-State: AOAM5330s9hHJS73BZYFN/bajD+WSXB0pNtJPD1+6kF4qkNS2AF4bPYM 5JZtiGcyW9VWfqL0Iir9STyNIIwQyjN6ByhymEQ= X-Google-Smtp-Source: ABdhPJxwn71xuuQA9On4yyyQDFGQbqcP97UJpAtTvvRGh0MwfxaacwiT8FyZzomhrU0Ita49TzOVldX2fXSiRC3WAzY= X-Received: by 2002:a5d:5146:: with SMTP id u6mr23392408wrt.66.1606659616774; Sun, 29 Nov 2020 06:20:16 -0800 (PST) MIME-Version: 1.0 Received: by 2002:a5d:4d47:0:0:0:0:0 with HTTP; Sun, 29 Nov 2020 06:20:15 -0800 (PST) In-Reply-To: References: From: Mateusz Guzik Date: Sun, 29 Nov 2020 15:20:15 +0100 Message-ID: Subject: Re: Laptop exhibits erratic responsiveness To: David Wolfskill , Jonathan Looney , current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4CkVrM1RvGz4gns X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Nov 2020 14:20:19 -0000 On 11/29/20, David Wolfskill wrote: > On Sat, Nov 28, 2020 at 10:47:57AM -0500, Jonathan Looney wrote: >> FWIW, I would try running lockstat on the box. (My supposition is that >> the >> delay is due to a lock. That could be incorrect. Lockstat may provide >> some >> clue as to whether this is a line of inquiry worth pursuing.) >> .... > > Thanks (again), Jonathan. > > So... I did that (during this morning's daily upgrade cycle); the > results may be "of interest" to some. > > I have placed copies of the typescripts in: > > http://www.catwhisker.org/~david/FreeBSD/head/lockstat/ > > I also scribbled a "README" in that same directory (though it doesn't > seem to show up in the listing); it may be accessed via > > http://www.catwhisker.org/~david/FreeBSD/head/lockstat/README > > My prior message in this thread showed what I saw during a "ping albert" > from the laptop while it was running head -- most RTTs were around 0.600 > ms, but some were notably longer, with at least one that was over 68 > seconds. > > So I did a "lockstat ping -c 64 albert" while the laptop was running > stable/12@r368123 (as a reference point); it is probably boring. :-} > > Then (this morning), I tried a simple "lockstat sleep 600" on the laptop > while it was running head@r368119 (and building head@r368143); we see > the "lockstat" output in the "lockstat_head" file. > > It then occurred to me that trying a "lockstat ping albert" might be > useful, so I fired up "lockstat ping -c 600 albert" -- which started up > OK, and demonstrated some long RTTs about every 11 packets or so, but we > see thing come to a screeching halt with: > > ... > 64 bytes from 172.16.8.13: icmp_seq=534 ttl=63 time=0.664 ms > lockstat: dtrace_status(): Abort due to systemic unresponsiveness > 64 bytes from 172.16.8.13: icmp_seq=535 ttl=63 time=9404.383 ms > > and we get no lockstat output. :-/ > > > Finally, as another "control," I ran similar commands from freebeast, > while it was running head@r368119 (and building head@r368143). Those > results are in the "lockstat_freebeast" file. > According to the data you got the entire kernel "freezes" every 11-12 seconds. So something way off is going on there. Given that the bug seems to be reproducible I think it would be best if you just bisected to the offending commit. -- Mateusz Guzik