From owner-freebsd-stable@freebsd.org Sun Mar 14 00:33:17 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4E97956969E for ; Sun, 14 Mar 2021 00:33:17 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com [IPv6:2607:f8b0:4864:20::32a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DygWc1V8dz3thk; Sun, 14 Mar 2021 00:33:15 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ot1-x32a.google.com with SMTP id f73-20020a9d03cf0000b02901b4d889bce0so3381991otf.12; Sat, 13 Mar 2021 16:33:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mVx8qiEcHu9B3gCKzW9g+7WvZ7ujYB9P9F7Ni765YXo=; b=Wd1X7OTuRoZLmKh5seIdEBOa2a+hLoUd+cp/jCf565IBBkkCagVKtQD2TM3r0mdAGv S0oihO5NzP2PUPJEJyZSv/PLj5yZ7UJHiR32Xlx3dqSQY6GnPArw0/eDG1JPZL16EVnY md7ztNEVmKacU7iI60mQCUC0jX3K6pdWgt28TmNqe5dNw/VG925+AofQDbHQdVQ+GV7I lljdet5U3E9wlq0Gt5txG3VGkxTrpbGYUazjxaZLogRbwVY5s1nAf2p/Yt/r0IeoOmK0 Ys3nGMCnEHMhoBlXEaxcKW2neDji8rtA50r30zWHPcc3Dtp5+5ilxvWId3nm6KwAqCdE UeaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mVx8qiEcHu9B3gCKzW9g+7WvZ7ujYB9P9F7Ni765YXo=; b=q6cZW8vzZrm1PfGyI9mXf3OU4lzHfRQ+UcbxHC545i2E3a3WexAVYesJtbkmaRLMlu he5CXyVFSRgyi6CVstP1qO435xqIAYS4KD3IJkmicdWv72OFwgnZf1WOsZVA5GiKUux+ G0o4GNsG/zBcAEcSLweKsBS+55n/2ahmbf+EmFIWjjTz5p4kbrJGkw3c461tZ4MHYXnX DkVmZofw0yeTMhKwTjABlsQX3Tz/UdXMafxNw8SWVwsKzgwDfPBUE1IAz+dz4NkyIMTs vUmL2+EB9MT0rih4FFGC0Gi65/fokxTfXaewQB6RGPXDAYLPPQg8E+81PF3BNtHdcTqe 5Rgg== X-Gm-Message-State: AOAM5309sFxEyDYgODOMY97xmLqwcwm60KaUK6p3I2yFDTGJ/gR2Q7A1 VA5x1qqbWxku6NSBTaCtaMwIa+CauNPldqz/s6Q= X-Google-Smtp-Source: ABdhPJzOzuIHaOHopUICMxPuk6Dhuuuuv9BEqmi8F3+wtAApdSwDtllTC4ATCeFfKQ2XHBTZx7alJAKw4hWVlyttC+s= X-Received: by 2002:a05:6830:1e14:: with SMTP id s20mr9188455otr.199.1615681994589; Sat, 13 Mar 2021 16:33:14 -0800 (PST) MIME-Version: 1.0 References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> In-Reply-To: <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> From: Kevin Oberman Date: Sat, 13 Mar 2021 16:32:58 -0800 Message-ID: Subject: Re: Filesystem operations slower in 13.0 than 12.2 To: Mark Millard Cc: Konstantin Belousov , FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4DygWc1V8dz3thk X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Wd1X7OTu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of kob6558@gmail.com designates 2607:f8b0:4864:20::32a as permitted sender) smtp.mailfrom=kob6558@gmail.com X-Spamd-Result: default: False [-3.70 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[yahoo.com]; FORGED_SENDER(0.30)[rkoberman@gmail.com,kob6558@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::32a:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[rkoberman@gmail.com,kob6558@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::32a:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::32a:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 00:33:17 -0000 Just spent a little time looking at my issue and have a few more notes: Seems to only occur on large r/w operations from/to the same disk. "sp big-file /other/file/on/same/disk" or tar/untar operations on large files. Hit this today updating firefox. I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing other things while it was running slowly, the disk would appear to lock up. E.g. pwd(1) seemed to completely lock up the system, but I could still ping it and, after about 30 seconds, things came back to life. It was also not instantaneous. Disc activity dropped to <1MB/s for a few seconds before everything froze. During the untar of firefox, I saw; this several times. I also looked at my console where I found these errors during : swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: 8192 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: 4096 I should note that some operations continue just fine while this is going on until I do something that freezes the system. I assume that this eliminates the disk drive and low-level driver. Is vfs a possible issue. It had some serious work in the past few months by markj. That does not explain why more people are not seeing this. I have been seeing this since at least September 2020, so it goes back a way. As this CometLake system will not run graphics on 12, I can't confirm operation before 13. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 On Fri, Mar 5, 2021 at 10:47 PM Mark Millard via freebsd-stable < freebsd-stable@freebsd.org> wrote: > > Konstantin Belousov kostikbel at gmail.com wrote on > Fri Mar 5 23:12:13 UTC 2021 : > > > On Sat, Mar 06, 2021 at 12:27:55AM +0200, Christos Chatzaras wrote: > . . . > > > Command: /usr/bin/time -l portsnap extract (these tests done with 2 > different idle servers but with same 4TB HDDs models) > > > > > > FreeBSD 12.2p4 > > > > > > 99.45 real 34.90 user 59.63 sys > > > 100.00 real 34.91 user 59.97 sys > > > 82.95 real 35.98 user 60.68 sys > > > > > > FreeBSD 13.0-RC1 > > > > > > 217.43 real 75.67 user 110.97 sys > > > 125.50 real 63.00 user 96.47 sys > > > 118.93 real 62.91 user 96.28 sys > > . . . > > In the portsnap results for 13RC1, the variance is too high to conclude > > anything, I think. > > I'll note that there are other reports of wide variance > in transfer rates observed during an overall operation > such as "make extract". The one I'm thinking of is: > > https://lists.freebsd.org/pipermail/freebsd-stable/2021-March/093251.html > > which is an update to earlier reports, but based on more recent > stable/13. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253968 > comment 4 has some more notes about the context. The "make extract" > for firefox likely is not as complicated as the portsnap extract > example's execution structure. > > Might be something to keep an eye on if there are on-going > examples of over time. > > === > Mark Millard > marklmi at yahoo.com > ( dsl-only.net went > away in early 2018-Mar) > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Sun Mar 14 00:36:15 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2049F56A330 for ; Sun, 14 Mar 2021 00:36:15 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dygb30DrLz3vMH for ; Sun, 14 Mar 2021 00:36:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x829.google.com with SMTP id r14so6770897qtt.7 for ; Sat, 13 Mar 2021 16:36:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bLee2USem4DuvKSLz7weeEyPI/tpawgezk4BrJxDMfA=; b=Dai8LDnp4uVnxKnbALjhXW6Dkg05g1fwY8I5UgeEiOiXarmxfHMYHaBykNwAfUDl9r kijrg+Y0a8TOukHaIj8lOxG349/4qOSk9FvUty9JlK/edK0js8wmkyAlJYsQLQbqhAct 49xDl5HP7+MYRxrYWfbOM4lH+MeDSx3wv3KmbvEsBj1eKZFMnJ2xX8TuP+ZSHS8TSvYt z/kF627fsWO/qio+YBcjP7CTD9dn1rcHI7waHgIdRWnkZU3PMwzeYmuftndKMZJSUbTH 3xPu4kKmH75b1sjgqjBmL8/IkIwSKLl/r3NxgRt4WXpK3sLx3Z8DQdeUdeyQVBpbT/wb jbHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bLee2USem4DuvKSLz7weeEyPI/tpawgezk4BrJxDMfA=; b=i6NWciHo5np8OMczsPlTMZu5VmG0vjUNmhk9OKbdEy+SySb9bQGY+c6vCfAwdiYJK/ f8DhzxQOG4YKGj/ptD8FHmm1bDUWZWDY39k7KCaPRNA2mcGAUzPBdT9plm2WpraWm4a5 JfneoyeG7B395TPchjHb7XPXrBKJBQR96DgqV9YGK0Oh5Oykd+RkaIXmW2SzcNznYEzQ FgIqotsZ3cgbWR/29g3nKXNdd9PSq1x7duE5ZEaHdqL8BjsJ7DFslcDGJonTok7pPssT 862uyTXsyp4Znjyvbjcrsst1yIuUilDqEGbdZ0QG8n72sRSvfigRQMYLptmODydE3dHJ Sh7A== X-Gm-Message-State: AOAM533XjUb9ozFDxdcs8Lkr0q9kVpOs/bqOtkiKWk7PcmKrF4LE0mDX RlftIuCfVTBdyU6eJuDP+abKhcNJI9g3kdgewW1cWA== X-Google-Smtp-Source: ABdhPJywPR1sHlV6kaseD2vMxyMzkSsemaNOqkAb8AMflzEZuyQj4iLaIv9EsJU8mbSQ+tv3iVNLx9bGJolfcU9XaKU= X-Received: by 2002:ac8:6787:: with SMTP id b7mr741203qtp.244.1615682174120; Sat, 13 Mar 2021 16:36:14 -0800 (PST) MIME-Version: 1.0 References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> In-Reply-To: From: Warner Losh Date: Sat, 13 Mar 2021 17:36:03 -0700 Message-ID: Subject: Re: Filesystem operations slower in 13.0 than 12.2 To: Kevin Oberman Cc: Mark Millard , Konstantin Belousov , FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4Dygb30DrLz3vMH X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 00:36:15 -0000 On Sat, Mar 13, 2021 at 5:33 PM Kevin Oberman wrote: > Just spent a little time looking at my issue and have a few more notes: > What version did you evaluate? There's a number of changes lately that could have a big impact on this... Warner > Seems to only occur on large r/w operations from/to the same disk. "sp > big-file /other/file/on/same/disk" or tar/untar operations on large files. > Hit this today updating firefox. > > I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing other > things while it was running slowly, the disk would appear to lock up. E.g. > pwd(1) seemed to completely lock up the system, but I could still ping it > and, after about 30 seconds, things came back to life. It was also not > instantaneous. Disc activity dropped to <1MB/s for a few seconds before > everything froze. > > During the untar of firefox, I saw; this several times. I also looked at my > console where I found these errors during : > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: 8192 > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: 4096 > > I should note that some operations continue just fine while this is going > on until I do something that freezes the system. I assume that this > eliminates the disk drive and low-level driver. Is vfs a possible issue. It > had some serious work in the past few months by markj. That does not > explain why more people are not seeing this. > > I have been seeing this since at least September 2020, so it goes back a > way. As this CometLake system will not run graphics on 12, I can't confirm > operation before 13. > -- > Kevin Oberman, Part time kid herder and retired Network Engineer > E-mail: rkoberman@gmail.com > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > > > On Fri, Mar 5, 2021 at 10:47 PM Mark Millard via freebsd-stable < > freebsd-stable@freebsd.org> wrote: > > > > > Konstantin Belousov kostikbel at gmail.com wrote on > > Fri Mar 5 23:12:13 UTC 2021 : > > > > > On Sat, Mar 06, 2021 at 12:27:55AM +0200, Christos Chatzaras wrote: > > . . . > > > > Command: /usr/bin/time -l portsnap extract (these tests done with 2 > > different idle servers but with same 4TB HDDs models) > > > > > > > > FreeBSD 12.2p4 > > > > > > > > 99.45 real 34.90 user 59.63 sys > > > > 100.00 real 34.91 user 59.97 sys > > > > 82.95 real 35.98 user 60.68 sys > > > > > > > > FreeBSD 13.0-RC1 > > > > > > > > 217.43 real 75.67 user 110.97 sys > > > > 125.50 real 63.00 user 96.47 sys > > > > 118.93 real 62.91 user 96.28 sys > > > . . . > > > In the portsnap results for 13RC1, the variance is too high to conclude > > > anything, I think. > > > > I'll note that there are other reports of wide variance > > in transfer rates observed during an overall operation > > such as "make extract". The one I'm thinking of is: > > > > > https://lists.freebsd.org/pipermail/freebsd-stable/2021-March/093251.html > > > > which is an update to earlier reports, but based on more recent > > stable/13. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253968 > > comment 4 has some more notes about the context. The "make extract" > > for firefox likely is not as complicated as the portsnap extract > > example's execution structure. > > > > Might be something to keep an eye on if there are on-going > > examples of over time. > > > > === > > Mark Millard > > marklmi at yahoo.com > > ( dsl-only.net went > > away in early 2018-Mar) > > > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org > " > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Sun Mar 14 01:37:23 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AE6DD56D828 for ; Sun, 14 Mar 2021 01:37:23 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com [IPv6:2607:f8b0:4864:20::32d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dyhxb454Bz4TQk; Sun, 14 Mar 2021 01:37:23 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ot1-x32d.google.com with SMTP id 75so5449824otn.4; Sat, 13 Mar 2021 17:37:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=U20PeC57cLjQ4SqXGqXFHwMp63bEBBeljU3hFF4TYQ4=; b=p0CShxxxp6fpELpznGBQkwwgZU87yE3w4FzyVZ3OLMaKLJIVT/BOotr4m+V15UN3QM OW6r0IVQtTx3Nh4kjhIya58ZlopwQDeP8zKm0QxrTEU2jo8bVaMk/9sXhB63dzG+JQOc kIGAY03WGE7dbAK98Y6ty6MmZ0kV9J8OVLD+XRiEmKfiTsohnCF2/thqBqRqdoJOk1yV Av5wAb8Igy0ME5mdNTDKz3Cd4b145CdoUBLL4D1CrNV2Z/57Z+wNFycG7KTHW6HKOAad UBV1VKLSPhI/JNimfqk/bS79qGoqcVJGcU2QucPYNNJ3/pBFvmjQtf4Cf2AuK9mxROLU gX3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=U20PeC57cLjQ4SqXGqXFHwMp63bEBBeljU3hFF4TYQ4=; b=FGyGiny9aN0gbv/Lp56ZNeyLJfZx15imB1KwWHPG518y2Cn4L44v8/5pldgGctWMSX 89Jv+QVGQXe+rg4Y1Xmw1OGeeP8+8bC09NrPuVb8NyTznfDkxGbThi779Jmr7a3ly1CE r95WJaVt6+2QfzlFaxMxV6/660j8S6BpHqhHhGO+qNnE/Ejt9u+7sW9dYlY6cc6Vp2MH TCv5XRQAhrlUg54cR+JjT9rSojwysrxOCCifnz/iIXmVX4D5VqRNSpDq2zxC+bpNa+pm wXShPYFTUH/S6obTrbHTbS3MemYBpiNi0UfBd30z7Lhxdc15P4iNZi9Rz8fq40DbiMYP IxmA== X-Gm-Message-State: AOAM5318hts6M6Kra5TXoFWpv4+5M34xxpIQDmPUZXKKNiHr39ozSxqd Z5241pYSHEkRDekZQF/45SAwtcUMZanPjTy67cw= X-Google-Smtp-Source: ABdhPJy1y+R/hp6LZMcaXVORxKudab63rCNtdUCADfqswp2ztypNppKkJudrCsC/g83Zqd2CU5YonbeNcXBu5UDoS6Y= X-Received: by 2002:a05:6830:1e14:: with SMTP id s20mr9348816otr.199.1615685842568; Sat, 13 Mar 2021 17:37:22 -0800 (PST) MIME-Version: 1.0 References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> In-Reply-To: From: Kevin Oberman Date: Sat, 13 Mar 2021 17:37:06 -0800 Message-ID: Subject: Re: Filesystem operations slower in 13.0 than 12.2 To: Warner Losh Cc: Mark Millard , Konstantin Belousov , FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4Dyhxb454Bz4TQk X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 01:37:23 -0000 I have been dealing with this for a long time since head back in September through 13-stable of Mar-4. I have seen no improvement over this time. It seems (my perception without supporting data) that it got worse in the timeframe of BETA-3 tag. I was running stable, so not quite BETA-3. It also does not help that I have also been bitten by the P-State related freeze issue which has some similarities. disabling p-states has almost eliminated this issue, though, with only three occurrences since I disabled them in late January. As a result, I don't think it is a recent change, but a problem that has existed for at least 3 months. This was made worse by two hardware issues that kept the system unavailable for most of the time between buying it last spring and getting the keyboard replaced in January. (Both the mainboard and the disk drive had already been replaced.) There was another slow I/O issue that I had assumed was the same as mine, but was reportedly fixed with BETA-4. A few are still seeing slow I/O, so I assume that there were different issues with I/O. Since CometLake systems seem pretty uncommon, it might be related to that. Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 On Sat, Mar 13, 2021 at 4:36 PM Warner Losh wrote: > > > On Sat, Mar 13, 2021 at 5:33 PM Kevin Oberman wrote: > >> Just spent a little time looking at my issue and have a few more notes: >> > > What version did you evaluate? There's a number of changes lately that > could have a big impact on this... > > Warner > > >> Seems to only occur on large r/w operations from/to the same disk. "sp >> big-file /other/file/on/same/disk" or tar/untar operations on large files. >> Hit this today updating firefox. >> >> I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing other >> things while it was running slowly, the disk would appear to lock up. E.g. >> pwd(1) seemed to completely lock up the system, but I could still ping it >> and, after about 30 seconds, things came back to life. It was also not >> instantaneous. Disc activity dropped to <1MB/s for a few seconds before >> everything froze. >> >> During the untar of firefox, I saw; this several times. I also looked at >> my >> console where I found these errors during : >> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: 8192 >> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: 4096 >> >> I should note that some operations continue just fine while this is going >> on until I do something that freezes the system. I assume that this >> eliminates the disk drive and low-level driver. Is vfs a possible issue. >> It >> had some serious work in the past few months by markj. That does not >> explain why more people are not seeing this. >> >> I have been seeing this since at least September 2020, so it goes back a >> way. As this CometLake system will not run graphics on 12, I can't confirm >> operation before 13. >> -- >> Kevin Oberman, Part time kid herder and retired Network Engineer >> E-mail: rkoberman@gmail.com >> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 >> >> >> On Fri, Mar 5, 2021 at 10:47 PM Mark Millard via freebsd-stable < >> freebsd-stable@freebsd.org> wrote: >> >> > >> > Konstantin Belousov kostikbel at gmail.com wrote on >> > Fri Mar 5 23:12:13 UTC 2021 : >> > >> > > On Sat, Mar 06, 2021 at 12:27:55AM +0200, Christos Chatzaras wrote: >> > . . . >> > > > Command: /usr/bin/time -l portsnap extract (these tests done with 2 >> > different idle servers but with same 4TB HDDs models) >> > > > >> > > > FreeBSD 12.2p4 >> > > > >> > > > 99.45 real 34.90 user 59.63 sys >> > > > 100.00 real 34.91 user 59.97 sys >> > > > 82.95 real 35.98 user 60.68 sys >> > > > >> > > > FreeBSD 13.0-RC1 >> > > > >> > > > 217.43 real 75.67 user 110.97 sys >> > > > 125.50 real 63.00 user 96.47 sys >> > > > 118.93 real 62.91 user 96.28 sys >> > > . . . >> > > In the portsnap results for 13RC1, the variance is too high to >> conclude >> > > anything, I think. >> > >> > I'll note that there are other reports of wide variance >> > in transfer rates observed during an overall operation >> > such as "make extract". The one I'm thinking of is: >> > >> > >> https://lists.freebsd.org/pipermail/freebsd-stable/2021-March/093251.html >> > >> > which is an update to earlier reports, but based on more recent >> > stable/13. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253968 >> > comment 4 has some more notes about the context. The "make extract" >> > for firefox likely is not as complicated as the portsnap extract >> > example's execution structure. >> > >> > Might be something to keep an eye on if there are on-going >> > examples of over time. >> > >> > === >> > Mark Millard >> > marklmi at yahoo.com >> > ( dsl-only.net went >> > away in early 2018-Mar) >> > >> > _______________________________________________ >> > freebsd-stable@freebsd.org mailing list >> > https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> > To unsubscribe, send any mail to " >> freebsd-stable-unsubscribe@freebsd.org" >> > >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >> > From owner-freebsd-stable@freebsd.org Sun Mar 14 02:00:19 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2AB5D56DEE6 for ; Sun, 14 Mar 2021 02:00:19 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DyjS30W8Kz4Vws for ; Sun, 14 Mar 2021 02:00:18 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x72b.google.com with SMTP id f124so28448524qkj.5 for ; Sat, 13 Mar 2021 18:00:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Nw0jle3MHJ3qYTlCxSq2yuQ4ewD+TAR7eiLF8iLYLe4=; b=xe9PC3J9EDwFIeQNsXV/3HvFQ1hCCZnTklUA1uj/cMdlJPHxkl7QsKVWQJJP+qPw9e dc9RI8bvLMLhiE4DTYlbupESkHGSHzz4eFiI2x5rV9uZxcNI2H53wL3LCZ1BMAxO1vR5 XxCxxrwQiuTAQzcmN/lJEpcQdUdDpDoSb2St6mRsIzA0U7bZGKfrUMsAokY6AYPFxRdK AlyTUtfa0H5k482gP+IMNHnjxg5aMeuov8hl/Jqi1n9lho9JcV5sl3uSpZudXwd6DaEM vH4E17Hd+OWz6rmoDW2cRSDNzsPF2LIapF95LqIpJtb7vC2OL3mCkIFSfgaSiq4jqPDc VwyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Nw0jle3MHJ3qYTlCxSq2yuQ4ewD+TAR7eiLF8iLYLe4=; b=ugM8Th8DHJdGKH6IU1cAOH3QPJpFzbnRtFQ6f8Xx4ZTrrF6G5Fm2u940enkJkx4/9Y HYJnOM+szdTnjHz72+AcFiIwx6Wt0WqPMFgivwzHcvZ5SFQvU3QIN/U0Kn/pwho4NuX7 YP8evWLlV+3fjluZ2c31bHBgG6bV2UoRKMa7okRDqxouQ4fl+2F9YdzYG3XUWdgjC30N O1io6XbCY5IdKqmPXTXOzZulaNxAgd1Dvqz3+vCzZEucbxi3KErOppA6fui/joGTvBkJ 6im3rDWxqj3S5YyfJW3FGDpdQkWVke7LWiB2P74QtIDO8QJ5k8hlUYP+zgCSKkKhaD42 PB5g== X-Gm-Message-State: AOAM532PpkdXdQG3f9Y0l5zRIV5E6pUtmAp8b1AaN2Eiil6AgtkKE7u0 py9UlvhyirQZFXP7xm5xS1ESug+nngpntAmA1YO57w== X-Google-Smtp-Source: ABdhPJxlJ2oC33y9q9Z+/rOx+HjZzllJrT8rsACV4AgW9o1cb/1YKpi3CKsa4owqE6Qg/oEkBfBk3kP0FfEKT4G/inQ= X-Received: by 2002:ae9:e010:: with SMTP id m16mr2560289qkk.44.1615687217951; Sat, 13 Mar 2021 18:00:17 -0800 (PST) MIME-Version: 1.0 References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> In-Reply-To: From: Warner Losh Date: Sat, 13 Mar 2021 19:00:07 -0700 Message-ID: Subject: Re: Filesystem operations slower in 13.0 than 12.2 To: Kevin Oberman Cc: Mark Millard , Konstantin Belousov , FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4DyjS30W8Kz4Vws X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 02:00:19 -0000 On Sat, Mar 13, 2021 at 6:37 PM Kevin Oberman wrote: > I have been dealing with this for a long time since head back in September > through 13-stable of Mar-4. I have seen no improvement over this time. It > seems (my perception without supporting data) that it got worse in the > timeframe of BETA-3 tag. I was running stable, so not quite BETA-3. It also > does not help that I have also been bitten by the P-State related freeze > issue which has some similarities. disabling p-states has almost eliminated > this issue, though, with only three occurrences since I disabled them in > late January. > > As a result, I don't think it is a recent change, but a problem that has > existed for at least 3 months. This was made worse by two hardware issues > that kept the system unavailable for most of the time between buying it > last spring and getting the keyboard replaced in January. (Both the > mainboard and the disk drive had already been replaced.) There was another > slow I/O issue that I had assumed was the same as mine, but was reportedly > fixed with BETA-4. A few are still seeing slow I/O, so I assume that there > were different issues with I/O. Since CometLake systems seem pretty > uncommon, it might be related to that. > It was a change from last fall, or set of changes. RC1 or defintely RC2 has fixes to regain performance lost. If BETA4 was the last one you evaluated, perhaps you could do a couple tests with RC2 now that it's out to see if it is the same thing? Warner > Kevin Oberman, Part time kid herder and retired Network Engineer > E-mail: rkoberman@gmail.com > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > > > On Sat, Mar 13, 2021 at 4:36 PM Warner Losh wrote: > >> >> >> On Sat, Mar 13, 2021 at 5:33 PM Kevin Oberman >> wrote: >> >>> Just spent a little time looking at my issue and have a few more notes: >>> >> >> What version did you evaluate? There's a number of changes lately that >> could have a big impact on this... >> >> Warner >> >> >>> Seems to only occur on large r/w operations from/to the same disk. "sp >>> big-file /other/file/on/same/disk" or tar/untar operations on large >>> files. >>> Hit this today updating firefox. >>> >>> I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing other >>> things while it was running slowly, the disk would appear to lock up. >>> E.g. >>> pwd(1) seemed to completely lock up the system, but I could still ping it >>> and, after about 30 seconds, things came back to life. It was also not >>> instantaneous. Disc activity dropped to <1MB/s for a few seconds before >>> everything froze. >>> >>> During the untar of firefox, I saw; this several times. I also looked at >>> my >>> console where I found these errors during : >>> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: 8192 >>> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: 4096 >>> >>> I should note that some operations continue just fine while this is going >>> on until I do something that freezes the system. I assume that this >>> eliminates the disk drive and low-level driver. Is vfs a possible issue. >>> It >>> had some serious work in the past few months by markj. That does not >>> explain why more people are not seeing this. >>> >>> I have been seeing this since at least September 2020, so it goes back a >>> way. As this CometLake system will not run graphics on 12, I can't >>> confirm >>> operation before 13. >>> -- >>> Kevin Oberman, Part time kid herder and retired Network Engineer >>> E-mail: rkoberman@gmail.com >>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 >>> >>> >>> On Fri, Mar 5, 2021 at 10:47 PM Mark Millard via freebsd-stable < >>> freebsd-stable@freebsd.org> wrote: >>> >>> > >>> > Konstantin Belousov kostikbel at gmail.com wrote on >>> > Fri Mar 5 23:12:13 UTC 2021 : >>> > >>> > > On Sat, Mar 06, 2021 at 12:27:55AM +0200, Christos Chatzaras wrote: >>> > . . . >>> > > > Command: /usr/bin/time -l portsnap extract (these tests done with 2 >>> > different idle servers but with same 4TB HDDs models) >>> > > > >>> > > > FreeBSD 12.2p4 >>> > > > >>> > > > 99.45 real 34.90 user 59.63 sys >>> > > > 100.00 real 34.91 user 59.97 sys >>> > > > 82.95 real 35.98 user 60.68 sys >>> > > > >>> > > > FreeBSD 13.0-RC1 >>> > > > >>> > > > 217.43 real 75.67 user 110.97 sys >>> > > > 125.50 real 63.00 user 96.47 sys >>> > > > 118.93 real 62.91 user 96.28 sys >>> > > . . . >>> > > In the portsnap results for 13RC1, the variance is too high to >>> conclude >>> > > anything, I think. >>> > >>> > I'll note that there are other reports of wide variance >>> > in transfer rates observed during an overall operation >>> > such as "make extract". The one I'm thinking of is: >>> > >>> > >>> https://lists.freebsd.org/pipermail/freebsd-stable/2021-March/093251.html >>> > >>> > which is an update to earlier reports, but based on more recent >>> > stable/13. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253968 >>> > comment 4 has some more notes about the context. The "make extract" >>> > for firefox likely is not as complicated as the portsnap extract >>> > example's execution structure. >>> > >>> > Might be something to keep an eye on if there are on-going >>> > examples of over time. >>> > >>> > === >>> > Mark Millard >>> > marklmi at yahoo.com >>> > ( dsl-only.net went >>> > away in early 2018-Mar) >>> > >>> > _______________________________________________ >>> > freebsd-stable@freebsd.org mailing list >>> > https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> > To unsubscribe, send any mail to " >>> freebsd-stable-unsubscribe@freebsd.org" >>> > >>> _______________________________________________ >>> freebsd-stable@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org >>> " >>> >> From owner-freebsd-stable@freebsd.org Sun Mar 14 05:43:54 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 135B35737BF for ; Sun, 14 Mar 2021 05:43:54 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ot1-x329.google.com (mail-ot1-x329.google.com [IPv6:2607:f8b0:4864:20::329]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DypQ16qk4z4mhf; Sun, 14 Mar 2021 05:43:53 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ot1-x329.google.com with SMTP id r24so5540293otp.12; Sat, 13 Mar 2021 21:43:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=W9cKpU1yMVdPgu9q5YCoDd3FhxPSGONzJS/jr8eC29c=; b=t385N8wKHgkr1LMVEubDzp6+ujvZQOBTmRpsp5f9mCAkdYyTfOuMiviH4YJvE53hvJ YiEA4mjrIX2kuBOme0Wycz9EM1Orn/Pvk7hYfFDJb3HQqZU3XEj3ugj2fS8fRLjkcaLw qjeOZgouE25YRXpu1tGYZy8Y2QeTGFqU5DDmva/URAgPOms2mY082d0/TtG9KlVTvpgm uSaLbkz1bohSEIlHh6rHW9wsyLXxTQPcDZ9L0a72cicZxZseo7Oj6dyMEGZNcuoBZAZI czYg7bVnjlS0QX1AaEgB55iVAVUNyiav0qtDCg5ha2fcVUbtEI3bhc6E3ruqaEIm4nO9 jssw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=W9cKpU1yMVdPgu9q5YCoDd3FhxPSGONzJS/jr8eC29c=; b=jWMjoRLL4DGEKeM2byqTWxcCZW7HWoP98tVoSULkOuAVdZmTtt201rWh1BApglAkeg D6q9jSQMECb0COVUjKs6PtEplP71Myg/28ZoZ63HBIYgi9yG5sFeN8ObzqJo5kKKyHeJ VBZQsZJUFF6FpZjz80QXNmSe1uWiCfiK8vle3/Uwg3uBg3DzLNF+aUIxoK1c8GRbJLMP KgAXKZPMulFbGJsvF1LU7iNa5Is28Fufd1Fd6pR1pepUT37t00L0t+yESEQG72zK2fCF PI/9xlSUATH1kuLrNMfTxIAtZMKwl/bR7JblB0UMTwPLNooUTbdKc+mWEjd53CUVv2IZ qE0Q== X-Gm-Message-State: AOAM532s8YfnO0P85ezvYUs6gUc5t3fpij2JoRz1u4SymMQcnB3GtJXB AyhQaSB7HPjCqPdrb5wBu/fKW2oo+IW6T3xDvjQ= X-Google-Smtp-Source: ABdhPJzhP1fPwj1AZcDznw+J1uNQK0uGhMVaxwVvyRbgsMpOXScUGsRqFOD3UXFUq2U2epUnLGumk1LYaJXxoBcVZzQ= X-Received: by 2002:a05:6830:1e14:: with SMTP id s20mr9940298otr.199.1615700632700; Sat, 13 Mar 2021 21:43:52 -0800 (PST) MIME-Version: 1.0 References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> In-Reply-To: From: Kevin Oberman Date: Sat, 13 Mar 2021 21:43:36 -0800 Message-ID: Subject: Re: Filesystem operations slower in 13.0 than 12.2 To: Warner Losh Cc: Mark Millard , Konstantin Belousov , FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4DypQ16qk4z4mhf X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 05:43:54 -0000 No improvement with stable/13-n244880-cec3990d347. It may be worse. worse. An attempt to unpack firefox-86.0.1,2 saw disk rates in the range of 800K to 2 MB/s range and with repeated 30 second freezes. I have no idea what made it so much worse, but I'm forced to start wondering if it could be a hardware issue. The disk drive was already replaced once due to a bad bearing. Went from a WD Black to a Seagate. Since it just keeps getting worse, I must consider that possibility. It is odd, though, that it was suddenly worse with the updated system. I think I will try going back to n244765-a00bf7d9bba (March 4) and see if it improves. If it does, I can likely eliminate bad hardware. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 On Sat, Mar 13, 2021 at 6:00 PM Warner Losh wrote: > > > On Sat, Mar 13, 2021 at 6:37 PM Kevin Oberman wrote: > >> I have been dealing with this for a long time since head back in >> September through 13-stable of Mar-4. I have seen no improvement over this >> time. It seems (my perception without supporting data) that it got worse in >> the timeframe of BETA-3 tag. I was running stable, so not quite BETA-3. It >> also does not help that I have also been bitten by the P-State related >> freeze issue which has some similarities. disabling p-states has almost >> eliminated this issue, though, with only three occurrences since I disabled >> them in late January. >> >> As a result, I don't think it is a recent change, but a problem that has >> existed for at least 3 months. This was made worse by two hardware issues >> that kept the system unavailable for most of the time between buying it >> last spring and getting the keyboard replaced in January. (Both the >> mainboard and the disk drive had already been replaced.) There was another >> slow I/O issue that I had assumed was the same as mine, but was reportedly >> fixed with BETA-4. A few are still seeing slow I/O, so I assume that there >> were different issues with I/O. Since CometLake systems seem pretty >> uncommon, it might be related to that. >> > > It was a change from last fall, or set of changes. RC1 or defintely RC2 > has fixes to regain performance lost. If BETA4 was the last one you > evaluated, perhaps you could do a couple tests with RC2 now that it's out > to see if it is the same thing? > > Warner > > >> Kevin Oberman, Part time kid herder and retired Network Engineer >> E-mail: rkoberman@gmail.com >> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 >> >> >> On Sat, Mar 13, 2021 at 4:36 PM Warner Losh wrote: >> >>> >>> >>> On Sat, Mar 13, 2021 at 5:33 PM Kevin Oberman >>> wrote: >>> >>>> Just spent a little time looking at my issue and have a few more notes: >>>> >>> >>> What version did you evaluate? There's a number of changes lately that >>> could have a big impact on this... >>> >>> Warner >>> >>> >>>> Seems to only occur on large r/w operations from/to the same disk. "sp >>>> big-file /other/file/on/same/disk" or tar/untar operations on large >>>> files. >>>> Hit this today updating firefox. >>>> >>>> I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing other >>>> things while it was running slowly, the disk would appear to lock up. >>>> E.g. >>>> pwd(1) seemed to completely lock up the system, but I could still ping >>>> it >>>> and, after about 30 seconds, things came back to life. It was also not >>>> instantaneous. Disc activity dropped to <1MB/s for a few seconds before >>>> everything froze. >>>> >>>> During the untar of firefox, I saw; this several times. I also looked >>>> at my >>>> console where I found these errors during : >>>> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: 8192 >>>> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: 4096 >>>> >>>> I should note that some operations continue just fine while this is >>>> going >>>> on until I do something that freezes the system. I assume that this >>>> eliminates the disk drive and low-level driver. Is vfs a possible >>>> issue. It >>>> had some serious work in the past few months by markj. That does not >>>> explain why more people are not seeing this. >>>> >>>> I have been seeing this since at least September 2020, so it goes back a >>>> way. As this CometLake system will not run graphics on 12, I can't >>>> confirm >>>> operation before 13. >>>> -- >>>> Kevin Oberman, Part time kid herder and retired Network Engineer >>>> E-mail: rkoberman@gmail.com >>>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 >>>> >>>> >>>> On Fri, Mar 5, 2021 at 10:47 PM Mark Millard via freebsd-stable < >>>> freebsd-stable@freebsd.org> wrote: >>>> >>>> > >>>> > Konstantin Belousov kostikbel at gmail.com wrote on >>>> > Fri Mar 5 23:12:13 UTC 2021 : >>>> > >>>> > > On Sat, Mar 06, 2021 at 12:27:55AM +0200, Christos Chatzaras wrote: >>>> > . . . >>>> > > > Command: /usr/bin/time -l portsnap extract (these tests done with >>>> 2 >>>> > different idle servers but with same 4TB HDDs models) >>>> > > > >>>> > > > FreeBSD 12.2p4 >>>> > > > >>>> > > > 99.45 real 34.90 user 59.63 sys >>>> > > > 100.00 real 34.91 user 59.97 sys >>>> > > > 82.95 real 35.98 user 60.68 sys >>>> > > > >>>> > > > FreeBSD 13.0-RC1 >>>> > > > >>>> > > > 217.43 real 75.67 user 110.97 sys >>>> > > > 125.50 real 63.00 user 96.47 sys >>>> > > > 118.93 real 62.91 user 96.28 sys >>>> > > . . . >>>> > > In the portsnap results for 13RC1, the variance is too high to >>>> conclude >>>> > > anything, I think. >>>> > >>>> > I'll note that there are other reports of wide variance >>>> > in transfer rates observed during an overall operation >>>> > such as "make extract". The one I'm thinking of is: >>>> > >>>> > >>>> https://lists.freebsd.org/pipermail/freebsd-stable/2021-March/093251.html >>>> > >>>> > which is an update to earlier reports, but based on more recent >>>> > stable/13. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253968 >>>> > comment 4 has some more notes about the context. The "make extract" >>>> > for firefox likely is not as complicated as the portsnap extract >>>> > example's execution structure. >>>> > >>>> > Might be something to keep an eye on if there are on-going >>>> > examples of over time. >>>> > >>>> > === >>>> > Mark Millard >>>> > marklmi at yahoo.com >>>> > ( dsl-only.net went >>>> > away in early 2018-Mar) >>>> > >>>> > _______________________________________________ >>>> > freebsd-stable@freebsd.org mailing list >>>> > https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>> > To unsubscribe, send any mail to " >>>> freebsd-stable-unsubscribe@freebsd.org" >>>> > >>>> _______________________________________________ >>>> freebsd-stable@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>> To unsubscribe, send any mail to " >>>> freebsd-stable-unsubscribe@freebsd.org" >>>> >>> From owner-freebsd-stable@freebsd.org Sun Mar 14 08:59:33 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 17B7F57816B for ; Sun, 14 Mar 2021 08:59:33 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [212.12.50.234]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gilb.zs64.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dytlm1ZMNz3DvY for ; Sun, 14 Mar 2021 08:59:31 +0000 (UTC) (envelope-from stb@lassitu.de) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 99AE7192370 for ; Sun, 14 Mar 2021 08:59:24 +0000 (UTC) From: Stefan Bethke Content-Type: multipart/signed; boundary="Apple-Mail=_C2F1DDBA-5F3E-4E1D-84D8-7A21EA414A53"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Updating to 13-stable and existing ZFS pools: any gotchas? Message-Id: <9A253CE5-8144-427C-9192-AF3E9ADBA3B4@lassitu.de> Date: Sun, 14 Mar 2021 09:59:21 +0100 To: freebsd-stable X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4Dytlm1ZMNz3DvY X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of stb@lassitu.de designates 212.12.50.234 as permitted sender) smtp.mailfrom=stb@lassitu.de X-Spamd-Result: default: False [-3.90 / 15.00]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ATTACHMENT(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; R_DKIM_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[212.12.50.234:from]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:13135, ipnet:212.12.48.0/21, country:DE]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[stb]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[lassitu.de]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[212.12.50.234:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 08:59:33 -0000 --Apple-Mail=_C2F1DDBA-5F3E-4E1D-84D8-7A21EA414A53 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I'm planning to upgrade three production machines with existing ZFS = pools to 13-stable. Is there anything I need to pay attention to wrt = OpenZFS? Or should it be fully transparent, apart from updating loader? My (limited) testing with VMs went without a hitch, but I want to make = sure I don't paint myself into a corner. Stefan -- Stefan Bethke Fon +49 151 14070811 --Apple-Mail=_C2F1DDBA-5F3E-4E1D-84D8-7A21EA414A53 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEJ+hF98o4r3eU/HiPD885WK4W4sEFAmBN0GkACgkQD885WK4W 4sFzbwf+LhcDUCTV0JZTQvsuJAUDLTBm6A1X7abpL6+o29YLI2MITJHJkao5w9lU JR9HepsghnzrFQX6g85m2py85kSfomGtWHG+V4itT7lf31q84kE3gLir9I10CceG CjrYLPSgNAsHhAalCivDj4q3VUBUjZ9za6ghkB8Yda5Qa0BXWA8XOs2sc4wlThsy AeH31iENfIrHJci673OFW8JrWVQnPP/4IQmgrHEzb4cux07wuC1nNKTaitWgza4p Efba/ifRBvwr7z5sLsfVsMQ/QG9jiNZeMcolIIltSuZZgfeJoLcEdGF+0cNhyXjh sIm8iH22KWYIQxqX0bDAJoByNzeLNQ== =8jP8 -----END PGP SIGNATURE----- --Apple-Mail=_C2F1DDBA-5F3E-4E1D-84D8-7A21EA414A53-- From owner-freebsd-stable@freebsd.org Sun Mar 14 09:08:05 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2F199578883 for ; Sun, 14 Mar 2021 09:08:05 +0000 (UTC) (envelope-from Mathias.Picker@virtual-earth.de) Received: from www94.your-server.de (www94.your-server.de [213.133.104.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dytxc0w33z3Fc5 for ; Sun, 14 Mar 2021 09:08:03 +0000 (UTC) (envelope-from Mathias.Picker@virtual-earth.de) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtual-earth.de; s=default_1811; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Message-ID:Date:In-reply-to:Subject:To:From:References:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=EGJib2R2OO0s0rZq+AK37spZKVJtNG/IAcsE6e5T4qw=; b=i0Px1AMeGMSQwPUGoklarbYDT1 Vs1IgMnouX0Xxk73bEIRI8irRjhO0nM6jAJYiVM1The5U9M/u9gFNY7XmQQ9op8qO45NclJVJ/OeX iAlKuI3P7RH/qQfJehcTdKWvh0/wfRtM3ciQ1FdHZHeXAv3d22ot+tBpxWg3Qg75PEGoLkhJ5ec3e rImrjkBfzSqmHjFRBDMWJK9F9nJF2l3A4zmfbAier5LZ7BwCui65hVTS5XblP2VKg92SUNPx/4h2A 36AvKDejs6F5yCbVFGBaYn7RhdwLDsh83rhd5Ekqw2BGhHV/+uzKF6MjJEzrDAy7IQZH+3hQLb2EP +p+YQ+qQ==; Received: from sslproxy05.your-server.de ([78.46.172.2]) by www94.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92.3) (envelope-from ) id 1lLMjB-0008uN-Bp for freebsd-stable@freebsd.org; Sun, 14 Mar 2021 10:08:01 +0100 Received: from [2a01:c22:7627:cb00:1a1d:eaff:fe16:cc77] (helo=Danton.virtual-earth.de) by sslproxy05.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lLMjB-000E6N-8G for freebsd-stable@freebsd.org; Sun, 14 Mar 2021 10:08:01 +0100 References: <9A253CE5-8144-427C-9192-AF3E9ADBA3B4@lassitu.de> User-agent: mu4e 1.2.0; emacs 27.1 From: Mathias Picker To: freebsd-stable@freebsd.org Subject: Re: Updating to 13-stable and existing ZFS pools: any gotchas? In-reply-to: <9A253CE5-8144-427C-9192-AF3E9ADBA3B4@lassitu.de> Date: Sun, 14 Mar 2021 10:08:00 +0100 Message-ID: <86zgz6nlvz.fsf@virtual-earth.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Authenticated-Sender: Mathias.Picker@virtual-earth.de X-Virus-Scanned: Clear (ClamAV 0.102.4/26107/Sat Mar 13 13:02:04 2021) X-Rspamd-Queue-Id: 4Dytxc0w33z3Fc5 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=virtual-earth.de header.s=default_1811 header.b=i0Px1AMe; dmarc=pass (policy=none) header.from=virtual-earth.de; spf=pass (mx1.freebsd.org: domain of Mathias.Picker@virtual-earth.de designates 213.133.104.94 as permitted sender) smtp.mailfrom=Mathias.Picker@virtual-earth.de X-Spamd-Result: default: False [-3.10 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[virtual-earth.de:+]; DMARC_POLICY_ALLOW(-0.50)[virtual-earth.de,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_LOW(-0.10)[213.133.104.94:from]; SUBJECT_ENDS_QUESTION(1.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[213.133.104.94:from]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; HAS_X_AS(0.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[virtual-earth.de:s=default_1811]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[213.133.104.94:from:127.0.2.255]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 09:08:05 -0000 I=E2=80=99ve updated two machines and had no problems, read the mailing=20 lists and remember no problem apart from the boot loader. Updated the boot loader before upgrading the pools, found the way=20 for EFI is different then I remembered (mount EFI partition, copy=20 /boot/loader.efi to it) but it worked just fine.. So, while I cannot 100% guarantee it=E2=80=99s without problems, I would=20 expect no hiccups. And the new bootloader is only needed if/once=20 you upgrade your pools, I actually run my laptop for a week with=20 quite an old loader ;) Cheers, Mathias Stefan Bethke writes: > I'm planning to upgrade three production machines with existing=20 > ZFS pools to 13-stable. Is there anything I need to pay=20 > attention to wrt OpenZFS? Or should it be fully transparent,=20 > apart from updating loader? > > My (limited) testing with VMs went without a hitch, but I want=20 > to make sure I don't paint myself into a corner. > > > Stefan --=20 Mathias Picker=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 Gesch=C3=A4ftsf=C3=BChrer Mathias.Picker@virtual-earth.de virtual earth Gesellschaft f=C3=BCr Wissens re/pr=C3=A4 sentation mbH http://www.virtual-earth.de/ HRB126870 support@virtual-earth.de Westendstr. 142 089 / 1250 3943=20=20=20=20=20=20=20=20=20=20=20=20 From owner-freebsd-stable@freebsd.org Sat Mar 13 21:33:07 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C853B5AC6E6 for ; Sat, 13 Mar 2021 21:33:07 +0000 (UTC) (envelope-from matheus@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4DybWk63wsz3hh5 for ; Sat, 13 Mar 2021 21:33:06 +0000 (UTC) (envelope-from matheus@arroway.org) Received: from [10.1.7.11] (unknown [187.61.214.250]) by hobbes.arroway.org (Postfix) with ESMTPA id 1D8DE14D1E5 for ; Sat, 13 Mar 2021 18:33:06 -0300 (-03) Received: from 10.1.4.100 (SquirrelMail authenticated user matheus) by 10.1.7.11 with HTTP; Sat, 13 Mar 2021 18:33:06 -0300 Message-ID: <3b55484103f1833743a1fbbd504b0c38.squirrel@10.1.7.11> Date: Sat, 13 Mar 2021 18:33:06 -0300 Subject: Re: FreeBSD 13.0-RC2 Now Available From: "Nenhum_de_Nos" To: freebsd-stable@freebsd.org User-Agent: SquirrelMail/1.4.23 [SVN] MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Rspamd-Queue-Id: 4DybWk63wsz3hh5 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of matheus@arroway.org designates 173.199.118.77 as permitted sender) smtp.mailfrom=matheus@arroway.org X-Spamd-Result: default: False [-0.17 / 15.00]; FAKE_REPLY(1.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[173.199.118.77:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[173.199.118.77:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[arroway.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.966]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; MID_BARE_IP(2.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-Mailman-Approved-At: Sun, 14 Mar 2021 09:25:14 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2021 21:33:07 -0000 On Fri, March 12, 2021 21:11, Glen Barber wrote: > === Upgrading === > > The freebsd-update(8) utility supports binary upgrades of amd64 and i386 systems running earlier FreeBSD releases. Systems running earlier FreeBSD releases can upgrade as follows: > > # freebsd-update upgrade -r 13.0-RC2 Hi Glen, this announce mentions just amd64 and i386 as able to use freebsd-update. On the BETA2 announce there was a mention of aarch64 as well (https://lists.freebsd.org/pipermail/freebsd-stable/2021-February/093081.html). I then tried to use it now on a 13.0-BETA4 system: root@rpi4_bsd:~ # freebsd-update -r 13.0-RC2 upgrade src component not installed, skipped Looking up update.FreeBSD.org mirrors... 3 mirrors found. Fetching metadata signature for 13.0-BETA4 from update4.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic kernel/generic-dbg world/base world/base-dbg The following components of FreeBSD do not seem to be installed: Does this look reasonable (y/n)? The system is: FreeBSD rpi4_bsd 13.0-BETA4 FreeBSD 13.0-BETA4 #0 releng/13.0-n244592-e32bc253629: Fri Feb 26 05:53:09 UTC 2021 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64 I did not typed "y" yet. Is it safe to go through with it? Thanks. matheus -- "We will call you Cygnus, the God of balance you shall be." From owner-freebsd-stable@freebsd.org Sun Mar 14 14:13:42 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BFD0B5A8446 for ; Sun, 14 Mar 2021 14:13:42 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dz1kF73G3z3q3S for ; Sun, 14 Mar 2021 14:13:41 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: by mail-ej1-x62c.google.com with SMTP id p7so50613503eju.6 for ; Sun, 14 Mar 2021 07:13:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=Rbzx/PJv4DdJA3Z/i6Mauow7UqTRlylkAo6GB1birVg=; b=HdafNZQYuAsBHWQNnvDrm8dpzk2Fwr0AYBKNGxbYnIqEAbX75Yj+LD68wK5c0pIhcS 0LShmuvIRWx5dB8ovVH/3cluiGSWXtOuEqKovGFfPTO+fatVWWhF1dNXRmlnzXCnac5F Gh/R4yr+IiMLY/pnb5feHJbNt9gRipYmUdC64pRNsbq3oPhozVF7voTqnnA+nmDnJma8 ikdsw7u/DyEsze8N0cBmf+dz1ZKkvD6LOd+ftE+jr0FWdCEM4i2muAwOEMwpf+JJGNKu r4wWPRkNGIQ0R0jFapudVZqkb6I8142OtS2HBSHlmW0A9+B8SNg6KSnk790r63dpjZOT Fysw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=Rbzx/PJv4DdJA3Z/i6Mauow7UqTRlylkAo6GB1birVg=; b=chQknVXnj16QgL4XmWgGd/p/jy+eUJBeIKXMxdppecj1foqfKKKEFsep3vtxsvoJJ2 pCUYBEB2nw/O3PbnOF1XaSI7zef5BVSB5/Uufmw+IqXCkt1kWEZxb+e876LINGwFEBs6 dK/isu8y1CTrLCKU/6gZYT2T8uuCxwZSoOi8WKH60frXorO8e7uzqQ3tUFZGLDwwGkPB 6+v3b+UhCRiuMU79FVA5B2IziRVq8ARQFFcIjCtLrMYOr0zraaW+9C6ycw9x2hgaBEew tDzV5+HQ2ATJ0Ng3uAoiT/kf7Ghw5fGVDJgomnmW/W9OsfD4GJy1O4vd5Auu2QKKSqtV XjAw== X-Gm-Message-State: AOAM530a8mjSnqJLVbRgNaKXFBVVmtPtTpKAPyTAN1DIr3rWxzFVTnMZ 7C8F6ROFh1Rk5+mZ4PNpjWYUsb4G/hU= X-Google-Smtp-Source: ABdhPJz/wPUXts2EsCCcREtUASg83LuGfU/c+eqIJ+DbBmwyplIybKxfSE/w+5EowAyp95Vm8EQhSw== X-Received: by 2002:a17:906:f1c8:: with SMTP id gx8mr19179746ejb.385.1615731219627; Sun, 14 Mar 2021 07:13:39 -0700 (PDT) Received: from MBP-van-Johan.lan (85-147-130-226.cable.dynamic.v4.ziggo.nl. [85.147.130.226]) by smtp.gmail.com with ESMTPSA id k12sm330197edr.60.2021.03.14.07.13.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 14 Mar 2021 07:13:39 -0700 (PDT) Subject: Re: Second pool not mounted automaticly on 13.0-RC2 To: Warner Losh Cc: FreeBSD-STABLE Mailing List References: <43641178-f07c-9f73-4d7a-e3cb6c28d9ee@gmail.com> From: Johan Hendriks Message-ID: <4fb66fc5-b566-a2ee-2bd0-d2da4697ac9f@gmail.com> Date: Sun, 14 Mar 2021 15:13:37 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Rspamd-Queue-Id: 4Dz1kF73G3z3q3S X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=HdafNZQY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of johhendriks@gmail.com designates 2a00:1450:4864:20::62c as permitted sender) smtp.mailfrom=johhendriks@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::62c:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::62c:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62c:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 14:13:42 -0000 On 13/03/2021 19:43, Warner Losh wrote: > > > On Sat, Mar 13, 2021 at 9:44 AM Johan Hendriks > wrote: > > On 13/03/2021 17:09, Johan Hendriks wrote: > > Hello all, i just upgrade my test server from 12.1 to 13.0-RC2. > This > > is a baremetal server. > > It has two ssd's on ada0 and ada1 using zfs. Also there is a 6 disk > > pool named storage. > > > > After the update the storage pool is not imported any more. I have > > updated the pool, thinking it was necessary, but the result is the > > same, after a reboot the storage pool is not imported. > > > > root@test-server:~ # zpool list > > NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG CAP DEDUP > > HEALTH  ALTROOT > > zroot   107G  1.60G   105G        -         - 0%     1% 1.00x > > ONLINE  - > > root@test-server:~ # zpool import > >    pool: storage > >      id: 17039452293665227158 > >   state: ONLINE > >  action: The pool can be imported using its name or numeric > identifier. > >  config: > > > >     storage         ONLINE > >       mirror-0      ONLINE > >         gpt/disk00  ONLINE > >         gpt/disk01  ONLINE > >       mirror-1      ONLINE > >         gpt/disk02  ONLINE > >         gpt/disk03  ONLINE > >       mirror-2      ONLINE > >         gpt/disk04  ONLINE > >         gpt/disk05  ONLINE > > root@test-server:~ # > > > > After importing the pool all is fine but after a reboot it needs > to be > > imported again. > > > > Did something change? > > I observed this earlier on a HEAD which was 13 at that time (around > > januari) but did not have the time to look at it. That was a > baremetal > > server also. > > > For the completeness, i have the following related settings in > /boot/loader.conf. > > # ZFS > zfs_load="YES" > > And in my /etc/rc.conf the following. > # ZFS > zfs_enable="YES" > > > Have you updated your rc scripts? I had the same problem a while ago > until I did that. > > Warner > I did do a mergemaster, but it seems mergemaster is not updating the files in /etc/rc.d anymore. Copied the files from /usr/src/libexec/rc/rc.d to /etc/rc.d make it work again. Thank you for the pointer, time to take a look at etcupdate. From owner-freebsd-stable@freebsd.org Sun Mar 14 14:42:51 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 69BC75A8FB8 for ; Sun, 14 Mar 2021 14:42:51 +0000 (UTC) (envelope-from ruben@verweg.com) Received: from erg.verweg.com (erg.verweg.com [IPv6:2a02:898:96::5e8e:f508]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dz2Mt1VnLz3r43 for ; Sun, 14 Mar 2021 14:42:49 +0000 (UTC) (envelope-from ruben@verweg.com) DKIM-Filter: OpenDKIM Filter v2.10.3 erg.verweg.com (unknown-jobid) Received: from [IPv6:2a10:3781:3e9:1:e838:c727:f218:a4de] ([IPv6:2a10:3781:3e9:1:e838:c727:f218:a4de]) (authenticated bits=0) by erg.verweg.com (8.16.1/8.15.2) with ESMTPSA id 12EEgfqJ020114 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 14 Mar 2021 14:42:41 GMT (envelope-from ruben@verweg.com) From: Ruben van Staveren Content-Type: multipart/signed; boundary="Apple-Mail=_B5046A8B-C7BB-4009-BF52-3CF38DE8A16A"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: FreeBSD 13.0 RC1 UEFI RAID-10 boot problems under VMware Fusion Date: Sun, 14 Mar 2021 15:42:39 +0100 References: <58352200-C53A-4B8F-9498-316FC852BD95@verweg.com> To: FreeBSD-STABLE Mailing List In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4Dz2Mt1VnLz3r43 X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ATTACHMENT(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[verweg.com:+]; DMARC_POLICY_ALLOW(-0.50)[verweg.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a02:898:96::5e8e:f508:from]; ASN(0.00)[asn:8283, ipnet:2a02:898::/32, country:NL]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[verweg.com:s=verweg]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a02:898:96::5e8e:f508:from:127.0.2.255]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 14:42:51 -0000 --Apple-Mail=_B5046A8B-C7BB-4009-BF52-3CF38DE8A16A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 I=E2=80=99ve updated the installation to RC2, the behaviour has changed = in vmware fusion to when I installed one of the BETA releases. The initial boot fails, the system will reboot again and then it = succeeds, though more predictable than with the BETAs I=E2=80=99m trying to see the differences in screen recording I made, = and I can only see the =E2=80=9CImage base=E2=80=9D is changed. Load path, load device, bootcurrent, bootinfo path, currdev (disk0p1, = EFI)) has not changed. Either something in the loader(s) was changed, or the RC1 to RC2 update = placed contents on more =E2=80=9Cfavourable" places on disk that made it = possible to boot again. But I do not understand how the loader interacts with the (Vmware) EFI = firmware to make sense out of it. > On 10 Mar 2021, at 12:22, Ruben van Staveren wrote: >=20 > To continue on the subject of UEFI booting weirdness >=20 >> On 9 Mar 2021, at 16:57, Ruben van Staveren wrote: >>=20 >> If I press escape and end up in VMWare=E2=80=99s UEFI setup screen I = can boot from any ada*p1 drive and continue as normal. >> Is UEFI with OpenZFS too new, or is this an issue in VMWare? >=20 > I got an off list tip to see whether this was also the case in bhyve, = so I also created the setup in there, using UEFI boot, and no problems = even with the special/log/cache NVMe vdevs attached to the pool. >=20 > So I=E2=80=99m starting to wonder whether the loader / VMware UEFI = firmware (??) interaction is a bug in VMware or an edge case that needs = to be supported too. >=20 > Btw, bhyve is looking nice these days! >=20 > Cheers, > Ruben >=20 --Apple-Mail=_B5046A8B-C7BB-4009-BF52-3CF38DE8A16A Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBAgAdFiEEJ5YXTZtFY5bgSLwXiG9r7NR3qT8FAmBOIOAACgkQiG9r7NR3 qT9YzA/9Fm9eCYKxyvpBPHwlr2aW9r13YZGqx4IJYHB8LBh/5KyzLMaJOOMJqe5y /yCdjvj4+NjIW0gk5qAidcXsI8d6YuwH1185rErTgJ0evOMqnpyI7ZxEbpi1tV4/ XIZZo1asK+SSlwz/XXQEgj8MXdDOAnvSW4AtFoXeI7glXhfKC5x230Cd3fUTTFwF sTtQlRbTzo+v0wO67za3RZE3Lp3rBFbnsSbgLlp+eY9880ht3ScvGXYpGqK8naBr u8tWU9TLcpZGBum0MlmOsvnTl5S2VNvWRAAsEGkJHg11+MKgGPZFb8IydMf8uGQY PQR60t8mkSJTBOqYO8k1r5ciFNeNfNH+OgPtFy/yvToXOGHAcYAbeWbIc2LJraBa 1YPv08RYh9Ehgr7e+zjpms/DZLpg4LT6PGYvVN5qZEch5sqgXlLLtiRaHsgvWxld lREY0BxrGqGR55eqn4KkJKCwHUNW3ok9lQoPobpYTb9OpBkiDorT43JMVmdm7vf/ PHAYKyjGS3b2khFsmRdLS4LGmPwF1Oup/rt4sv+SC6VaBlRrYGfiQSVzYUUa6jsm hkgbIomM5v5mFTG5UdvSD8BN9Ed6iMhSY6352YVEgX+qYaClM0CLkEmz5QqUQ3CN l0NHr9yb7HJST35C7C5KrxjMRGBfuMtXkD3v+rz9goi6AULMm7s= =Nv/2 -----END PGP SIGNATURE----- --Apple-Mail=_B5046A8B-C7BB-4009-BF52-3CF38DE8A16A-- From owner-freebsd-stable@freebsd.org Sun Mar 14 18:10:02 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 04E795ADFFC for ; Sun, 14 Mar 2021 18:10:02 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com [IPv6:2607:f8b0:4864:20::235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dz6yx098Vz4YhJ; Sun, 14 Mar 2021 18:10:00 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-oi1-x235.google.com with SMTP id x135so27834409oia.9; Sun, 14 Mar 2021 11:10:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=v5i//1ex05dIve5lGMFOLX50py+eLaRkniWR+fprGrU=; b=cgTbAb3J9694e3mgvpcJBH22cHY8hM0Vm1NBGXIxXXX5YcTWUwQQ9GuNmykREcEo2A COvkBDw3J0byevhnWDOvvpqvE9q9MOWVneyTwna+uKgIz2FCO2I+Ynv7gVoNHvjGmHYL KannCmnxf5LFhYfq+zrGNkKBHNNmW75sEKSMMd07sOfmSh1srvun53SWghJMhyNGxgcP vfWxX01zA9loHzCcbGZIJP+Hnz3o8UHYB9BDCWezSqiPxVa/mqyfYkf6K2X8kGihxRw/ gwQ6ib0OTiaZ5zCdvmt3PMPSsRhqIODnN89u6ku/GQXMwQFEp0qdSCpWHsGQSZGvP71i MvcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=v5i//1ex05dIve5lGMFOLX50py+eLaRkniWR+fprGrU=; b=RqCR6K3DwlBIB0IvOOE03icu6XBeCXU7pdN6EsVkCIdklOgytlR3eTbNljsEcI5MPQ LrSXiIIB1OnukI7jcCQHBBmX/yilc8N5uVAYsgCpTF6s/Tknn9sfZa3tVRArVPpbmCvz Qr2zhEYjVcbltH0q8BX/zvtoPP5c5DkwsH8lXDV0/hhqSZyKe2mNlIrodF8fJ/+MiJAc +XqVvrVUZ3a5UdMaZ+ivW0m9sqe1KRvO8poehiltJ9DLygYh3+V2r4DtwG9mEqzlE2SF qfLasxTNUaA6Stq/MEfbq7lGXCP5tRyhuaPOSd5Kax0ODZU+ngHX57eGwfyDGFg8f+Zn gxMg== X-Gm-Message-State: AOAM531wtQ2w03k6frl/YX2ZhWvWXoDwnuUkEp46Sbm/TMgg6tKCEvd/ WfrumOjhe9HVMLXsLcgruYkUZUf5AGdLQnOM20I= X-Google-Smtp-Source: ABdhPJx/4F49AFPIDS+bbhz7/tkuWQ78VY9MXfbWylQZGezSIMIKv77hZzn0nincM3R9mTxjpN81BrrL/14X8vaKCno= X-Received: by 2002:aca:ac04:: with SMTP id v4mr16080835oie.57.1615745399741; Sun, 14 Mar 2021 11:09:59 -0700 (PDT) MIME-Version: 1.0 References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> In-Reply-To: From: Kevin Oberman Date: Sun, 14 Mar 2021 11:09:43 -0700 Message-ID: Subject: Re: Filesystem operations slower in 13.0 than 12.2 To: Warner Losh Cc: Mark Millard , Konstantin Belousov , FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4Dz6yx098Vz4YhJ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=cgTbAb3J; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of kob6558@gmail.com designates 2607:f8b0:4864:20::235 as permitted sender) smtp.mailfrom=kob6558@gmail.com X-Spamd-Result: default: False [-3.70 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[rkoberman@gmail.com,kob6558@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::235:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[rkoberman@gmail.com,kob6558@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::235:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::235:from]; FREEMAIL_CC(0.00)[yahoo.com,freebsd.org]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 18:10:02 -0000 On Sat, Mar 13, 2021 at 9:43 PM Kevin Oberman wrote: > No improvement with stable/13-n244880-cec3990d347. It may be worse. > worse. An attempt to unpack firefox-86.0.1,2 saw disk rates in the range > of 800K to 2 MB/s range and with repeated 30 second freezes. I have no idea > what made it so much worse, but I'm forced to start wondering if it could > be a hardware issue. The disk drive was already replaced once due to a bad > bearing. Went from a WD Black to a Seagate. Since it just keeps getting > worse, I must consider that possibility. It is odd, though, that it was > suddenly worse with the updated system. > > I think I will try going back to n244765-a00bf7d9bba (March 4) and see if > it improves. If it does, I can likely eliminate bad hardware. > -- > Kevin Oberman, Part time kid herder and retired Network Engineer > E-mail: rkoberman@gmail.com > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > > > On Sat, Mar 13, 2021 at 6:00 PM Warner Losh wrote: > >> >> >> On Sat, Mar 13, 2021 at 6:37 PM Kevin Oberman >> wrote: >> >>> I have been dealing with this for a long time since head back in >>> September through 13-stable of Mar-4. I have seen no improvement over this >>> time. It seems (my perception without supporting data) that it got worse in >>> the timeframe of BETA-3 tag. I was running stable, so not quite BETA-3. It >>> also does not help that I have also been bitten by the P-State related >>> freeze issue which has some similarities. disabling p-states has almost >>> eliminated this issue, though, with only three occurrences since I disabled >>> them in late January. >>> >>> As a result, I don't think it is a recent change, but a problem that >>> has existed for at least 3 months. This was made worse by two hardware >>> issues that kept the system unavailable for most of the time between buying >>> it last spring and getting the keyboard replaced in January. (Both the >>> mainboard and the disk drive had already been replaced.) There was another >>> slow I/O issue that I had assumed was the same as mine, but was reportedly >>> fixed with BETA-4. A few are still seeing slow I/O, so I assume that there >>> were different issues with I/O. Since CometLake systems seem pretty >>> uncommon, it might be related to that. >>> >> >> It was a change from last fall, or set of changes. RC1 or defintely RC2 >> has fixes to regain performance lost. If BETA4 was the last one you >> evaluated, perhaps you could do a couple tests with RC2 now that it's out >> to see if it is the same thing? >> >> Warner >> >> >>> Kevin Oberman, Part time kid herder and retired Network Engineer >>> E-mail: rkoberman@gmail.com >>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 >>> >>> >>> On Sat, Mar 13, 2021 at 4:36 PM Warner Losh wrote: >>> >>>> >>>> >>>> On Sat, Mar 13, 2021 at 5:33 PM Kevin Oberman >>>> wrote: >>>> >>>>> Just spent a little time looking at my issue and have a few more notes: >>>>> >>>> >>>> What version did you evaluate? There's a number of changes lately that >>>> could have a big impact on this... >>>> >>>> Warner >>>> >>>> >>>>> Seems to only occur on large r/w operations from/to the same disk. "sp >>>>> big-file /other/file/on/same/disk" or tar/untar operations on large >>>>> files. >>>>> Hit this today updating firefox. >>>>> >>>>> I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing other >>>>> things while it was running slowly, the disk would appear to lock up. >>>>> E.g. >>>>> pwd(1) seemed to completely lock up the system, but I could still ping >>>>> it >>>>> and, after about 30 seconds, things came back to life. It was also not >>>>> instantaneous. Disc activity dropped to <1MB/s for a few seconds before >>>>> everything froze. >>>>> >>>>> During the untar of firefox, I saw; this several times. I also looked >>>>> at my >>>>> console where I found these errors during : >>>>> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: 8192 >>>>> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: 4096 >>>>> >>>>> I should note that some operations continue just fine while this is >>>>> going >>>>> on until I do something that freezes the system. I assume that this >>>>> eliminates the disk drive and low-level driver. Is vfs a possible >>>>> issue. It >>>>> had some serious work in the past few months by markj. That does not >>>>> explain why more people are not seeing this. >>>>> >>>>> I have been seeing this since at least September 2020, so it goes back >>>>> a >>>>> way. As this CometLake system will not run graphics on 12, I can't >>>>> confirm >>>>> operation before 13. >>>>> -- >>>>> Kevin Oberman, Part time kid herder and retired Network Engineer >>>>> E-mail: rkoberman@gmail.com >>>>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 >>>>> >>>>> >>>>> On Fri, Mar 5, 2021 at 10:47 PM Mark Millard via freebsd-stable < >>>>> freebsd-stable@freebsd.org> wrote: >>>>> >>>>> > >>>>> > Konstantin Belousov kostikbel at gmail.com wrote on >>>>> > Fri Mar 5 23:12:13 UTC 2021 : >>>>> > >>>>> > > On Sat, Mar 06, 2021 at 12:27:55AM +0200, Christos Chatzaras wrote: >>>>> > . . . >>>>> > > > Command: /usr/bin/time -l portsnap extract (these tests done >>>>> with 2 >>>>> > different idle servers but with same 4TB HDDs models) >>>>> > > > >>>>> > > > FreeBSD 12.2p4 >>>>> > > > >>>>> > > > 99.45 real 34.90 user 59.63 sys >>>>> > > > 100.00 real 34.91 user 59.97 sys >>>>> > > > 82.95 real 35.98 user 60.68 sys >>>>> > > > >>>>> > > > FreeBSD 13.0-RC1 >>>>> > > > >>>>> > > > 217.43 real 75.67 user 110.97 sys >>>>> > > > 125.50 real 63.00 user 96.47 sys >>>>> > > > 118.93 real 62.91 user 96.28 sys >>>>> > > . . . >>>>> > > In the portsnap results for 13RC1, the variance is too high to >>>>> conclude >>>>> > > anything, I think. >>>>> > >>>>> > I'll note that there are other reports of wide variance >>>>> > in transfer rates observed during an overall operation >>>>> > such as "make extract". The one I'm thinking of is: >>>>> > >>>>> > >>>>> https://lists.freebsd.org/pipermail/freebsd-stable/2021-March/093251.html >>>>> > >>>>> > which is an update to earlier reports, but based on more recent >>>>> > stable/13. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253968 >>>>> > comment 4 has some more notes about the context. The "make extract" >>>>> > for firefox likely is not as complicated as the portsnap extract >>>>> > example's execution structure. >>>>> > >>>>> > Might be something to keep an eye on if there are on-going >>>>> > examples of over time. >>>>> > >>>>> > === >>>>> > Mark Millard >>>>> > marklmi at yahoo.com >>>>> > ( dsl-only.net went >>>>> > away in early 2018-Mar) >>>>> > >>>>> >>>> Backing off to Mar. 4 was not an improvement. My untar did seem better for a couple of minutes, but then the display froze again for 30 seconds and disk performance dropped to <1M. then things got really bad and behaved in a manner that was baffling to me. The screen froze again, but stayed frozen after half a minute. I clicked on a couple of buttons in Firefox to no effect and then hit ctrl-q to quit. After the long pause, I pressed the power button to try to force a shutdown. Suddenly, it started unwinding everything I had done during the freeze. My browser did the updates from my mouse clicks including quitting. It then switched to a different workspace from ctrl-alt-right and did a clean shutdown. ???? Do I also have a graphics issue? Examining log files show no indication that anything was happening. SMART shows no errors and reasonable values for everything. No indication of a HW problem. The system performs well unless I do something that tries a bulk disk data move. Building world takes about 75 minutes. I just have a very hard time building big ports. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Sun Mar 14 22:09:42 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E2BDE5B4994 for ; Sun, 14 Mar 2021 22:09:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-55.consmr.mail.gq1.yahoo.com (sonic308-55.consmr.mail.gq1.yahoo.com [98.137.68.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DzDHV31bGz4qBn for ; Sun, 14 Mar 2021 22:09:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1615759780; bh=uxHOJ6Xj9ilBg5OyCWOnuVBYhXCjHAejBito4UfMGyS=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ibQjLKEMYhfdN3xGk7zhrE5FdEdQpWPq+eeieB9oYeg1fqYJgzdHvby73ZJ8mnMClKQA4lZvz+oJ/ZnfrzTLT6mRA8O06UJb1Dmpc5VbNEW0Uw347GrezUbPw5kfgmBnpge93aYE8iz16woWLoQz84sxdegRv5Ams9Go/leS02NpVgOSV0OY68+NXzmV7Jo1967RyETA5HDW+UwwConklY0eIO8tIr6jgatnlQXBNMlPL4shzC2fJ4wjNutbvi1kePTzPlEtnqb2wNIhHgsm2lxtbdtEn6TTtR1QfXWsXb56QbKTWQsvWZn+3Dx4bRNksnsQhciJ2o71j7o/ztkkZQ== X-YMail-OSG: LRXkRH4VM1lkTY9ThI6ZgNGW8CTsVrulc.8V8nVqVP8bMHLpmpp1NNn10PQCyn0 60LnvRhk3vXFdaVkt9cP.hafsEYjXsDPogeiDGBrjTrBNlA5WWPj6drrYTvXqmAXSqx2qkVlmsov bL082ScFw.5FBoAUU1sQutz4wsSEMeEBb.8JSwYGqB5DZ3rpaPnyGlHUZ_q.Z.RDbrPA2_u3_ZiX oUzT4F67UOKdDQ6FWPGjGxyTcSFKJVBCF8WZLzWJRXJ4GDM1p0J4mCRaJRzCnItsRsHZkvHJ20Nh oUHffg1HPzNrS1WnHQwSBhiANCkzTvy8b4EvZD9.Q2tiK8dDgGHKuvQnMJjlspBo2Y8FKR2vmdbw _FGsVL4rpomGaoAyOvVCM6uH.Pyjl9BP1uyB0onXbSQJs8xMOX7STh8f6Qlczf9VcvQIFTu7wYvB j5hQSdp9Hg9AW0RnRJXx5AAmuvCeS9WyyFMLdZglPVNMgf_PTASXJ51stC7bN2PJBJ2bWMvpDjz_ 3j3opmTtt5HT_0QH9zjLxPFBf1q54WnwCHJ5.JqCSzmR3ov6C2wbe2UI5fKGbBaI.eYCBzYDuEPi OmzNpDQbFnxdlpqPjvZ_d9pXe6_aEA2Xhry2Z_gL2p6FcLc5IwHT86UbDBuLH3OTz68PcNwHvsRH ESoeBGb1vW76NIY1hxWj6J8NJnOoT1xbc41sRn1X8t7o23AZ5fR6i_LulkvCvrmu2E9htHn.gQM7 TibPqNj_Hj0zFbcI8idKeQU7S7_Azeu9uOrwFRPc3swWIzitdDiXybLHVDP2Pz8M51taMmXvX.Ey k0hDdRdB6NUQyXOLV7cm.Et3tF5DlU4OWglLh7viY5x2BWBT2g.xBPBpnRPpzf0dzitqKSN_0FyQ ljrZ.hAAlA5Em0MVKL86B2FZ9HPKAZGK2sWrSsu.TWOjRp_oAfmNKPh1LV32NQyVnBxEB260crzz yJqD8hzKfv35wVcBBaJn7JEHmxOe1lDxtntL9WANRjPIoUS4FDJOz9_gjHVxt1yqpoPCHBsr5IWw G9GAOWBAcLECZ8aCJyKz9DX_UrxfnkL4vwbeJ0G6t8ZAqj4o0VFhOxpdbArurj8a.2icwNe7Kqpo YEpzeo45lqdsFe95gVHliHnXpDgN_VQa32MAGa0rCugFVP1rjt16mmIPoSbipfJZeYq1viZEHHQf eKGN9iEU4Qklte2TwSB8XS4pp5SSC_b30He8hO0QShRljnKOr7aspsZi4IYPsMreb5UX9pC5MR6w tOqm0P0j._YozRr05F6IyJwNMFOZgn2t.83zLWbxzf8WdFuBQMfZUmUtLQxWozEo9PR_u4EtDZjo azKIRuOb0z5UPby8f1t0mOWsUqBfqCEn6mZNuBd3_LZdy9qWoiR6nXZ6FOcRh.aPhH1HEiTfP4OV 06mZpg6nSQxwG0oDyytHweMEpcIlOpEXhCh018G.15LmVPY9trYklMAoWCNJGP3HWM67WWOQEsAu DnCEwrF63apZzBdRbZB7E6UWSPDXQiRZtnEXPsRgroePUdrwlgwZ0hNMaExheBXU5xOAZwdl9OvB naNEj696u5fC5u9uRKU5iqJ_ELeKfvfklfB5TLoqEYYXekoqTWlbjM6wt.8IlSiVnUfJ49b7eXTT IUgTW6J3K8o_AvsyuLGxrBDLi86lpNv_ymfiCgRTh3im0BDiub0x3.1EH8HrccSiuXq9AKN7GMIb sQ_0..NwjZlZxyHn5JHmxkeyQuSb6jYHS8QvG4a_kM9MxUWodqqqHPSqBVs1ZUaRKBGEhcQMkuwX ZJoCB8.2VizhMxLtMcTMBcbK1ExBDcdsdHShOpIxR.jH0UBju2LvGuD4wCcGQivypSHlQ.6IknyE C.Edqxo73ZR8cla4mjw4HmA_wtPYYb7uLzXBfZcZyRVRs6H67NP2JapFOThV9UOuPbH3Mr_DO9c_ s6s51qz065YyRoKlEPxtNBLDjaE7g1tPhy1GBmL1eRBM2TBibrHMDCXYKpQpbOEoFgsU8McMNg0S JTZPz2zz45ZPvDAEN_1CRc7pePHsc4sTK_SaYO3arH4DlIz1ILrsR9UlSwPrg70CQNCJZ9y6zZpO wHHDaAvs8ijbqeyVMpTDwvuUVEFpwIFfI20FyFMWKZJkv36xC3q8bhw.BOPXrOi3KtV8KKGTGZIy PgR24f6ZMT8b_bH1ahmciXZc.sm7vtPaWT.mUkwcBQCJPho8Fk7ulAz76qdWe7MrF7WtGT36uTK4 1Ji.5JfPIatFgBQ781VY0gem4Q0NwyyzphEaPXUYF4ucv7ONiGjsOmJ0PBEMDRsyPNxiTrrU5_wW nkAdCwFVfjOzMr.abVS_Uhe3zEDMYrSiH2mWSi5bDXjde2k0y89Dof1oBa2f6X0yE_zpsrMWlHMl QLWhzf0ZwyrZsNEbhsk5eur8EQJbQ_2Ogq4Df7xIWTyFeHsWSeZRzzdwvROtHNw.E55BFCrmEnCU BMqGzfYUJmhHZ4Qq_GtEY59oyWYA9wN0gLQp_IHV7byr9SAwikTNXixw0OiN3ym7MJqHtHb5ajcq P5FOhAqOi6T1p0n0mQc2Uliw3C9mP4dAATentAGIGySj2RSEGH3jLhonA4vOoU2rPLk6a7yjlEmz ioVF0R_R9PlG7a2QTGcrk6nP4rl.a.HVmnLfT_6XVLsDIp28HWMUJ6_9AY.BS3RympJDUA9bhXk8 18fYaUwFp3FZVKvixy1XKMvCxGn_4a7wPQkE_UY1tZW6pgMbt9XutLcrmyn2reIBK9BwQ4f1w X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Sun, 14 Mar 2021 22:09:40 +0000 Received: by smtp425.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID b0486c35d9eb324d9c76bc9806a1a491; Sun, 14 Mar 2021 22:09:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: Filesystem operations slower in 13.0 than 12.2 From: Mark Millard In-Reply-To: Date: Sun, 14 Mar 2021 15:09:34 -0700 Cc: Warner Losh , Konstantin Belousov , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> To: Kevin Oberman X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4DzDHV31bGz4qBn X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Mar 2021 22:09:42 -0000 On 2021-Mar-14, at 11:09, Kevin Oberman wrote: > . . . > =20 > Seems to only occur on large r/w operations from/to the same disk. "sp > big-file /other/file/on/same/disk" or tar/untar operations on large = files. > Hit this today updating firefox. >=20 > I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing = other > things while it was running slowly, the disk would appear to lock up. = E.g. > pwd(1) seemed to completely lock up the system, but I could still ping = it > and, after about 30 seconds, things came back to life. It was also not > instantaneous. Disc activity dropped to <1MB/s for a few seconds = before > everything froze. >=20 > During the untar of firefox, I saw; this several times. I also looked = at my > console where I found these errors during : > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: = 8192 > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: = 4096 Does anyone know: Are those messages normal "reading is taking a rather long time" notices or is their presence more useful information in some way about the type of problem or context for the problem? As for the tests: Are these messages always present when near a time frame when the problem occurs? Never present in a near time frame to a period when the problem does not occur? It appears that the messages are associated with reading the disk(s), not directly with writing them, where the reads take more than "hz * 20" time units to complete. (I'm looking at main (14) code.) What might contribute to the time taken for the pending read(s)? /* * swap_pager_getpages() - bring pages in from swap * * Attempt to page in the pages in array "ma" of length "count". = The * caller may optionally specify that additional pages preceding = and * succeeding the specified range be paged in. The number of such = pages * is returned in the "rbehind" and "rahead" parameters, and they = will * be in the inactive queue upon return. * * The pages in "ma" must be busied and will remain busied upon = return. */ static int swap_pager_getpages_locked(vm_object_t object, vm_page_t *ma, int count, int *rbehind, int *rahead) { . . . /* * Wait for the pages we want to complete. VPO_SWAPINPROG is = always * cleared on completion. If an I/O error occurs, SWAPBLK_NONE * is set in the metadata for each page in the request. */ VM_OBJECT_WLOCK(object); /* This could be implemented more efficiently with aflags */ while ((ma[0]->oflags & VPO_SWAPINPROG) !=3D 0) { ma[0]->oflags |=3D VPO_SWAPSLEEP; VM_CNT_INC(v_intrans); if (VM_OBJECT_SLEEP(object, &object->handle, PSWP, "swread", hz * 20)) { printf( "swap_pager: indefinite wait buffer: bufobj: %p, blkno: %jd, size: = %ld\n", bp->b_bufobj, (intmax_t)bp->b_blkno, = bp->b_bcount); } } VM_OBJECT_WUNLOCK(object); . . . where: #define VM_OBJECT_SLEEP(object, wchan, pri, wmesg, timo) = \ rw_sleep((wchan), &(object)->lock, (pri), (wmesg), (timo)) and: #define rw_sleep(chan, rw, pri, wmesg, timo) = \ _sleep((chan), &(rw)->lock_object, (pri), (wmesg), = \ tick_sbt * (timo), 0, C_HARDCLOCK) (I do not claim to be able to interpret the implications of the code that leads to the messages. But seeing some of the code might prompt a thought by someone that knows the code's context and operation.) > . . . > Backing off to Mar. 4 was not an improvement. My untar did seem better = for a couple of minutes, but then the display froze again for 30 seconds = and disk performance dropped to <1M. You were able to see the disk performance drop while the display was frozen? It might not be the best for monitoring but I'll ask this in terms of top output: Does Inact, Laundry, Wired, Free, or other such show anything fairly unique for around the problematical time frame(s)? > then things got really bad and behaved in a manner that was baffling = to me. The screen froze again, but stayed frozen after half a minute. I = clicked on a couple of buttons in Firefox to no effect and then hit = ctrl-q to quit. After the long pause, I pressed the power button to try = to force a shutdown. Suddenly, it started unwinding everything I had = done during the freeze. My browser did the updates from my mouse clicks = including quitting. It then switched to a different workspace from = ctrl-alt-right and did a clean shutdown. ????=20 >=20 > Do I also have a graphics issue? Examining log files show no = indication that anything was happening. SMART shows no errors and = reasonable values for everything. No indication of a HW problem. The = system performs well unless I do something that tries a bulk disk data = move. Building world takes about 75 minutes. I just have a very hard = time building big ports. Almost like things were stuck-sleeping and then the sleep(s) finished? =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Mon Mar 15 09:06:40 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A1CA657586F for ; Mon, 15 Mar 2021 09:06:40 +0000 (UTC) (envelope-from mr.xanto@gmail.com) Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [IPv6:2a00:1450:4864:20::135]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DzVsX09Hpz4XxY for ; Mon, 15 Mar 2021 09:06:39 +0000 (UTC) (envelope-from mr.xanto@gmail.com) Received: by mail-lf1-x135.google.com with SMTP id p21so55550324lfu.11 for ; Mon, 15 Mar 2021 02:06:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:message-id:to:cc:subject:in-reply-to:references :mime-version:content-transfer-encoding; bh=XRzGcMbLnFsQaX+qxhoRdvU3UI0Ipw1jLOWR6daABPA=; b=orcTj9iA8fbVvHVWvqyOMvRn9pChxM/hw5tSVxniGR9fLx5lulwUuaG0DeT1ToJnM7 q+dejwQtZLU30tUDK3b/rz7jvQ2Z99vnHk9XFNWkNFifUyQof8giEljEGWBxBmAtjWqy VogSPHn9mqjiNwbJnJSGBYlxqTNjAdPVfbSf9HTjlJEgOtxOhDueg5dvY+fJdP+f2Pky kWbM3BqT7wv2SU1eDmizRvX6jKb39xQrwLA8jz+5rRGoP40YSLxK9tFJSAVoee66nmJ5 EgyQ4GdF9M4cDv8R5kUd8qdYOdUpY5Bx03TRV1wyrk/iiZKvac6zEYl8rX3ICcDCCt7k 1lWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:message-id:to:cc:subject:in-reply-to :references:mime-version:content-transfer-encoding; bh=XRzGcMbLnFsQaX+qxhoRdvU3UI0Ipw1jLOWR6daABPA=; b=UIUPQds95lnaK4hFB2O0kU3NHTW8irMfQ/Zc9I6LPQNWN5mog+BB3HaqePlKKCTFPp nSUdd7e/9J9OSz11joHR1uQzJJodHDB7TladLaiHiqXcTI/NZVSoVV+V8jOoZ7XkukLF HnogPQWKCiun40rN7y94hL1kh1oT4nsHA7H+FnP27rmH5REa/FTSTehUw6Ya2GchLwS9 9qNVV4GSKxy/0so+emmLevM7rsM8/jidU4EOnMIVRBye9MflPXgWhxLk9z9vQHeDHqbQ 5qXt7LyV+Cwmo4em57/WXx4VSYXYuvNbNlkshrtE/vw88G/boDhr0aSGykmmydAs7kyQ CcEA== X-Gm-Message-State: AOAM530j5BhvhztTOGHC8lzbD9XsFfreBlH45TRFze1gCPFtNyPLFy9u FEvejte139p+e99mph0XWUkJJn+K37o= X-Google-Smtp-Source: ABdhPJyDXoCu/zY/rKI8nKDVnXJFKW6R9YwYY9bDnY7e1I98pCXzoEIOvJUtBImWgM+wxWVid1uvTA== X-Received: by 2002:a19:4147:: with SMTP id o68mr7925190lfa.295.1615799198246; Mon, 15 Mar 2021 02:06:38 -0700 (PDT) Received: from [192.168.70.66] ([37.235.174.147]) by smtp.gmail.com with ESMTPSA id i185sm2580320lfd.279.2021.03.15.02.06.37 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 15 Mar 2021 02:06:37 -0700 (PDT) Date: Mon, 15 Mar 2021 12:06:37 +0300 From: Mamontov Roman Message-ID: <184858182.20210315120637@gmail.com> To: "Alexander V. Chernikov" CC: FreeBSD-STABLE Mailing List Subject: Re: Troubles with netdump(4) In-Reply-To: <44FCBDFB-8962-4F5D-BFCE-682662524245@ipfw.ru> References: <1903978782.20210310130715@gmail.com> <44FCBDFB-8962-4F5D-BFCE-682662524245@ipfw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1250 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4DzVsX09Hpz4XxY X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=orcTj9iA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mrxanto@gmail.com designates 2a00:1450:4864:20::135 as permitted sender) smtp.mailfrom=mrxanto@gmail.com X-Spamd-Result: default: False [-2.02 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RECEIVED_SPAMHAUS_PBL(0.00)[37.235.174.147:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::135:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; NEURAL_SPAM_SHORT(0.98)[0.982]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::135:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::135:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2021 09:06:40 -0000 =0D=0A>> Next =A0I =A0try =A0netdumping =A0to =A0another =A0FreeBSD-host = =A0(virtual machine on VMWare=20 >> hypervisor): >> root@host-3:~ # uname -mv >> FreeBSD 12.2-STABLE r369412 GENERIC =A0amd64 >> root@host-3:~ # ifconfig em0 >> em0: flags=3D8843 metric 0 mtu 1= 500 >> =A0 =A0 =A0 =A0options=3D81009b >> =A0 =A0 =A0 =A0inet 192.168.7.18 netmask 0xffffff00 broadcast 192.168.7.= 255 >> =A0 =A0 =A0 =A0media: Ethernet autoselect (1000baseT ) >> =A0 =A0 =A0 =A0status: active >> =A0 =A0 =A0 =A0nd6 options=3D29 >> root@host-3:~ # >>=20 >> And netdumping to this host are still slow (~the same 5KB/s).=20 >>=20 >> Next step another FreeBSD-host (virtual machine on VirtualBox hypevisor): >> root@host-4:~ # uname -mv >> FreeBSD 12.2-STABLE r369412 GENERIC =A0amd64 >> root@host-4:~ # ifconfig em0 >> em0: flags=3D8843 metric 0 mtu 1= 500 >> =A0 =A0 =A0 =A0options=3D81009b >> =A0 =A0 =A0 =A0inet 192.168.7.19 netmask 0xffffff00 broadcast 192.168.7.= 255 >> =A0 =A0 =A0 =A0media: Ethernet autoselect (1000baseT ) >> =A0 =A0 =A0 =A0status: active >> =A0 =A0 =A0 =A0nd6 options=3D29 >> root@host-4:~ # >>=20 >> Both (host-3 and host-4) are installed from FreeBSD-12.2-STABLE-amd64-20= 210304-r369412-bootonly.iso >> When =A0I =A0caused =A0a =A0kernel panic by sysctl debug.kdb.panic on ho= st-4, netdumping to=20 >> host-3 show the same 5 KB/s. > netdump requires explicit acks from the other side. > Could you try dumping the exchange between the dumping host and the serve= r to > verify that acks are sent immediately after receiving the next chunk? Sure. There is a dump: https://disk.yandex.ru/d/i27n1m-I80DaEg What should be tuned: netdumpd, netdumpd-host or something else? Now I can't completely done netdump: there frequenly error: ** DUMP FAILED (ERROR 60) ** Cannot dump: unknown error (error=3D60). and panic inside iflib: https://pastebin.com/1BTTjGLJ From owner-freebsd-stable@freebsd.org Mon Mar 15 21:58:00 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 638615698DC for ; Mon, 15 Mar 2021 21:58:00 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-oi1-x231.google.com (mail-oi1-x231.google.com [IPv6:2607:f8b0:4864:20::231]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DzqzX27P8z4WT8; Mon, 15 Mar 2021 21:57:59 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-oi1-x231.google.com with SMTP id d16so26751271oic.0; Mon, 15 Mar 2021 14:57:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=FxN5eXlmu0JdVUsn0x5OJ0pOkVxu7o9h28jAEPTMdsA=; b=R5VbbBvqebz0JaQqrxWLibMpVAnXC2Ngt8plwQLSqU/pfUQzcPJ5Eh+GcNHxrCgzLx 2FEZh1XnAyPwXluYp4whGG2fex4OzTfN3nywFqGtJZXXDyzGQRGiSazs3B5zTGyxmDOR XfvsdCbcMezgURmPNGlj1r5ZIip6/CdDxOPMaotHpbWJUj/LEt4WHT9DTlge67EqyvCS lrjqYtUelLgZjqe4+be8kgPhxNpKqkChBlAq67aV5j6j4d0RgBtgRMRoZ8VJsbEyHAnj kXEdqGa/u9H7psGw73jU4TLsS4dhhiKBTQDBb3aZdV7bJVXnX7OeoT/4pfMCoYcY5DiY 52Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FxN5eXlmu0JdVUsn0x5OJ0pOkVxu7o9h28jAEPTMdsA=; b=GjrIXzwpsGZozIiO6+5heqmTvPAw5/0UXmyHj1578J0Pw09Kckkxd/GSWEzJcta8vB zcD+vzcomlCeddaJw5IxIqfVuqVhghayEXFIuPqcH/RMzInedHUyIAQ/eFk1vXTds3ye w2UCurhJ4njmmTuw1qXgQ87QlpH+IsupsnpZmOYlOAQG4E/KL2nEDBdXduYjLnb4T+00 FmxwTEOiF6r83KsdlI2B8OZKWFMlXHFAWIi7Rz+mX4u1xQcj7aHDCd1Xn9/QjtQcyt18 IHfMO0Ghfb9myEKVe4AflnlULGqdY9axFvf/QgC7SEgTqFZOygNVnudFDwv6aa33JHtr xF3A== X-Gm-Message-State: AOAM531jH3ZqMgxLC5X0jBjZuuacNXoHHyvpkDUMQTxuTmqUEM7N3BVR ed4yy+iKRpe4tn2kdTlvOlH5kaFWB354jF7KvYY= X-Google-Smtp-Source: ABdhPJwfVFceZ7LLOwbeGm+KvFez4quCYbizBPZ8K8k4QYlWvGr49ytM/6IXJBSu6878clKA+t1RjkhOJTPvGXBQkgQ= X-Received: by 2002:aca:ac04:: with SMTP id v4mr921987oie.57.1615845479014; Mon, 15 Mar 2021 14:57:59 -0700 (PDT) MIME-Version: 1.0 References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> In-Reply-To: From: Kevin Oberman Date: Mon, 15 Mar 2021 14:57:42 -0700 Message-ID: Subject: Re: Filesystem operations slower in 13.0 than 12.2 To: Mark Millard Cc: Warner Losh , Konstantin Belousov , FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4DzqzX27P8z4WT8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2021 21:58:00 -0000 Responses in-line. On Sun, Mar 14, 2021 at 3:09 PM Mark Millard wrote: > > > On 2021-Mar-14, at 11:09, Kevin Oberman wrote: > > > . . . > > > > Seems to only occur on large r/w operations from/to the same disk. "sp > > big-file /other/file/on/same/disk" or tar/untar operations on large > files. > > Hit this today updating firefox. > > > > I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing other > > things while it was running slowly, the disk would appear to lock up. > E.g. > > pwd(1) seemed to completely lock up the system, but I could still ping it > > and, after about 30 seconds, things came back to life. It was also not > > instantaneous. Disc activity dropped to <1MB/s for a few seconds before > > everything froze. > > > > During the untar of firefox, I saw; this several times. I also looked at > my > > console where I found these errors during : > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: 8192 > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: 4096 > > Does anyone know: > Are those messages normal "reading is taking a rather long > time" notices or is their presence more useful information > in some way about the type of problem or context for the > problem? > > As for the tests: > Are these messages always present when near a time frame > when the problem occurs? Never present in a near time > frame to a period when the problem does not occur? > In a large number of test, these errors have not repeated. They baffle me for another reason. This system has 20G or RAM. Tyically, all swap space is unused. ATM I see 16384M free out of 16384. Not sure that I have ever seen it used, though it might have been while building rust. I have not built rust for a month. > > It appears that the messages are associated with reading > the disk(s), not directly with writing them, where the > reads take more than "hz * 20" time units to complete. > (I'm looking at main (14) code.) What might contribute > to the time taken for the pending read(s)? > The reference to hz * 20 woke up a few sleeping memory cells. I forgot that I cleaned up my loader.conf. It was largely a copy of the one on my decade-old T520. I commented out "kern.hz=100". I don't recall the details, but I think it was actually from an even older system, my T42 from before I retired. In any case, restoring this setting has greatly improved the situation. I now have really bad disk I/O performance on large disk to disk activity (untarring the firefox distro) instead of terrible performance and the system freezes have vanished, though I do see pauses in response to clicks or text entry, but the display remains active and the pauses are short... 1 to 15 seconds, I'd guess. No, I have no idea what this indicates. I'm still not seeing the performance I was seeing back in February when 40 MB/s for extended intervals was common and I once untarred firefox.tar.gz2 in under a minute and performance seldom dropped below 1.4 MB/s. > > /* > * swap_pager_getpages() - bring pages in from swap > * > * Attempt to page in the pages in array "ma" of length "count". The > * caller may optionally specify that additional pages preceding and > * succeeding the specified range be paged in. The number of such > pages > * is returned in the "rbehind" and "rahead" parameters, and they will > * be in the inactive queue upon return. > * > * The pages in "ma" must be busied and will remain busied upon > return. > */ > static int > swap_pager_getpages_locked(vm_object_t object, vm_page_t *ma, int count, > int *rbehind, int *rahead) > { > . . . > /* > * Wait for the pages we want to complete. VPO_SWAPINPROG is > always > * cleared on completion. If an I/O error occurs, SWAPBLK_NONE > * is set in the metadata for each page in the request. > */ > VM_OBJECT_WLOCK(object); > /* This could be implemented more efficiently with aflags */ > while ((ma[0]->oflags & VPO_SWAPINPROG) != 0) { > ma[0]->oflags |= VPO_SWAPSLEEP; > VM_CNT_INC(v_intrans); > if (VM_OBJECT_SLEEP(object, &object->handle, PSWP, > "swread", hz * 20)) { > printf( > "swap_pager: indefinite wait buffer: bufobj: %p, blkno: %jd, size: %ld\n", > bp->b_bufobj, (intmax_t)bp->b_blkno, > bp->b_bcount); > } > } > VM_OBJECT_WUNLOCK(object); > . . . > > where: > > #define VM_OBJECT_SLEEP(object, wchan, pri, wmesg, timo) \ > rw_sleep((wchan), &(object)->lock, (pri), (wmesg), (timo)) > > and: > > #define rw_sleep(chan, rw, pri, wmesg, timo) \ > _sleep((chan), &(rw)->lock_object, (pri), (wmesg), \ > tick_sbt * (timo), 0, C_HARDCLOCK) > > (I do not claim to be able to interpret the implications > of the code that leads to the messages. But seeing some > of the code might prompt a thought by someone that knows > the code's context and operation.) > > > . . . > > Backing off to Mar. 4 was not an improvement. My untar did seem better > for a couple of minutes, but then the display froze again for 30 seconds > and disk performance dropped to <1M. > > You were able to see the disk performance drop while > the display was frozen? > No, but it refreshed the display when it un-froze and the refreshed display showed a one-minute history that showed that data was still transferring during the pause. > > It might not be the best for monitoring but I'll ask > this in terms of top output: Does Inact, Laundry, > Wired, Free, or other such show anything fairly unique > for around the problematical time frame(s)? > These all look pretty normal. Free memory stays at 13G throughout hte operatioin. > > > then things got really bad and behaved in a manner that was baffling to > me. The screen froze again, but stayed frozen after half a minute. I > clicked on a couple of buttons in Firefox to no effect and then hit ctrl-q > to quit. After the long pause, I pressed the power button to try to force a > shutdown. Suddenly, it started unwinding everything I had done during the > freeze. My browser did the updates from my mouse clicks including quitting. > It then switched to a different workspace from ctrl-alt-right and did a > clean shutdown. ???? > > > > Do I also have a graphics issue? Examining log files show no indication > that anything was happening. SMART shows no errors and reasonable values > for everything. No indication of a HW problem. The system performs well > unless I do something that tries a bulk disk data move. Building world > takes about 75 minutes. I just have a very hard time building big ports. > > Almost like things were stuck-sleeping and then the > sleep(s) finished? > Exactly! > > === > Mark Millard > marklmi at yahoo.com > ( dsl-only.net went > away in early 2018-Mar) > -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Mon Mar 15 22:54:59 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D2AB356AD84 for ; Mon, 15 Mar 2021 22:54:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DzsFH3Qc1z4ZM5 for ; Mon, 15 Mar 2021 22:54:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1615848897; bh=WcJSlW5veV0YwIl2A5DuGZK0llrTP7oz9bEeWjM5WKA=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=jkih8rstrP+DPncCWkgCvz8jcb6k+BypcJb9mvZDVR4Mx+HS4f+Bo9+N69/llCWN15wUz/fLYf8vJ1AhHXNP8Y8J2nZfb+uuLVMf6lfUAV6GQfbGRuB+TKLPsmhCjoxFW9NwWALmkrhai/I1HyBl9JCMdCAKmXHlauKdjflJmycOpDTM3YJ7fznZ5DIpnP2Szt5v4K9DTqsoco5chymtfZnmpZjBO4zjMSUZw2gZf2YK1zyo9nxhSWT5yzKo2JQq3QQeq6HgDPZhxi4cQT/kllFWsOgqRikV9Uw6qoHkzHDr7Dh/xgQOHyK8p7HhTLcZsoD6zKrr3A9WSk0whIjk3w== X-YMail-OSG: fIP3zaIVM1ned7w5eLGCL6LY0Khar3bncROfRtMwANgtk2oCX8DLitEwSR7Sf06 izQgWRAv7p0IQJM5SRVTZmRsEzewuF8DLRXz_U7.9.Ao25FXMUZ1GHDBK0J1dQX_o5t6DAxA84yt N.VQKj_bK_WlVeNjKTJBUJ5Aug8KJQSsRx0G4UQh8wh.BXBIBSfwkh73FnabbQs7SKi9HkVWehOX LbN9bFMXBglzRwZ63E2tDN0yaEq9cBSOAp8sxAOgRd1yeJ9R6A1skIpg4v4DQReuwrtlxhBaqWlK nt6AuDRAHfcbVnX9t7IiZHd9AVdqXF2qlXTx7KNgcI2BjURbIElSn_m8FyiMWMdQBSaPoHSwIj1T c.d_GBCZIUBCnbQHhihILNUPXixl.PsTqIkUesVmhgCfYNl5CnKHUS497hSg43wdtXpUbgFe9cL4 okL59C0.tfLUdJ1zr1.RRgZAgk.4Wd5jPMpjrkjTXd5QrU5TXZapQ5LP7ESqmbO66oEF0L1f3qLe SM1MVK.Vdxzn.xalFSzTA3xG6CirK3nHw744oIytEnanEE3Rl16ye_._9O93uVGvceicdjZZm0sQ B7fApADqR7deisKsZKecZod4iSLW6dJiaAyQXDXhSRk7PMJveNaH8l6vADD8fwD4Zt1K383uu3ji f_tvwYcWD_xuCbq.WFgwCdHLIujrSLzw2yUp1Kod8MGjHXPd1bbyRh2sGqFM.2yANgjEKodWUoNY Ww8ohhS5VrL.WMQwJYOkE25ayOI7O1xuoSVCqEjX6T.pKt6WVhf2Gq6w3qSl2DL0rtLd_H9vJssg 8iQQmDllAJv4izW.BTpTJQY8pDyjPj6eM5JfLCXewxI9GzWgDXkaidazA91ichoQtxJPQ7zBPOJ8 kHdbaQODIrU.mfpFn6XH8CX5BMSG0y29d8zZ_vuHDsVkqcTl.1TN8LcTEtdvXq0yNlinCpdO1XZT 04SYNW8VismaTUXPS7s.6a0yYZ4n3FwV1E2vglqRaZZ41PuuTe7Un7FekhN8VHO9tt506W29gcGq DC1D00oe8j397oNlVJTOgtwVSp0egG1RdnChT9c1u6tm77CjwlS7sy.nZTyUf2WYRtrPbiAEw1qv iGi5wDwAiRE_z6Qk0SYz1xIDi_8pRtE3RHV2rxv3.GJ1.oWxxxkY8PQqfvl19izcjLe4nmxxQYOg XL5sGgZubyFVPC53GzA2.YszwP4qY_PhvePlYxv8xGvlcJtxlRzdAZjjmHSWfD0zr4kIQLQl1vdR g6vRV16sE9Ea6s_GndHYEVPJypmxfN0HDcEQ.PRmbJeA4HkxZLiMUO_SizzvZgQMFYi7H_ginTKp iEVXdq665eyZjmaKutIjr_H0HLkkjT7_03etl1TiXRAzvkK.h8LpS5MWi2dTFccUpPJETZ6rk1jr Ip4x5168f9UafZ65vBvA6lSvUPqzpRuInjxF8TsP_XSwPIfQ8CznzxqpiBuWhYxgnS68y.33490B 63s1JHu1xipLxXFjAHbmwbIYOL2O_YlelX6fMVzyaIdAXmwg11Ep9W1Xkh5M37sKkDfPBy4kNXtR Xx4Wut83aQWbQHZbumT5Zwm5D_15SbyQEs.zfXNZS9CxCxfoedJZuTNvFw13.UJG71SuncbUwlvl nZLm_OV3QKM.g.n0aBLKLEHninj._IQkn1ztOIJlgfppFymPc9ISqJgxOdSgXM7K1l7o3IDj3hsn 1f7lJRB_8KqE2f8IrbcFuTia5qa6oA98fSD5lxoGCdjv1mZb_ce6uanpiqBX2TND_fUI3CSV9Xpx 4sXYyPlpIXXxDdjo1MPvENp70Wczn.Romlad23L22qfnKizdt51Fli1YM7eOcfAtxbhhjM4bK1DP EH.Zxvj5ZBqQzo0s7vv2PDMs6m_SZ2wwnBKRF411Ez0Krg2.hYCwLUmKI1K0sCiKtjCALJqNv7d3 bzcMMD3Ka4_zYrhd_cGEIG3qrEzLDbiv.ppa_1ke8yN5l6H.jCvEtroXqynrwdZ4P6S4por5IS.q 0x9bwGlH42uVN.luZAWHEeHylwPPZ16D1hzAiYz4jPvO1KOPaeFMID28Q7S8Bd7233QLkbOd44VA 9lx_TL5mvgeoM2jjAwY3cQoXE8bQabw6lEHkefB0i_TGe3QQXA6_b8VqKIprpdYn8KLs.jjCMFx2 9RqE4LwFn3bWgRtYXEWMmSdyIzZEr4AAhY8EAQZM8FMULjdJxpm382RvLT6llEe9OyQjuKVLw0vs fQYbs.iSulUe3feDMylCFUlTXpzwUwXD_2YyrheiOo.J7DrriybuQeFwPZRkp52d2Xz6MJzj31gK DswzO3N.hQHXhpeLpwD4xyDamYUTm_OPri36RH6iM0kp9fcHaV.A4WFxB.A.mxpFK5.1mJtd2H9e 0Ozshno7XsuZc8sEHOjVwXI2Vqtd9K8O39j3sYH7NqIAjq7BVM1ubeQhNW8ZqojfVBKzakonrW6j scgeAN2q86fQy0Qzhqqm_g8eTt0OjKXKUyl1kVjwvQNAUZLmbCRbpopFLD8zHg0PgMye1V34EYjl rlUq_k8iPekcHvRLzd.Z0FzhDlqYYeJaF5YBf7hOWGJTH_CVY5yFR4_6r02qAMD3Hl4zNJRCNhD_ M6dc3cirkUtiFDmOCRSr3HN7J49vkX3N6Ms_CQORyinAZHMj5wHYMBTemx4hqKIG1E3iM5H6mrhF 2O7lK7Nt9q3o0DFD5ci2TasOkG4hCd8zfyBNn_kPwTld30R_8Gy0L_zUeqnlS01KsfpzR_ukLJQ- - X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Mon, 15 Mar 2021 22:54:57 +0000 Received: by smtp421.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 48b1efc5d9231444b6b61852b155321c; Mon, 15 Mar 2021 22:54:54 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Re: Filesystem operations slower in 13.0 than 12.2 From: Mark Millard In-Reply-To: Date: Mon, 15 Mar 2021 15:54:51 -0700 Cc: Warner Losh , Konstantin Belousov , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: References: <12705C29-53EA-4484-8291-C409AF4B3DE5.ref@yahoo.com> <12705C29-53EA-4484-8291-C409AF4B3DE5@yahoo.com> To: Kevin Oberman X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4DzsFH3Qc1z4ZM5 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2021 22:54:59 -0000 On 2021-Mar-15, at 14:57, Kevin Oberman wrote: > Responses in-line. >=20 > On Sun, Mar 14, 2021 at 3:09 PM Mark Millard = wrote: >=20 >> On 2021-Mar-14, at 11:09, Kevin Oberman = wrote: >>=20 >> > . . . >> > =20 >> > Seems to only occur on large r/w operations from/to the same disk. = "sp >> > big-file /other/file/on/same/disk" or tar/untar operations on large = files. >> > Hit this today updating firefox. >> >=20 >> > I/O starts at >40MB/s. Dropped to about 1.5MB/s. If I tried doing = other >> > things while it was running slowly, the disk would appear to lock = up. E.g. >> > pwd(1) seemed to completely lock up the system, but I could still = ping it >> > and, after about 30 seconds, things came back to life. It was also = not >> > instantaneous. Disc activity dropped to <1MB/s for a few seconds = before >> > everything froze. >> >=20 >> > During the untar of firefox, I saw; this several times. I also = looked at my >> > console where I found these errors during : >> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 55043, size: = 8192 >> > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 51572, size: = 4096 >>=20 >> Does anyone know: >> Are those messages normal "reading is taking a rather long >> time" notices or is their presence more useful information >> in some way about the type of problem or context for the >> problem? >>=20 > As for the tests: > Are these messages always present when near a time frame > when the problem occurs? Never present in a near time > frame to a period when the problem does not occur? > In a large number of test, these errors have not repeated. They baffle = me for another reason. This system has 20G or RAM. Tyically, all swap = space is unused. ATM I see 16384M free out of 16384. Not sure that I = have ever seen it used, though it might have been while building rust. I = have not built rust for a month. >=20 > It appears that the messages are associated with reading > the disk(s), not directly with writing them, where the > reads take more than "hz * 20" time units to complete. > (I'm looking at main (14) code.) What might contribute > to the time taken for the pending read(s)? > The reference to hz * 20 woke up a few sleeping memory cells. I forgot = that I cleaned up my loader.conf. It was largely a copy of the one on my = decade-old T520. I commented out "kern.hz=3D100". I don't recall the = details, but I think it was actually from an even older system, my T42 = from before I retired. >=20 > In any case, restoring this setting has greatly improved the = situation. I now have really bad disk I/O performance on large disk to = disk activity (untarring the firefox distro) instead of terrible = performance and the system freezes have vanished, though I do see pauses = in response to clicks or text entry, but the display remains active and = the pauses are short... 1 to 15 seconds, I'd guess. No, I have no idea = what this indicates. Interesting. > I'm still not seeing the performance I was seeing back in February = when 40 MB/s for extended intervals was common and I once untarred = firefox.tar.gz2 in under a minute and performance seldom dropped below = 1.4 MB/s. > =20 >>> /* >>> * swap_pager_getpages() - bring pages in from swap >>> * >>> * Attempt to page in the pages in array "ma" of length = "count". The >>> * caller may optionally specify that additional pages = preceding and >>> * succeeding the specified range be paged in. The number of = such pages >>> * is returned in the "rbehind" and "rahead" parameters, and = they will >>> * be in the inactive queue upon return. >>> * >>> * The pages in "ma" must be busied and will remain busied upon = return. >>> */ >>> static int >>> swap_pager_getpages_locked(vm_object_t object, vm_page_t *ma, int = count, >>> int *rbehind, int *rahead) >>> { >>> . . . >>> /* >>> * Wait for the pages we want to complete. VPO_SWAPINPROG = is always >>> * cleared on completion. If an I/O error occurs, = SWAPBLK_NONE >>> * is set in the metadata for each page in the request. >>> */ >>> VM_OBJECT_WLOCK(object); >>> /* This could be implemented more efficiently with aflags */ >>> while ((ma[0]->oflags & VPO_SWAPINPROG) !=3D 0) { >>> ma[0]->oflags |=3D VPO_SWAPSLEEP; >>> VM_CNT_INC(v_intrans); >>> if (VM_OBJECT_SLEEP(object, &object->handle, PSWP, >>> "swread", hz * 20)) { >>> printf( >>> "swap_pager: indefinite wait buffer: bufobj: %p, blkno: %jd, size: = %ld\n", >>> bp->b_bufobj, (intmax_t)bp->b_blkno, = bp->b_bcount); >>> } >>> } >>> VM_OBJECT_WUNLOCK(object); >>> . . . >>>=20 >>> where: >>>=20 >>> #define VM_OBJECT_SLEEP(object, wchan, pri, wmesg, timo) = \ >>> rw_sleep((wchan), &(object)->lock, (pri), (wmesg), (timo)) >>>=20 >>> and: >>>=20 >>> #define rw_sleep(chan, rw, pri, wmesg, timo) = \ >>> _sleep((chan), &(rw)->lock_object, (pri), (wmesg), = \ >>> tick_sbt * (timo), 0, C_HARDCLOCK) >>>=20 >>> (I do not claim to be able to interpret the implications >>> of the code that leads to the messages. But seeing some >>> of the code might prompt a thought by someone that knows >>> the code's context and operation.) >>>=20 >>> > . . . >>> > Backing off to Mar. 4 was not an improvement. My untar did seem = better for a couple of minutes, but then the display froze again for 30 = seconds and disk performance dropped to <1M. >>=20 >> You were able to see the disk performance drop while >> the display was frozen? >>=20 >=20 > No, but it refreshed the display when it un-froze and the refreshed = display showed a one-minute history that showed that data was still = transferring during the pause. >=20 >> It might not be the best for monitoring but I'll ask >> this in terms of top output: Does Inact, Laundry, >> Wired, Free, or other such show anything fairly unique >> for around the problematical time frame(s)? > =20 > These all look pretty normal. Free memory stays at 13G throughout hte = operatioin.=20 >=20 >>> > then things got really bad and behaved in a manner that was = baffling to me. The screen froze again, but stayed frozen after half a = minute. I clicked on a couple of buttons in Firefox to no effect and = then hit ctrl-q to quit. After the long pause, I pressed the power = button to try to force a shutdown. Suddenly, it started unwinding = everything I had done during the freeze. My browser did the updates from = my mouse clicks including quitting. It then switched to a different = workspace from ctrl-alt-right and did a clean shutdown. ????=20 >>> >=20 >>> > Do I also have a graphics issue? Examining log files show no = indication that anything was happening. SMART shows no errors and = reasonable values for everything. No indication of a HW problem. The = system performs well unless I do something that tries a bulk disk data = move. Building world takes about 75 minutes. I just have a very hard = time building big ports. >>=20 >> Almost like things were stuck-sleeping and then the >> sleep(s) finished? > Exactly!=20 Multi-socket (and possibly multi-core) PowerMacs have not historically had the times used for controlling sleeping that can be used across FreeBSD's cpus well matched in any FreeBSD build without extra patches. They suffer threads being stuck-sleeping for periods not intended. This leads to fans running wild and obvious problems during shutdown having timeouts, though there are more consequences than are so obvious. That is where I got the idea for the question: some similarity to operational problems that I've seen when not using my patches that provide a work around matching the times better in my contexts. (I'm told the type of issue is not limited to PowerMacs, but PowerMacs are the only PowerPCish machines I've had access to. Doing the most accurate time matching gets into platform specific operations, no general solution for such accuracy. Similarly, only platform specifics might scale to lots of sockets/cores well, even without trying to be as well matched. My workaround is generic to the range of PowerMacs that I've had access to but is not as accurate about matching the times.) For your context: how many sockets? Cores per socket? Any other information that might be relevant to matching times across sockets/cores? I suppose that the board matters, not just the processor(s) in the sockets. But what all would be appropriate information? I do not know. I'm not sure if the kern.hz=3D100 results fit with this idea or not. (Such was never involved in my PowerMac experiments.) It is only somewhat suggestive evidence as stands. But time mismatches across socket/cores might be a direction for investigation? (Not that I've a great idea for how to investigate such.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-stable@freebsd.org Tue Mar 16 12:34:44 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E0DAE5ABB0F for ; Tue, 16 Mar 2021 12:34:44 +0000 (UTC) (envelope-from matheus@eternamente.info) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4F0CR74sW2z4VlF for ; Tue, 16 Mar 2021 12:34:42 +0000 (UTC) (envelope-from matheus@eternamente.info) Received: from [10.1.7.11] (unknown [187.61.214.250]) by hobbes.arroway.org (Postfix) with ESMTPA id C911E14D1C1 for ; Tue, 16 Mar 2021 09:34:34 -0300 (-03) Received: from 10.1.4.100 (SquirrelMail authenticated user matheus) by 10.1.7.11 with HTTP; Tue, 16 Mar 2021 09:34:35 -0300 Message-ID: <9ddc468be2d1ad2306bb996749a86b6e.squirrel@10.1.7.11> In-Reply-To: <3b55484103f1833743a1fbbd504b0c38.squirrel@10.1.7.11> References: <3b55484103f1833743a1fbbd504b0c38.squirrel@10.1.7.11> Date: Tue, 16 Mar 2021 09:34:35 -0300 Subject: Re: FreeBSD 13.0-RC2 Now Available From: "Nenhum_de_Nos" To: freebsd-stable@freebsd.org User-Agent: SquirrelMail/1.4.23 [SVN] MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Rspamd-Queue-Id: 4F0CR74sW2z4VlF X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of matheus@eternamente.info has no SPF policy when checking 173.199.118.77) smtp.mailfrom=matheus@eternamente.info X-Spamd-Result: default: False [0.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[173.199.118.77:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[173.199.118.77:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[eternamente.info]; NEURAL_HAM_SHORT(-1.00)[-1.000]; HAS_X_PRIO_THREE(0.00)[3]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; MID_BARE_IP(2.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2021 12:34:44 -0000 On Sat, March 13, 2021 18:33, Nenhum_de_Nos wrote: > On Fri, March 12, 2021 21:11, Glen Barber wrote: >> === Upgrading === >> >> The freebsd-update(8) utility supports binary upgrades of amd64 and i386 > systems running earlier FreeBSD releases. Systems running earlier > FreeBSD releases can upgrade as follows: >> >> # freebsd-update upgrade -r 13.0-RC2 > > Hi Glen, > > this announce mentions just amd64 and i386 as able to use freebsd-update. > On the BETA2 announce there was a mention of aarch64 as well > (https://lists.freebsd.org/pipermail/freebsd-stable/2021-February/093081.html). > > I then tried to use it now on a 13.0-BETA4 system: > > root@rpi4_bsd:~ # freebsd-update -r 13.0-RC2 upgrade > src component not installed, skipped > Looking up update.FreeBSD.org mirrors... 3 mirrors found. > Fetching metadata signature for 13.0-BETA4 from update4.freebsd.org... > done. Fetching metadata index... done. > Fetching 1 metadata patches. done. > Applying metadata patches... done. > Fetching 1 metadata files... done. > Inspecting system... done. > > The following components of FreeBSD seem to be installed: > kernel/generic kernel/generic-dbg world/base world/base-dbg > > The following components of FreeBSD do not seem to be installed: > > Does this look reasonable (y/n)? > > The system is: > > FreeBSD rpi4_bsd 13.0-BETA4 FreeBSD 13.0-BETA4 #0 > releng/13.0-n244592-e32bc253629: Fri Feb 26 05:53:09 UTC 2021 > root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC > arm64 > > > I did not typed "y" yet. Is it safe to go through with it? > > Thanks. > > matheus Just an update. I finished the update on Raspberry Pi 4B 4GB using freebsd-update and all worked just fine. It worked fine also from BETA4 to RC1. matheus -- "We will call you Cygnus, the God of balance you shall be." From owner-freebsd-stable@freebsd.org Tue Mar 16 13:38:58 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D39C85AD5F7 for ; Tue, 16 Mar 2021 13:38:58 +0000 (UTC) (envelope-from bugs@irregulaire.info) Received: from www231.your-server.de (www231.your-server.de [188.40.28.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0DsG0gxSz4Z9v for ; Tue, 16 Mar 2021 13:38:57 +0000 (UTC) (envelope-from bugs@irregulaire.info) Received: from sslproxy06.your-server.de ([78.46.172.3]) by www231.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92.3) (envelope-from ) id 1lM9uR-0006J0-Dy for freebsd-stable@freebsd.org; Tue, 16 Mar 2021 14:38:55 +0100 Received: from [91.65.101.78] (helo=a68n.lokal) by sslproxy06.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lM9uR-000TqN-AD for freebsd-stable@freebsd.org; Tue, 16 Mar 2021 14:38:55 +0100 Date: Tue, 16 Mar 2021 14:38:46 +0100 From: andrew glaeser To: freebsd-stable@freebsd.org Subject: Upgrade-report: 12.1-RELEASE --> 12.2-RELEASE Message-ID: <20210316143846.57886b7d@a68n.lokal> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: base64 X-Authenticated-Sender: bugs@irregulaire.info X-Virus-Scanned: Clear (ClamAV 0.102.4/26110/Tue Mar 16 12:05:23 2021) X-Rspamd-Queue-Id: 4F0DsG0gxSz4Z9v X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bugs@irregulaire.info designates 188.40.28.11 as permitted sender) smtp.mailfrom=bugs@irregulaire.info X-Spamd-Result: default: False [-0.30 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_DN_NONE(0.00)[]; MIME_BASE64_TEXT_BOGUS(1.00)[]; RCVD_COUNT_THREE(0.00)[3]; MIME_BASE64_TEXT(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[188.40.28.11:from]; R_DKIM_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[188.40.28.11:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; HAS_X_AS(0.00)[]; ASN(0.00)[asn:24940, ipnet:188.40.0.0/16, country:DE]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[91.65.101.78:received]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[irregulaire.info]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[188.40.28.11:from:127.0.2.255]; NEURAL_SPAM_SHORT(1.00)[0.999]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-Mailman-Approved-At: Tue, 16 Mar 2021 16:31:32 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2021 13:38:58 -0000 LS0tLS1CRUdJTiBQR1AgU0lHTkVEIE1FU1NBR0UtLS0tLQ0KSGFzaDogU0hBMQ0KDQoNCkluIGEg d2hpbGUgSSBoYWQgbm90IGluc3RhbGxlZCBhbnkgcGFja2FnZS11cGdyYWRlcywgbm8gaWRlYSBo b3cNCmRucy1yZXNvbHZpbmcgYmVjYW1lIGRlLXJlZ2lzdGVyZWQsIHdoZW4gdGhhdCB3YXMgc29s dmVkIGJ5IG1hbnVhbGx5DQpjaGFuZ2luZyAvZXRjL3Jlc29sdi5jb25mLCBzeXMtdXBncmFkZSB3 YXMgYSBwdXJlIHBsZWFzdXJlLCBpdCB3YXMgc28gZ29vZCwNCmJvcmluZyBteXNlbGYgdG8gZGVh dGggYWxtb3N0IGluIHRoZSB3YWl0aW5nLXRpbWUsIGp1c3QgcGVyZmVjdCAoaW5zaWRlIHRoZQ0K ZnJhbWUgb2YgdG9kYXkncyBwb3NzaWJpbGl0aWVzLCBvZiBjb3Vyc2UpOg0KDQpbc2VlIG15IHNl c3Npb24gdHJhbnNzY2lwdDpdDQoNCmh0dHBzOi8vYXR0YWNobWVudC5pcnJlZ3VsYWlyZS5pbmZv L0ZCU0QxMi4xLXVwZ3JhZGUtcmVwb3J0LnR4Lnh6DQoNCkdyZWV0aW5ncywgd2UgbmVlZCBtdWNo IG1vcmUgYm9yZWRvbSBhbmQgdW5zcGVjdGFjdWxhciBzb2Z0d2FyZS11cGdyYWRlcywgZXZlbg0K c3lzdGVtLXVwZ3JhZGVzIHdpdGhvdXQgYW55IHJlZ3Jlc3Npb25zIQ0KDQoNCg0KLS0tLS1CRUdJ TiBQR1AgU0lHTkFUVVJFLS0tLS0NCg0KaUYwRUFSRUNBQjBXSVFURjl1TmFzbHZuSnBXdDhrWG42 c0VmSlMzbkN3VUNZRkMwNWdBS0NSRG42c0VmSlMzbg0KQzQzSUFKNDFURmVWZVozVHpXaTBUQm5J dFNVMmZiUlE1QUNlT0hjT2JRWWZoTTZDU0sxakV0ZjNGcHN4Z2ZjPQ0KPTBMOU0NCi0tLS0tRU5E IFBHUCBTSUdOQVRVUkUtLS0tLQ0K From owner-freebsd-stable@freebsd.org Tue Mar 16 18:57:21 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CF66F568E7B for ; Tue, 16 Mar 2021 18:57:21 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qv1-xf29.google.com (mail-qv1-xf29.google.com [IPv6:2607:f8b0:4864:20::f29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0Mwc6xgpz3Fqw for ; Tue, 16 Mar 2021 18:57:20 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-qv1-xf29.google.com with SMTP id h3so137482qvh.8 for ; Tue, 16 Mar 2021 11:57:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=+fC4I/vQFORqkZATq5eWCY1N2moS8GkSY2WVnhGXj7M=; b=rLGlTVNt6MSL3u7gomrj2TNpbNJ6FRnLJrLpyOFahPouWk9DURmOYyVeSEEAFuw9+Q 8kzmtmM5ys5OBGChnrw16Z5amEl2wUt4MuAcufK4HB2Pq814bzuGSSlx/FgPN+ebBE6A x6g6rSbpv8FmZwAPWIRhbzlvQK9pTHsfYvuEGLmt98pRR1V0HAB95bopILrxtTXbuVi6 GVvTiFgM6iy4EizRBPv3qCK1xXT8WvfPOX4Je5tK9I0vdAw2wkTjW028bDMWve30xHuc k+3pUZPRTL0x4vTJbxhmdPEpbo2+qgFEbczoc6cP00GAhjqX4T4wfHl4fE4A8CSQsWVz 7riA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=+fC4I/vQFORqkZATq5eWCY1N2moS8GkSY2WVnhGXj7M=; b=gK+J3AafTBd7mfiXHUspOCEBYsEJcxUhOHHfciepkS4P05bFDtVbPLk6DIvZicOsX0 pyIIdK5C/VX8pWbOsUFDF+gPefuVa9S2+3rYuQDVFNHP07w8AZTYdtLjuA0UekcjhbXu HZDRP352tT3PZnNVY5scCHfeZ+CdwI4eapb0lsuV2X1kYEyzuzdrrUfLiSgEbM4LNK61 42enLlsKVVAlE9LzPFlBbVKpiQ3kWRWCuZJ9t+9UHxEbK+4/VlWuMd+ax3cF8NC51aNf lZWeC6BBDUXIO5z/yN9zNdbzAFCeHElGL9MSzhpFjEH1qj2VXB8+UANVcAOenIybrsup /Drw== X-Gm-Message-State: AOAM533HL+V3xsOrsu/Js0OmdUvatv25AgvGQRLUfsrDwcY0oExyijUO BnzpFrxfGLPZGSbRxNpxn5Zcng0As80CQsgDYxW6Nsvj X-Google-Smtp-Source: ABdhPJy1AKc1GFJ7Dg7o6QA3oH7WvMM7dUWBR+7IOV/N4xraZ7yW9UQTE5WCrKXF3vGtM8TGXOUpNy+QvzMXcyTqooM= X-Received: by 2002:a05:6214:d4b:: with SMTP id 11mr1263457qvr.42.1615921039809; Tue, 16 Mar 2021 11:57:19 -0700 (PDT) MIME-Version: 1.0 From: Freddie Cash Date: Tue, 16 Mar 2021 11:57:09 -0700 Message-ID: Subject: freebsd-update not removing old libraries To: FreeBSD Stable X-Rspamd-Queue-Id: 4F0Mwc6xgpz3Fqw X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rLGlTVNt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of fjwcash@gmail.com designates 2607:f8b0:4864:20::f29 as permitted sender) smtp.mailfrom=fjwcash@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::f29:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::f29:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f29:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2021 18:57:21 -0000 There seems to be a bug in freebsd-update on 11.x and 12.x systems where it's not removing old library files in the final "freebsd-update install" run. System 1: 10.2 upgrade to 11.2. /lib/libreadline.so.8 is left behind, which breaks bash pkg. 11.2 upgrade to 11.4. /lib/libreadline.so.8 is left behind, which breaks bash pkg. 11.4 upgrade to 12.2. /lib/libreadline.so.8 is left behind, which breaks bash pkg. System 2: 11.2 upgrade to 12.2. /lib/libreadline.so.8 doesn't exist, no issues with any packages. This one has the source tree installed as I needed it to compile the openzfs port, and I manually ran "make delete-old" and "make delete-old-libs" on this one. System 3: 11.x upgrade to 12.2. /lib/libreadline.so.8 left behind, which breaks bash pkg. I don't remember which specific minor version of 11 was installed on system 3, but it was most likely 11.2. No source tree is installed on any of the broken systems (1 and 3). They are strictly binary upgrades/pkg installs. How would one go about running the equivalent of "make delete-old" and "make delete-old-libs" on these systems, when "freebsd-update install" after a "pkg upgrade -f" doesn't do anything? I'd prefer to keep the source tree off these, but if I have to install it temporarily to run delete-old/delete-old-libs, then so be it. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@freebsd.org Wed Mar 17 05:48:04 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 483DA57B067 for ; Wed, 17 Mar 2021 05:48:04 +0000 (UTC) (envelope-from rollingbits@gmail.com) Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0fMR1kpFz4gqD for ; Wed, 17 Mar 2021 05:48:03 +0000 (UTC) (envelope-from rollingbits@gmail.com) Received: by mail-qk1-x72d.google.com with SMTP id a9so37750183qkn.13 for ; Tue, 16 Mar 2021 22:48:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=RG/SA134pNdiYaMyaE/V/pH6OSR9C5DyVZWJ4qB2+TE=; b=H0tZGNGXeyqo8358o6GbRdE2ySc6PMFF5WWPZs5DqSTATy+1P1sSRF5/kqJ7hmMwNj clxeZtW3OT2IROc4riCYZakJNCwCNj/niVlr7JhFAa+TxVOWWhvgpR1dD7UYVqHRApWD 1FjCsaJVDSawUQH2G0b2r5ziw4YdKYCfXX87JC+J/0UiArXl77cbuMaf+OjJ67DyWjWh u821zAjt40Tei12MqImXohNnx81NUqOgjHgYd24acbNL2RXPqUKOcopXNn42zLOMZA1a oQIicDCir0eMdiO65p3yPSmpZL7LYdVmXsTzIOLdXrQ9lvjS673NmI6jnaO0YwZvcJoT H7xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=RG/SA134pNdiYaMyaE/V/pH6OSR9C5DyVZWJ4qB2+TE=; b=CIJQ/nZVkGI0OtLcoMReoExV5K7FiLY/uulAKySI1jro84oWp99q4H1r915lDkqygr 3O9efj0zZXQ8ZFQSX/4eq1p7cxoe7EGED61o801iW7T+X26XdWc1BQW72I1lNNCaOQ1D hLgyosU1NZtG3kJ/gX2qwV7vJkOz8sGEr143UKEcsulqu9WK3X0a2t796sVzkIim2fKD 3N+/VKcewVVuzYahjs57oJyC5j+V7PJL1lxJGWhKmflKIQ9/TRUIE56+LEZE57OeM8Ln GoGEgVfQQXsg6sQhyc721aJti47ORpbWa6AuW8pLhaMCQjboNuMDCftMDsHzuChxhG+j 2skw== X-Gm-Message-State: AOAM533P7Jtxk4Ef1tEDRxXlmEFSrhzlNpKzd+E+zetgph3kDTlJ7iPc pcFjerg7dzA6Rk7D/vulyuf+dov0uEc= X-Google-Smtp-Source: ABdhPJyzP2pQR1b2WhPD3lroK1Uz1oHF/njENntOGys2Zr9T046alMzAiy3XP1IrTQAz1Zx+ejj+Ow== X-Received: by 2002:a37:2749:: with SMTP id n70mr2917776qkn.105.1615960082286; Tue, 16 Mar 2021 22:48:02 -0700 (PDT) Received: from [10.31.7.49] (189-92-112-6.3g.claro.net.br. [189.92.112.6]) by smtp.gmail.com with ESMTPSA id j30sm14895736qtv.90.2021.03.16.22.48.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 16 Mar 2021 22:48:01 -0700 (PDT) From: =?utf-8?Q?Lucas_Nali_de_Magalh=C3=A3es?= Mime-Version: 1.0 (1.0) Subject: Re: freebsd-update not removing old libraries Date: Wed, 17 Mar 2021 02:47:59 -0300 Message-Id: References: Cc: FreeBSD Stable In-Reply-To: To: Freddie Cash X-Mailer: iPhone Mail (18D61) X-Rspamd-Queue-Id: 4F0fMR1kpFz4gqD X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=H0tZGNGX; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rollingbits@gmail.com designates 2607:f8b0:4864:20::72d as permitted sender) smtp.mailfrom=rollingbits@gmail.com X-Spamd-Result: default: False [-1.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::72d:from]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[189.92.112.6:received]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::72d:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72d:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 05:48:04 -0000 > On Mar 16, 2021, at 3:57 PM, Freddie Cash wrote: >=20 > =EF=BB=BFThere seems to be a bug in freebsd-update on 11.x and 12.x system= s where > it's not removing old library files in the final "freebsd-update install" > run. >=20 > System 1: > 10.2 upgrade to 11.2. /lib/libreadline.so.8 is left behind, which breaks > bash pkg. >=20 > (=E2=80=A6) >=20 > No source tree is installed on any of the broken systems (1 and 3). They > are strictly binary upgrades/pkg installs. Do you tried an extra round of "freebsd-update install" ? --=20 rollingbits =E2=80=94 =F0=9F=93=A7 rollingbits@icloud.com =F0=9F=93=A7 rolli= ngbits@gmail.com =F0=9F=93=A7 rollingbits@yahoo.com =F0=9F=93=A7 rollingbits= @terra.com.br =F0=9F=93=A7 rollingbits@globo.com From owner-freebsd-stable@freebsd.org Wed Mar 17 06:23:57 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A641057BAAD for ; Wed, 17 Mar 2021 06:23:57 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0g8r4M3Kz4jq4 for ; Wed, 17 Mar 2021 06:23:56 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-qk1-x736.google.com with SMTP id 130so37827877qkh.11 for ; Tue, 16 Mar 2021 23:23:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=x/tSYAsbDUGwPPevWlCg/qozLlG6uvmQTA8v2ltc2mw=; b=c9SmaIUEpRDtbuiIFRtZ1W+Ai1w4ySnGC3Ok7QHLTLvC1E5FcEnocIGqv7iVQsQ89g AZT1GJDbQ+ouxk3CYyT3iGgIE00tft7WoxL8MFYGTLrzBq7q3fyIQIfnA8P8AdnG2bNR zsIOcUn6ixYVZDrZWvvzbOC5a6hR9XqoWXONvpSIwJ5S0uAydBGYlXQ7bWDzD1XUBJAg Ho71OEB68jJHspxf4fKJUhXAA5JxQ0xsUw2UE4IkBSYngZ2MG6x8I5d6vmQ7mgfgi/Pf YiDhia6zLLoa2BSpVFKAUidkgcapXdZAUokEiD/IzKqCqqNvtnwWFdwQ2Bepb9TVOZHa +27A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=x/tSYAsbDUGwPPevWlCg/qozLlG6uvmQTA8v2ltc2mw=; b=gaM9rnbEn4aj/oCTDo4UmvP6+eJN0H52KOevyYHMJ9kJ0QfpbC0ONo03oYvsyG8ilo drbOHRdKzNxGT5HNIpNBFA6XWqJDnqks60kgppv1FAoECdyWJsZnyvW6OwbuIs1Jmv2d jL8dZy1Zb0dSOOp4vr/oRpkV5y9sRPefgVoDnfmSAoZHAVZn3YCTQbG0Y4zwRtTCYBZy wcJz4KKKGghw2SjvCMhhx2HT1xy9tXXOhb8pDClwbBmXVSy1SCZBi3ANdHy6VFmrQ4FS jlaet5ZNnaS5YE7g9J4P/sw46xGgIB1ITQcVizuv2Dicli7iQAdXPQ3Y1uyLYLIaEKHi betw== X-Gm-Message-State: AOAM5312jP2BOGDPnIawlqvnyj5bfv1zMvnE6gcviOZS+8+I/dHmd9aU Gf8QJm5KXflqUO27hk8MVwNFYW3V+1+H0BgtlCo= X-Google-Smtp-Source: ABdhPJysa8/i3CvkXOyE5zZiRp6lK7FuKTAt05fe+7LzoUG9td9p2TRGQAIBIDYLfehDnVT1oI2t50MSKmYKkHlaHV0= X-Received: by 2002:a37:392:: with SMTP id 140mr3089947qkd.236.1615962235627; Tue, 16 Mar 2021 23:23:55 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Freddie Cash Date: Tue, 16 Mar 2021 23:23:45 -0700 Message-ID: Subject: Re: freebsd-update not removing old libraries To: =?UTF-8?Q?Lucas_Nali_de_Magalh=C3=A3es?= Cc: FreeBSD Stable X-Rspamd-Queue-Id: 4F0g8r4M3Kz4jq4 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=c9SmaIUE; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of fjwcash@gmail.com designates 2607:f8b0:4864:20::736 as permitted sender) smtp.mailfrom=fjwcash@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::736:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::736:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::736:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 06:23:57 -0000 On Tue., Mar. 16, 2021, 10:48 p.m. Lucas Nali de Magalh=C3=A3es, < rollingbits@gmail.com> wrote: > On Mar 16, 2021, at 3:57 PM, Freddie Cash wrote: > > =EF=BB=BFThere seems to be a bug in freebsd-update on 11.x and 12.x syste= ms where > it's not removing old library files in the final "freebsd-update install" > run. > > System 1: > 10.2 upgrade to 11.2. /lib/libreadline.so.8 is left behind, which breaks > bash pkg. > > (=E2=80=A6) > > No source tree is installed on any of the broken systems (1 and 3). They > are strictly binary upgrades/pkg installs. > > > Do you tried an extra round of "freebsd-update install" ? > Yeah, many times. Just comes back with the "no updates to install, run fetch first" message. Cheers, Freddie Typos due to smartphone keyboard. From owner-freebsd-stable@freebsd.org Wed Mar 17 13:12:04 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EEDEB5AEA8C for ; Wed, 17 Mar 2021 13:12:04 +0000 (UTC) (envelope-from anonymous@m3.coreserver.jp) Received: from m3.coreserver.jp (m3.coreserver.jp [202.172.26.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0rCk0XyBz3mNX for ; Wed, 17 Mar 2021 13:12:01 +0000 (UTC) (envelope-from anonymous@m3.coreserver.jp) Received: (qmail 375278 invoked by uid 10348); 17 Mar 2021 22:11:59 +0900 Date: 17 Mar 2021 22:11:59 +0900 Message-ID: <20210317131159.375276.qmail@m3.coreserver.jp> To: freebsd-stable@freebsd.org Subject: =?UTF-8?B?QWNjb3VudCBEZS1hY3RpdmF0aW9uIGZvciBmcmVlYnNkLXN0YWJsZUBmcmVlYnNkLm9yZy4gKEZpbmFsIFdhcm5pbmcpLg==?= From: =?UTF-8?B?Q3BhbmVsIEVtYWlsIFNlcnZlciBBZG1pbmlzdHJhdG9y?= <> MIME-Version: 1.0; X-Rspamd-Queue-Id: 4F0rCk0XyBz3mNX X-Spamd-Bar: ++++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of anonymous@m3.coreserver.jp designates 202.172.26.4 as permitted sender) smtp.mailfrom=anonymous@m3.coreserver.jp X-Spamd-Result: default: False [6.01 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[202.172.26.4:from]; GREYLIST(0.00)[pass,meta]; FROM_HAS_DN(0.00)[]; FROM_EXCESS_BASE64(1.50)[]; R_SPF_ALLOW(-0.20)[+a:m3.coreserver.jp:c]; SUBJ_EXCESS_BASE64(1.50)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[No domain in From header]; TO_DN_NONE(0.00)[]; ENVFROM_SERVICE_ACCT(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[202.172.26.4:from:127.0.2.255]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.987]; CTYPE_MIXED_BOGUS(1.00)[]; NEURAL_SPAM_LONG(1.00)[1.000]; FORGED_SENDER(0.30)[,anonymous@m3.coreserver.jp]; RCVD_COUNT_ZERO(0.00)[0]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:~,2:+]; ASN(0.00)[asn:37907, ipnet:202.172.24.0/21, country:JP]; FROM_NEQ_ENVFROM(0.00)[,anonymous@m3.coreserver.jp]; MAILMAN_DEST(0.00)[freebsd-stable] X-Spam: Yes Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 13:12:05 -0000 From owner-freebsd-stable@freebsd.org Wed Mar 17 13:33:30 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E64BE5AF898; Wed, 17 Mar 2021 13:33:30 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (mail1.yamagi.org [IPv6:2001:19f0:b001:853::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0rhT3RZxz3np8; Wed, 17 Mar 2021 13:33:29 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from [2001:470:6845:1:7f9f:3dc1:2752:4334] (helo=killua.home.yamagi.org) by mail1.yamagi.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94 (FreeBSD)) (envelope-from ) id 1lMWIb-000G2Z-EX; Wed, 17 Mar 2021 14:33:22 +0100 Date: Wed, 17 Mar 2021 14:33:07 +0100 From: Yamagi To: freebsd-current@freebsd.org Cc: freebsd-stable@freebsd.org Subject: 13.0-RC2 / 14-CURRENT: Processes getting stuck in vlruwk state Message-Id: <20210317143307.20beb5fca0814246f2a91e9a@yamagi.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA256"; boundary="Signature=_Wed__17_Mar_2021_14_33_08_+0100_n+tlFzmG7X+B+oes" X-Rspamd-Queue-Id: 4F0rhT3RZxz3np8 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lists@yamagi.org designates 2001:19f0:b001:853::3 as permitted sender) smtp.mailfrom=lists@yamagi.org X-Spamd-Result: default: False [-2.90 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:19f0:b001:853::3:from]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; SPAMHAUS_ZRD(0.00)[2001:19f0:b001:853::3:from:127.0.2.255]; DMARC_NA(0.00)[yamagi.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:20473, ipnet:2001:19f0:b000::/38, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 13:33:31 -0000 --Signature=_Wed__17_Mar_2021_14_33_08_+0100_n+tlFzmG7X+B+oes Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, me and some other users in the ##bsdforen.de IRC channel have the problem that during Poudriere runs processes getting stuck in the 'vlruwk' state. For me it's fairly reproduceable. The problems begin about 20 to 25 minutes after I've started poudriere. At first only some ccache processes hang in the 'vlruwk' state, after another 2 to 3 minutes nearly everything hangs and the total CPU load drops to about 5%. When I stop poudriere with ctrl-c it takes another 3 to 5 minutes until the system recovers. First the setup: * poudriere runs in a bhyve vm on zvol. The host is a 12.2-RELEASE-p2. The zvol has a 8k blocksize, the guests partition are aligned to 8k. The guest has only zpool, the pool was created with ashift=3D13. The vm has 16 E5-2620 and 16 gigabytes RAM assigned to it. * poudriere is configured with ccache and ALLOW_MAKE_JOBS=3Dyes. Removing either of these options lowers the probability of the problem to show up significantly. I've tried several git revisions starting with 14-CURRENT at 54ac6f721efccdba5a09aa9f38be0a1c4ef6cf14 in the hope that I can find at least one known to be good revision. No chance, even a kernel build from 0932ee9fa0d82b2998993b649f9fa4cc95ba77d6 (Wed Sep 2 19:18:27 2020 +0000) has the problem. The problem isn't reproduceable with 12.2-RELEASE. The kernel stack ('procstat -kk') of a hanging process is: mi_switch+0x155 sleepq_switch+0x109 sleepq_catch_signals+0x3f1 sleepq_wait_sig+0x9 _sleep+0x2aa kern_wait6+0x482 sys_wait4+0x7d amd64_syscall+0x140 fast_syscall_common+0xf8 The kernel stack of vnlru is changing, even while the processes are hanging: * mi_switch+0x155 sleepq_switch+0x109 sleepq_timedwait+0x4b _sleep+0x29b vnlru_proc+0xa05 fork_exit+0x80 fork_trampoline+0xe * fork_exit+0x80 fork_trampoline+0xe Since vnlru is accumulating CPU time it looks like it's doing at least something. As an educated guess I would say that vn_alloc_hard() is waiting a long time or even forever to allocate new vnodes. I can provide more information, I just need to know what. Regards, Yamagi --=20 Homepage: https://www.yamagi.org Github: https://github.com/yamagi GPG: 0x1D502515 --Signature=_Wed__17_Mar_2021_14_33_08_+0100_n+tlFzmG7X+B+oes Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEOXu/lxyufwz0gC5x6xRy5x1QJRUFAmBSBRQACgkQ6xRy5x1Q JRW/cQ//VlgtFrHyRBWN3zVg6EfJbPZMJd9Ky9y7uFltYmWorK4bDjfU1n1XQlrn 0q7+c5ejtVCva3DZ7APSPse23QI6tR8iRbba8wCnj+2pnB+x/b4czUOg8mSVST9I JShktEcHUoYgoC7hzJDVMbqMR+DTkRl/Mp9EC3psu/mRglEw5LVy5Qq8IGTlvEU4 FVOXlS1+DEAJD/lCqS4sgut+2z3sm/EAbmqrsB1+6CHuZcbxmd0Zi14UFMkLNSDK kitZTp1SruKg1BEjyT8Vsfa98vB6/IYQdUsv5agjxA6/ClsnN+VLZXa7xbOEbPUw l3XxKqySy/+tX9SGMnbArS7E+E0/6mBMNVRxWCEPu9qQuShfmGXOhDDxaIFqVMGt AmF1hmiEF5Z8QiLcdHx+gtYjW0tAJfgTRSQRozQ44dLoTAI67S1tVnZ3Z5m9AHOz 4PzSvnIL2+ScCDzyJraT/vr2u6Z5hKCQflWh/1jhDH1J87QlvbQ9ZKoaMMKHYMg4 Fyi/Xe5dlUN1YAzci8rag4WBcT6NJiL/u/S5hsXiPZ8vCSo/9hj6KaVfq8RUSLHO mOe387YdBlmz4PiPFBeZG9vfeczUgQ29XxYLWi9B1I/waJ3Iea5Py4oLHJii1UTO 2ZsKeZWn175gQ8WUOuxC+7ZbzVc/B/vSdtmC1xXciR80NGm+jMY= =AiJB -----END PGP SIGNATURE----- --Signature=_Wed__17_Mar_2021_14_33_08_+0100_n+tlFzmG7X+B+oes-- From owner-freebsd-stable@freebsd.org Wed Mar 17 14:59:14 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6D44356BDD6 for ; Wed, 17 Mar 2021 14:59:14 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0tbP6xJgz4V95 for ; Wed, 17 Mar 2021 14:59:13 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 9D0122373 for ; Wed, 17 Mar 2021 10:59:12 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 17 Mar 2021 10:59:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h= date:from:to:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm2; bh=BRReFCox9nKCceolZI3LpCEesW6 c43dEj3IuaeLn29U=; b=DhrtzVKBomlkF4t8PiscFsoCjb565wSngQ2+pmCqTic LbigikoL9ajXjwgnHLtaJAXFO27BKmwjRtSJWJaQQcJIXlbBXDMgsXx4Adu9/FUH RJu16WT//oiPpvf6KVS05/NxB1iBF0u6gSmQVqwcgIExosQGMXbOWzK8JUloUjK4 EZeEUZzKtVbhzBLk9faugcCGFYYbwe8W0vOhz1ys2pzXQMi17qnIGSNL0OaPZIpg CUTv0dSwSZnbmSiWrA/nV4qys5ZHfOZE+SxWKcbEyhzTdBukNHE6cPPeY5UwYHBs SqN4+YXhS82/PtXJUod/8qL0ucrO47geBuNA1BtD0yg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=BRReFC ox9nKCceolZI3LpCEesW6c43dEj3IuaeLn29U=; b=aj37vVLv02IyFoz4jFCZ0z GeWRbkkTHWLn9JjaJAmN9eC1OFiMC7APerHscilfCCVX3dTWiw5mcvBGcOEyWnMS YFlmKMfKi9eha7cYUXojUVrDmy3Ghsy/w+Q3se9LNtIjWEM+3acgJP5lz9FgYC3p Kp2DBKUmyiz0cE3xs2PMei2hLHr9dh8SVNec/0L/6Rdu/qViJ3TIBmHEKMGtkXPS SZbbFna3Z+wwGaOvHicchbUyBXbyhIM0Zv0rrpgL0jUzzj4sbBJsqx+oNFd71CiX 48/AQ+NDbG8Yb5SCG15GNfLO9x0kmL6KZa8sUW3Uj6EFSzUUECf3qbltZ2KcUIJg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudefgedgjedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesghdtre ertddtvdenucfhrhhomhepthgvtghhqdhlihhsthhsuceothgvtghhqdhlihhsthhsseii hiigshhtrdhnvghtqeenucggtffrrghtthgvrhhnpedtheeigfdvudefkeekvddtfedvte dttdekuddvgeevlefftdekffdujedvhfduteenucfkphepkedvrdejtddrledurddutddt necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepthgvtg hhqdhlihhsthhsseiihiigshhtrdhnvght X-ME-Proxy: Received: from ceres.zyxst.net (ceres.zyxst.net [82.70.91.100]) by mail.messagingengine.com (Postfix) with ESMTPA id 979EC24006A for ; Wed, 17 Mar 2021 10:59:11 -0400 (EDT) Date: Wed, 17 Mar 2021 14:59:10 +0000 From: tech-lists To: freebsd-stable@freebsd.org Subject: Re: Updating to 13-stable and existing ZFS pools: any gotchas? Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <9A253CE5-8144-427C-9192-AF3E9ADBA3B4@lassitu.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="UfdymPnpGWYVJzi/" Content-Disposition: inline In-Reply-To: <9A253CE5-8144-427C-9192-AF3E9ADBA3B4@lassitu.de> X-Rspamd-Queue-Id: 4F0tbP6xJgz4V95 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zyxst.net header.s=fm2 header.b=DhrtzVKB; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=aj37vVLv; dmarc=none; spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates 64.147.123.20 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net X-Spamd-Result: default: False [-4.70 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm2,messagingengine.com:s=fm2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[64.147.123.20:from]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.20:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[zyxst.net]; DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.20:from] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 14:59:14 -0000 --UfdymPnpGWYVJzi/ Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Mar 14, 2021 at 09:59:21AM +0100, Stefan Bethke wrote: > I'm planning to upgrade three production machines with existing ZFS pools= =20 > to 13-stable. Is there anything I need to pay attention to wrt OpenZFS?= =20 > Or should it be fully transparent, apart from updating loader? > > My (limited) testing with VMs went without a hitch, but I want to make su= re=20 > I don't paint myself into a corner. Hi, I'm interested in this as well. I'm not using root-on-zfs. The zpool is 5* spinning rust, the booting media is ssd/ufs2. Is updating the bootcode only relevant for root-on-zfs? I've not done that for a similarly configured desktop system, and it seems to be running stable/13 fine. (the desktop was a stable/12 to stable/13 upgrade). thanks, --=20 J. --UfdymPnpGWYVJzi/ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAmBSGTUACgkQs8o7QhFz NAXI8Q//TyMgckwGt9owlJkVYuLHqEGknxwciJvYsGXbrvL2FepRDWpH/dsjkmjn s04zg3SkiO7ERdHTWPl6BmI3i/QnMUk2WigGgpuyZuCLbNOg21EYE4LNVTcBSphP paLHKdEYzow1yXdwxooTK927o1gbKYcubxuBMpKRJbD13KK5+EzF89C/sv94tSqx mE/KR5YZEuRzNgVHuuzE+GGtot0Q1sk+NHfPowSX4mYSFCwbgILAElTnm+f6fY38 ggWYSEAxWGnLJYtuGRvkIqzZWI7T+920TkrTodKCbfBCZUi3TI4Gc8Y09b2Ns5GO ynXbfP+liw7oNeZS2pbqAOg89jPkD5EN6B8/riD+fE3U3ngUMsd/BBZ3oBi/3Wbt 44WHtxJF0V0P2LiTFxnjrES/QP95WKqwcZbHYOE54rhOs3cBHLxITfGJ6RDNiY8a Dq2ieDwLzUrPFgv22WF5K8Dz2q3EyOuTS8Nc1S8hfVhbdAP8em/3CEQGsE5QMqwU Q7cDZ7OP02beSllU5hfLIGuq83r/JTl2kCs6i1i3itv4uvmgf3EyWu1JBGiwHH5d V4kW45jCd63ooladyns+pfvS5ecvjPSn28RyKihapoK1dNcfKEzq4HmRPp4wiYnN V2Sdu3E3OtSdrwoGcPfochxNEUNrU6YnC4afK86akfxITmqyOMI= =gPh5 -----END PGP SIGNATURE----- --UfdymPnpGWYVJzi/-- From owner-freebsd-stable@freebsd.org Wed Mar 17 15:24:19 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3E12E56C63D for ; Wed, 17 Mar 2021 15:24:19 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received: from webmail.dweimer.net (024-240-198-186.biz.spectrum.com [24.240.198.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "dweimer.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0v8L0R1Zz4X5J for ; Wed, 17 Mar 2021 15:24:17 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received-SPF: pass (webmail.dweimer.net: authenticated connection) receiver=webmail.dweimer.net; client-ip=10.9.5.1; helo=www.dweimer.net; envelope-from=dweimer@dweimer.net; x-software=spfmilter 2.001 http://www.acme.com/software/spfmilter/ with libspf2-1.2.10; Received: from www.dweimer.net (pfsense.dweimer.me [10.9.5.1]) (authenticated bits=0) by webmail.dweimer.net (8.16.1/8.16.1) with ESMTPSA id 12HFOBjO006599 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 17 Mar 2021 10:24:11 -0500 (CDT) (envelope-from dweimer@dweimer.net) MIME-Version: 1.0 Date: Wed, 17 Mar 2021 10:24:06 -0500 From: "Dean E. Weimer" To: freebsd-stable@freebsd.org Subject: Re: Updating to 13-stable and existing ZFS pools: any gotchas? Reply-To: dweimer@dweimer.net In-Reply-To: References: <9A253CE5-8144-427C-9192-AF3E9ADBA3B4@lassitu.de> User-Agent: Roundcube Webmail/1.4.11 Message-ID: <6e022fe28d6717caeed28dd142018385@dweimer.net> X-Sender: dweimer@dweimer.net Organization: dweimer.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4F0v8L0R1Zz4X5J X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.00 / 15.00]; HAS_REPLYTO(0.00)[dweimer@dweimer.net]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:24.240.198.184/29]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; HAS_ORG_HEADER(0.00)[]; DKIM_TRACE(0.00)[dweimer.net:+]; DMARC_POLICY_ALLOW(-0.50)[dweimer.net,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[24.240.198.186:from]; ASN(0.00)[asn:20115, ipnet:24.240.196.0/22, country:US]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[dweimer.net:s=2017.01.31]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[24.240.198.186:from:127.0.2.255]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 15:24:19 -0000 On 2021-03-17 9:59 am, tech-lists wrote: > On Sun, Mar 14, 2021 at 09:59:21AM +0100, Stefan Bethke wrote: >> I'm planning to upgrade three production machines with existing ZFS >> pools to 13-stable. Is there anything I need to pay attention to wrt >> OpenZFS? Or should it be fully transparent, apart from updating >> loader? >> >> My (limited) testing with VMs went without a hitch, but I want to make >> sure I don't paint myself into a corner. > > Hi, I'm interested in this as well. > > I'm not using root-on-zfs. The zpool is 5* spinning rust, the booting > media is ssd/ufs2. Is updating the bootcode only relevant for > root-on-zfs? I've not done that for a similarly configured desktop > system, and it seems to be running stable/13 fine. (the desktop was a > stable/12 to stable/13 upgrade). > > thanks, If you are not booting from zfs its seemless. Just remembe once you issue zpool upgrade you wont be able to go back to older version. If booting from ZFS, just update your boot loader. I have updated a couple of systems from 12.2-RELEASE-p3 to 13.0-RC2. One using UEFI and the other using legacy bios with no problems. for UEFI I mounted EFI Boot partition EFI update, New ZFS data set was mounted at ROOT, zpool is mirror. Updated both EFI partitions in case primary goes offline. zpool set bootfs=ssd/ROOT/13.0-RC2 ssd mount -t msdosfs /dev/ada0p1 /mnt cp ROOT/boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi umount /mnt mount -t msdosfs /dev/ada1p1 /mnt cp ROOT/boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi umount /mnt Legacy Bios, again new ZFS data set was mounted at ROOT, and zpool is mirror, updated both boot partitions zpool set bootfs=zroot/ROOT/13.0-RC2 zroot gpart bootcode -b ROOT/boot/pmbr -p ROOT/boot/gptzfsboot -i 1 ada0 gpart bootcode -b ROOT/boot/pmbr -p ROOT/boot/gptzfsboot -i 1 ada1 Everything worked as expected. -- Thanks, Dean E. Weimer http://www.dweimer.net/ From owner-freebsd-stable@freebsd.org Wed Mar 17 15:48:35 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E3EE756D8B8; Wed, 17 Mar 2021 15:48:35 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (mail1.yamagi.org [IPv6:2001:19f0:b001:853::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0vhL6jhsz4ZVC; Wed, 17 Mar 2021 15:48:34 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from [2001:470:6845:1:7f9f:3dc1:2752:4334] (helo=killua.home.yamagi.org) by mail1.yamagi.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94 (FreeBSD)) (envelope-from ) id 1lMYPP-000Gbm-KM; Wed, 17 Mar 2021 16:48:33 +0100 Date: Wed, 17 Mar 2021 16:48:21 +0100 From: Yamagi To: mjguzik@gmail.com Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: 13.0-RC2 / 14-CURRENT: Processes getting stuck in vlruwk state Message-Id: <20210317164821.a65559ba0df6645085466484@yamagi.org> In-Reply-To: References: <20210317143307.20beb5fca0814246f2a91e9a@yamagi.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA256"; boundary="Signature=_Wed__17_Mar_2021_16_48_21_+0100_DEdMUc8dIpFmFhem" X-Rspamd-Queue-Id: 4F0vhL6jhsz4ZVC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lists@yamagi.org designates 2001:19f0:b001:853::3 as permitted sender) smtp.mailfrom=lists@yamagi.org X-Spamd-Result: default: False [-2.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:19f0:b001:853::3:from]; ASN(0.00)[asn:20473, ipnet:2001:19f0:b000::/38, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MIME_UNKNOWN(0.10)[application/x-xz]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; DMARC_NA(0.00)[yamagi.org]; SPAMHAUS_ZRD(0.00)[2001:19f0:b001:853::3:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 15:48:36 -0000 --Signature=_Wed__17_Mar_2021_16_48_21_+0100_DEdMUc8dIpFmFhem Content-Type: multipart/mixed; boundary="Multipart=_Wed__17_Mar_2021_16_48_21_+0100_/VOKigFwl+jSLIdi" --Multipart=_Wed__17_Mar_2021_16_48_21_+0100_/VOKigFwl+jSLIdi Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Mateusz, the sysctl output after about 10 minutes into the problem is attached. In case that its stripped by Mailman a copy can be found here: https://deponie.yamagi.org/temp/sysctl_vlruwk.txt.xz Regards, Yamagi On Wed, 17 Mar 2021 15:57:59 +0100 Mateusz Guzik wrote: > Can you reproduce the problem and run obtain "sysctl -a"? >=20 > In general, there is a vnode limit which is probably too small. The > reclamation mechanism is deficient in that it will eventually inject > an arbitrary pause. >=20 > On 3/17/21, Yamagi wrote: > > Hi, > > me and some other users in the ##bsdforen.de IRC channel have the > > problem that during Poudriere runs processes getting stuck in the > > 'vlruwk' state. > > > > For me it's fairly reproduceable. The problems begin about 20 to 25 > > minutes after I've started poudriere. At first only some ccache > > processes hang in the 'vlruwk' state, after another 2 to 3 minutes > > nearly everything hangs and the total CPU load drops to about 5%. > > When I stop poudriere with ctrl-c it takes another 3 to 5 minutes > > until the system recovers. > > > > First the setup: > > * poudriere runs in a bhyve vm on zvol. The host is a 12.2-RELEASE-p2. > > The zvol has a 8k blocksize, the guests partition are aligned to 8k. > > The guest has only zpool, the pool was created with ashift=3D13. The > > vm has 16 E5-2620 and 16 gigabytes RAM assigned to it. > > * poudriere is configured with ccache and ALLOW_MAKE_JOBS=3Dyes. Removi= ng > > either of these options lowers the probability of the problem to show > > up significantly. > > > > I've tried several git revisions starting with 14-CURRENT at > > 54ac6f721efccdba5a09aa9f38be0a1c4ef6cf14 in the hope that I can find at > > least one known to be good revision. No chance, even a kernel build > > from 0932ee9fa0d82b2998993b649f9fa4cc95ba77d6 (Wed Sep 2 19:18:27 2020 > > +0000) has the problem. The problem isn't reproduceable with > > 12.2-RELEASE. > > > > The kernel stack ('procstat -kk') of a hanging process is: > > mi_switch+0x155 sleepq_switch+0x109 sleepq_catch_signals+0x3f1 > > sleepq_wait_sig+0x9 _sleep+0x2aa kern_wait6+0x482 sys_wait4+0x7d > > amd64_syscall+0x140 fast_syscall_common+0xf8 > > > > The kernel stack of vnlru is changing, even while the processes are > > hanging: > > * mi_switch+0x155 sleepq_switch+0x109 sleepq_timedwait+0x4b > > _sleep+0x29b vnlru_proc+0xa05 fork_exit+0x80 fork_trampoline+0xe > > * fork_exit+0x80 fork_trampoline+0xe > > > > Since vnlru is accumulating CPU time it looks like it's doing at least > > something. As an educated guess I would say that vn_alloc_hard() is > > waiting a long time or even forever to allocate new vnodes. > > > > I can provide more information, I just need to know what. > > > > > > Regards, > > Yamagi > > > > -- > > Homepage: https://www.yamagi.org > > Github: https://github.com/yamagi > > GPG: 0x1D502515 > > >=20 >=20 > --=20 > Mateusz Guzik --=20 Homepage: https://www.yamagi.org Github: https://github.com/yamagi GPG: 0x1D502515 --Multipart=_Wed__17_Mar_2021_16_48_21_+0100_/VOKigFwl+jSLIdi-- --Signature=_Wed__17_Mar_2021_16_48_21_+0100_DEdMUc8dIpFmFhem Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEOXu/lxyufwz0gC5x6xRy5x1QJRUFAmBSJMUACgkQ6xRy5x1Q JRX8CBAAtm3zBXsxt+dcfDOXr2xMq2pwEM2mvmNkUYwwPv7puP44Awx/CzaZ7e02 uLqxfdujyP5w7oVTekTDbGx6KvbJipd4gGacbkaKOGhL3p66eYH/rECztkfA0xMA y6YacNIlYYHuwyTKvG9BjbhDNDWz7xmDz4WZxsnPIdWShMFtri0LHkiOztCcN8pc Hb5n78MplG+Q5qlvE0zDykhS1PrYJy+TZwk/x+4FUISEilzfZ2hyAOoFBYhuCFIM decjPEPGFzGTIiR9j8Sa0/shdVGGn4b6ZBFBBCMRxYFUn6Ar+Z9Q5+/ez+loiKR2 Po/i/eCY2MgcaKn8UyiwR7Ip4HvFgfvHH0WNzZM8CV8xaR6Hfu3zw7vnAsyd82A5 jTZoLKxbknXvQ4A+qq5hmVzJ/Nli+hnf8BUk3/T3EkPCTsqh5hfAv6sewAHV+rtU hXckf7qbzcuqLhflEkC9SQJxT9i0U8lxSE1h5vSYT4TgqTzomIQM1pzJiBLNl76l Zo0lqEcLkT6oGQEjTTBtFY2PW38q2pgY3hJi4yzMxEaJ6FizREXFEND/u0n4+L92 RSrfXOg/AdjFb4e/nWNt/xFxxw6lF4vPq+B55NWD223hCPFBDIFEc01qiS6weWQK 0d1c92lOT2CUX/JNzCD5N2/p6Hn+Wx7GUnfBxWWoEH6LVYO+JFc= =P6Ld -----END PGP SIGNATURE----- --Signature=_Wed__17_Mar_2021_16_48_21_+0100_DEdMUc8dIpFmFhem-- From owner-freebsd-stable@freebsd.org Wed Mar 17 14:58:02 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 98F8D56B6F7; Wed, 17 Mar 2021 14:58:02 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [IPv6:2a00:1450:4864:20::131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0tZ15fRHz4Tkt; Wed, 17 Mar 2021 14:58:01 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-lf1-x131.google.com with SMTP id 20so3392898lfj.13; Wed, 17 Mar 2021 07:58:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=KsqTpO03/iFBNBm5WZJgYwegqTYGCEZsEFOWJXY140I=; b=pA9EKQn44vwovoXnATurdXC81IUtFZToFHCZAe8CJJTV2VvtPlU4sRJ8X6hRnvI/qm xH0Vwoyp9DxeALnuccX3K3Gy1jbBOp2xAzGSEUQuANT7LRsiYjwMkTl0SbNrxtZt7WGQ RTNG4Y3/xOea+cpdcHLlCRK12fiL4qwuNXgf0KA9Othr7j6sKRHIc2hCB0bPh/xk+za8 UTbuq6zniR/TM5YhPw2rFf9yWYa0kwp7F8Rme80O3EjvI3UZOln3zUreuw1dHcB8uwYg MYe5Mz06EqcGzCMPs5uYT/sEuT7btkRa6gnkIbFiRPBwptnB9zko++jM9b5dMlbrOkYQ 5LUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=KsqTpO03/iFBNBm5WZJgYwegqTYGCEZsEFOWJXY140I=; b=XzCh/GS+jrl772F3tUouR1nknMMnfwhQQCPg8R9DOZqj6RM0V52lH5gT2Piq/EN0kv r91sshSa2oEHtd+cHi/nLgpZL/Bu3R7Toi4ecPjaQXjn/jV5Ecsb1vVkaJyxHt/xmLCU FrIucRDBO9nWKj+zzJSpldNveg3Ra45wCclYGiO+O6cxxbePDeT1Di5d8XOgh19F8Jqb eIulMP3dxo5z0zC49g0eYXLmvp5dTKE6RaEdksWIuAq+uOvhesWU8UygoB5bIxS2MWCg iSvZAmvcXroNoPpYADaLnRmMXpPGbCXqDMN+BeMfj4BdoJgDGAvPuUx7iKqPygAIFPW3 DGTQ== X-Gm-Message-State: AOAM531ZSGmOCfP8XyhD4eBHEY67A67dF6FXd8A+aV6fXW7wiIlbOEwx HyyTgRpI19mNdTp0Hz6ZQs62PxXgg5cHg8qSWTXpmVWDsEI= X-Google-Smtp-Source: ABdhPJzwhyIEWz9+w1KBJQ9Hzp0Nk+TY59aPjiom/pdoICsjWiaS4/ZRAZ12gr3ycrTkGG6yaABFWywu3rGL94tLt/s= X-Received: by 2002:a19:5e14:: with SMTP id s20mr2624622lfb.110.1615993079840; Wed, 17 Mar 2021 07:57:59 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:b54e:0:0:0:0:0 with HTTP; Wed, 17 Mar 2021 07:57:59 -0700 (PDT) In-Reply-To: <20210317143307.20beb5fca0814246f2a91e9a@yamagi.org> References: <20210317143307.20beb5fca0814246f2a91e9a@yamagi.org> From: Mateusz Guzik Date: Wed, 17 Mar 2021 15:57:59 +0100 Message-ID: Subject: Re: 13.0-RC2 / 14-CURRENT: Processes getting stuck in vlruwk state To: Yamagi Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4F0tZ15fRHz4Tkt X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=pA9EKQn4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2a00:1450:4864:20::131 as permitted sender) smtp.mailfrom=mjguzik@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::131:from]; TO_DN_SOME(0.00)[]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::131:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::131:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Mailman-Approved-At: Wed, 17 Mar 2021 16:40:21 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 14:58:02 -0000 Can you reproduce the problem and run obtain "sysctl -a"? In general, there is a vnode limit which is probably too small. The reclamation mechanism is deficient in that it will eventually inject an arbitrary pause. On 3/17/21, Yamagi wrote: > Hi, > me and some other users in the ##bsdforen.de IRC channel have the > problem that during Poudriere runs processes getting stuck in the > 'vlruwk' state. > > For me it's fairly reproduceable. The problems begin about 20 to 25 > minutes after I've started poudriere. At first only some ccache > processes hang in the 'vlruwk' state, after another 2 to 3 minutes > nearly everything hangs and the total CPU load drops to about 5%. > When I stop poudriere with ctrl-c it takes another 3 to 5 minutes > until the system recovers. > > First the setup: > * poudriere runs in a bhyve vm on zvol. The host is a 12.2-RELEASE-p2. > The zvol has a 8k blocksize, the guests partition are aligned to 8k. > The guest has only zpool, the pool was created with ashift=13. The > vm has 16 E5-2620 and 16 gigabytes RAM assigned to it. > * poudriere is configured with ccache and ALLOW_MAKE_JOBS=yes. Removing > either of these options lowers the probability of the problem to show > up significantly. > > I've tried several git revisions starting with 14-CURRENT at > 54ac6f721efccdba5a09aa9f38be0a1c4ef6cf14 in the hope that I can find at > least one known to be good revision. No chance, even a kernel build > from 0932ee9fa0d82b2998993b649f9fa4cc95ba77d6 (Wed Sep 2 19:18:27 2020 > +0000) has the problem. The problem isn't reproduceable with > 12.2-RELEASE. > > The kernel stack ('procstat -kk') of a hanging process is: > mi_switch+0x155 sleepq_switch+0x109 sleepq_catch_signals+0x3f1 > sleepq_wait_sig+0x9 _sleep+0x2aa kern_wait6+0x482 sys_wait4+0x7d > amd64_syscall+0x140 fast_syscall_common+0xf8 > > The kernel stack of vnlru is changing, even while the processes are > hanging: > * mi_switch+0x155 sleepq_switch+0x109 sleepq_timedwait+0x4b > _sleep+0x29b vnlru_proc+0xa05 fork_exit+0x80 fork_trampoline+0xe > * fork_exit+0x80 fork_trampoline+0xe > > Since vnlru is accumulating CPU time it looks like it's doing at least > something. As an educated guess I would say that vn_alloc_hard() is > waiting a long time or even forever to allocate new vnodes. > > I can provide more information, I just need to know what. > > > Regards, > Yamagi > > -- > Homepage: https://www.yamagi.org > Github: https://github.com/yamagi > GPG: 0x1D502515 > -- Mateusz Guzik From owner-freebsd-stable@freebsd.org Wed Mar 17 19:54:18 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CC90D577141; Wed, 17 Mar 2021 19:54:18 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (mail1.yamagi.org [IPv6:2001:19f0:b001:853::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F117s6myqz3G2x; Wed, 17 Mar 2021 19:54:17 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from [2001:470:6845:1:fc9a:4782:611e:4abf] (helo=anemone.localdomain) by mail1.yamagi.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94 (FreeBSD)) (envelope-from ) id 1lMcFD-000HWk-FY; Wed, 17 Mar 2021 20:54:16 +0100 Date: Wed, 17 Mar 2021 20:53:52 +0100 From: Yamagi Burmeister To: mjguzik@gmail.com Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: 13.0-RC2 / 14-CURRENT: Processes getting stuck in vlruwk state Message-Id: <20210317205352.4ea8384e3ca1e6660dbbb06a@yamagi.org> In-Reply-To: References: <20210317143307.20beb5fca0814246f2a91e9a@yamagi.org> <20210317164821.a65559ba0df6645085466484@yamagi.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA256"; boundary="Signature=_Wed__17_Mar_2021_20_53_52_+0100_xJ+NbZ5mtNrRuJVe" X-Rspamd-Queue-Id: 4F117s6myqz3G2x X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lists@yamagi.org designates 2001:19f0:b001:853::3 as permitted sender) smtp.mailfrom=lists@yamagi.org X-Spamd-Result: default: False [-2.90 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; DMARC_NA(0.00)[yamagi.org]; SPAMHAUS_ZRD(0.00)[2001:19f0:b001:853::3:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:19f0:b001:853::3:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:20473, ipnet:2001:19f0:b000::/38, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 19:54:18 -0000 --Signature=_Wed__17_Mar_2021_20_53_52_+0100_xJ+NbZ5mtNrRuJVe Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable This time poudriere came to an end: % sysctl vfs.highest_numvnodes vfs.highest_numvnodes: 500976 On Wed, 17 Mar 2021 18:55:43 +0100 Mateusz Guzik wrote: > Thanks, I'm going to have to ponder a little bit. >=20 > In the meantime can you apply this: > https://people.freebsd.org/~mjg/maxvnodes.diff >=20 > Once you boot, tweak maxvnodes: > sysctl kern.maxvnodes=3D1049226 >=20 > Run poudriere. Once it finishes, inspect sysctl vfs.highest_numvnodes >=20 > On 3/17/21, Yamagi wrote: > > Hi Mateusz, > > the sysctl output after about 10 minutes into the problem is attached. > > In case that its stripped by Mailman a copy can be found here: > > https://deponie.yamagi.org/temp/sysctl_vlruwk.txt.xz > > > > Regards, > > Yamagi > > > > On Wed, 17 Mar 2021 15:57:59 +0100 > > Mateusz Guzik wrote: > > > >> Can you reproduce the problem and run obtain "sysctl -a"? > >> > >> In general, there is a vnode limit which is probably too small. The > >> reclamation mechanism is deficient in that it will eventually inject > >> an arbitrary pause. > >> > >> On 3/17/21, Yamagi wrote: > >> > Hi, > >> > me and some other users in the ##bsdforen.de IRC channel have the > >> > problem that during Poudriere runs processes getting stuck in the > >> > 'vlruwk' state. > >> > > >> > For me it's fairly reproduceable. The problems begin about 20 to 25 > >> > minutes after I've started poudriere. At first only some ccache > >> > processes hang in the 'vlruwk' state, after another 2 to 3 minutes > >> > nearly everything hangs and the total CPU load drops to about 5%. > >> > When I stop poudriere with ctrl-c it takes another 3 to 5 minutes > >> > until the system recovers. > >> > > >> > First the setup: > >> > * poudriere runs in a bhyve vm on zvol. The host is a 12.2-RELEASE-p= 2. > >> > The zvol has a 8k blocksize, the guests partition are aligned to 8= k. > >> > The guest has only zpool, the pool was created with ashift=3D13. T= he > >> > vm has 16 E5-2620 and 16 gigabytes RAM assigned to it. > >> > * poudriere is configured with ccache and ALLOW_MAKE_JOBS=3Dyes. Rem= oving > >> > either of these options lowers the probability of the problem to s= how > >> > up significantly. > >> > > >> > I've tried several git revisions starting with 14-CURRENT at > >> > 54ac6f721efccdba5a09aa9f38be0a1c4ef6cf14 in the hope that I can find= at > >> > least one known to be good revision. No chance, even a kernel build > >> > from 0932ee9fa0d82b2998993b649f9fa4cc95ba77d6 (Wed Sep 2 19:18:27 20= 20 > >> > +0000) has the problem. The problem isn't reproduceable with > >> > 12.2-RELEASE. > >> > > >> > The kernel stack ('procstat -kk') of a hanging process is: > >> > mi_switch+0x155 sleepq_switch+0x109 sleepq_catch_signals+0x3f1 > >> > sleepq_wait_sig+0x9 _sleep+0x2aa kern_wait6+0x482 sys_wait4+0x7d > >> > amd64_syscall+0x140 fast_syscall_common+0xf8 > >> > > >> > The kernel stack of vnlru is changing, even while the processes are > >> > hanging: > >> > * mi_switch+0x155 sleepq_switch+0x109 sleepq_timedwait+0x4b > >> > _sleep+0x29b vnlru_proc+0xa05 fork_exit+0x80 fork_trampoline+0xe > >> > * fork_exit+0x80 fork_trampoline+0xe > >> > > >> > Since vnlru is accumulating CPU time it looks like it's doing at lea= st > >> > something. As an educated guess I would say that vn_alloc_hard() is > >> > waiting a long time or even forever to allocate new vnodes. > >> > > >> > I can provide more information, I just need to know what. > >> > > >> > > >> > Regards, > >> > Yamagi > >> > > >> > -- > >> > Homepage: https://www.yamagi.org > >> > Github: https://github.com/yamagi > >> > GPG: 0x1D502515 > >> > > >> > >> > >> -- > >> Mateusz Guzik > > > > > > -- > > Homepage: https://www.yamagi.org > > Github: https://github.com/yamagi > > GPG: 0x1D502515 > > >=20 >=20 > --=20 > Mateusz Guzik --=20 Homepage: https://www.yamagi.org Github: https://github.com/yamagi GPG: 0x1D502515 --Signature=_Wed__17_Mar_2021_20_53_52_+0100_xJ+NbZ5mtNrRuJVe Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEOXu/lxyufwz0gC5x6xRy5x1QJRUFAmBSXlAACgkQ6xRy5x1Q JRWtcBAAm7O41n9F2mCThr8qOwG/RhcjzOXpRN5taHUYf1ghzGkHRClh9whxyPIl x/yVNqHAy7wLGVWrB3YiPc9C1LVd09Re/aQF0NuSATXm5XPo0PKMFO14NcJ2sR/k oLBPW8zO08WmhWWVSYUBkjHDVn+vcjs2Wj9Cg/rxCUrjHX0+MPvOms6lV64zdUYH YQeZaUHYyhNOvzoTRPvV4nxrHZS+crWhu+jdcH0ysg5xIVevsnWj6yDNrdyyvC5/ ZoJEz4Z/TP2IS0znVXnfJGyOXhRORN6Vyttupo4z9YsN4Dz9oWkOUeCYDVNkrz8U 5YXEqcZAyEZeoY3i7cEvDkelNQi0qax3zuFb9fhXJUCp7sKMJwMP8NQ2TzfZ6cLH 4wdf/t2hnsJSujMvr4+FFfZGX1CHlQy+A5EgQxreqlisq5PsZPLNoBgfielzqiG8 MGaWjIV/yiaZu+SCEScx/JHB0LvuayiuCZiDzNyojtQhTT2X+cYLNXQv4cuBBX85 AYY3gQnuLwjb7fXUcRwAae+sddfvI/RHvNEiUbSWONYFj7uBcvIetP+ziM6ef3FT 8spzkPjgJT56pTOVJJlMOIbwZWKBZrbNmof6sXlMzlv6mX5QdSqX11glx28QwPQq QHZ9ro5nMU68J5APrPmj2EoUkYV2TPFmgHKRIuU6rHqwXYdcHZo= =NdvJ -----END PGP SIGNATURE----- --Signature=_Wed__17_Mar_2021_20_53_52_+0100_xJ+NbZ5mtNrRuJVe-- From owner-freebsd-stable@freebsd.org Wed Mar 17 17:55:46 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5601B57366A; Wed, 17 Mar 2021 17:55:46 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [IPv6:2a00:1450:4864:20::135]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F0yW55BF7z4pjH; Wed, 17 Mar 2021 17:55:45 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-lf1-x135.google.com with SMTP id b83so240477lfd.11; Wed, 17 Mar 2021 10:55:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=oJHhs2pYPd1wT4t+P0dWfdjDLXJX7od6HXr5i5wj058=; b=ezNYL5XcUNCCu+U+Bq1h/LyzEgs3j5NHeyR88KFlEUrUWfS5T4SZ/ZMt9Z7E41Dld7 8X99Tm5y6SuhfCOfrBR2JfiLdbfHK5esuXoBXi3zryJpqvP3hJt6gUDERu2FsQ7ucy1m Nnf4ZcxYBGaRLbsQKaA6hTNXe4DPP/jwOi3iUealkR9hFuuofcZsy+KFMRJCoupt+I7C I9iMBVPsUm60PRVs7Bnk5YiykR/6WoZI5RqH6Q3AeZiEb1UDmnhVm8ZfO4X38j70SevB wxoyBPTqRRLm58wDAncoqKGE/GHMpSHOAnqVDSZn74WJMgziLv4pw6PyYAdu5eIuzlRU EVHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=oJHhs2pYPd1wT4t+P0dWfdjDLXJX7od6HXr5i5wj058=; b=rgQgfTXe+PeRR9aTc/Vu3cYROGnE46GiBxO0Au3LDAtnM8zWtO1Nm4k93BB4qRZPRc Fiw3uZwVjq9PYJgjfdmXx4tGuyuNlghJQwnkRhcbEUjxe80U3M/7/41UzrEIdFRhM14X gRwvYoP5ULAy93cpTmkDvLkAG9K7nRFxU1+o8hQ1rVJ/5OFqt/yIJnoTKFF9296jE4RA /Jnlw9tO3+9RidIvd2uljZpu9Bac2WbbEQH5TLKhomaAS4Y8VIjjhVptBLvWBdOLgUK2 jiai31Ho6igvpnacMw+1sL/UiH5w5rq52lr0hmyo8clN3lt/mU3wi6QVNJ8kvc9LYyRL U+kA== X-Gm-Message-State: AOAM530iInvislLDYcrUIFWVSKI5PobmVkIf1LI4L9VHZRxc4nedIieh HQ+HvvPTMHCB5HNhPVrEa204sOPdiucBas1/zi6a71tpOsE= X-Google-Smtp-Source: ABdhPJw2Bo68AWjDH8qvaWgflvTgz5V87ey0wWm6suU5gO/1UWzKQBPMd1HST0RITG+QS7rCeYRh/Ts3heR16JRxG+U= X-Received: by 2002:a19:e0d:: with SMTP id 13mr2932646lfo.549.1616003744179; Wed, 17 Mar 2021 10:55:44 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:b54e:0:0:0:0:0 with HTTP; Wed, 17 Mar 2021 10:55:43 -0700 (PDT) In-Reply-To: <20210317164821.a65559ba0df6645085466484@yamagi.org> References: <20210317143307.20beb5fca0814246f2a91e9a@yamagi.org> <20210317164821.a65559ba0df6645085466484@yamagi.org> From: Mateusz Guzik Date: Wed, 17 Mar 2021 18:55:43 +0100 Message-ID: Subject: Re: 13.0-RC2 / 14-CURRENT: Processes getting stuck in vlruwk state To: Yamagi Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4F0yW55BF7z4pjH X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=ezNYL5Xc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2a00:1450:4864:20::135 as permitted sender) smtp.mailfrom=mjguzik@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::135:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_DN_SOME(0.00)[]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::135:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::135:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Mailman-Approved-At: Thu, 18 Mar 2021 09:10:46 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2021 17:55:46 -0000 Thanks, I'm going to have to ponder a little bit. In the meantime can you apply this: https://people.freebsd.org/~mjg/maxvnodes.diff Once you boot, tweak maxvnodes: sysctl kern.maxvnodes=1049226 Run poudriere. Once it finishes, inspect sysctl vfs.highest_numvnodes On 3/17/21, Yamagi wrote: > Hi Mateusz, > the sysctl output after about 10 minutes into the problem is attached. > In case that its stripped by Mailman a copy can be found here: > https://deponie.yamagi.org/temp/sysctl_vlruwk.txt.xz > > Regards, > Yamagi > > On Wed, 17 Mar 2021 15:57:59 +0100 > Mateusz Guzik wrote: > >> Can you reproduce the problem and run obtain "sysctl -a"? >> >> In general, there is a vnode limit which is probably too small. The >> reclamation mechanism is deficient in that it will eventually inject >> an arbitrary pause. >> >> On 3/17/21, Yamagi wrote: >> > Hi, >> > me and some other users in the ##bsdforen.de IRC channel have the >> > problem that during Poudriere runs processes getting stuck in the >> > 'vlruwk' state. >> > >> > For me it's fairly reproduceable. The problems begin about 20 to 25 >> > minutes after I've started poudriere. At first only some ccache >> > processes hang in the 'vlruwk' state, after another 2 to 3 minutes >> > nearly everything hangs and the total CPU load drops to about 5%. >> > When I stop poudriere with ctrl-c it takes another 3 to 5 minutes >> > until the system recovers. >> > >> > First the setup: >> > * poudriere runs in a bhyve vm on zvol. The host is a 12.2-RELEASE-p2. >> > The zvol has a 8k blocksize, the guests partition are aligned to 8k. >> > The guest has only zpool, the pool was created with ashift=13. The >> > vm has 16 E5-2620 and 16 gigabytes RAM assigned to it. >> > * poudriere is configured with ccache and ALLOW_MAKE_JOBS=yes. Removing >> > either of these options lowers the probability of the problem to show >> > up significantly. >> > >> > I've tried several git revisions starting with 14-CURRENT at >> > 54ac6f721efccdba5a09aa9f38be0a1c4ef6cf14 in the hope that I can find at >> > least one known to be good revision. No chance, even a kernel build >> > from 0932ee9fa0d82b2998993b649f9fa4cc95ba77d6 (Wed Sep 2 19:18:27 2020 >> > +0000) has the problem. The problem isn't reproduceable with >> > 12.2-RELEASE. >> > >> > The kernel stack ('procstat -kk') of a hanging process is: >> > mi_switch+0x155 sleepq_switch+0x109 sleepq_catch_signals+0x3f1 >> > sleepq_wait_sig+0x9 _sleep+0x2aa kern_wait6+0x482 sys_wait4+0x7d >> > amd64_syscall+0x140 fast_syscall_common+0xf8 >> > >> > The kernel stack of vnlru is changing, even while the processes are >> > hanging: >> > * mi_switch+0x155 sleepq_switch+0x109 sleepq_timedwait+0x4b >> > _sleep+0x29b vnlru_proc+0xa05 fork_exit+0x80 fork_trampoline+0xe >> > * fork_exit+0x80 fork_trampoline+0xe >> > >> > Since vnlru is accumulating CPU time it looks like it's doing at least >> > something. As an educated guess I would say that vn_alloc_hard() is >> > waiting a long time or even forever to allocate new vnodes. >> > >> > I can provide more information, I just need to know what. >> > >> > >> > Regards, >> > Yamagi >> > >> > -- >> > Homepage: https://www.yamagi.org >> > Github: https://github.com/yamagi >> > GPG: 0x1D502515 >> > >> >> >> -- >> Mateusz Guzik > > > -- > Homepage: https://www.yamagi.org > Github: https://github.com/yamagi > GPG: 0x1D502515 > -- Mateusz Guzik From owner-freebsd-stable@freebsd.org Thu Mar 18 15:11:40 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 11D1F578A6E for ; Thu, 18 Mar 2021 15:11:40 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received: from webmail.dweimer.net (024-240-198-186.biz.spectrum.com [24.240.198.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "dweimer.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F1VqG6DLJz3psP for ; Thu, 18 Mar 2021 15:11:38 +0000 (UTC) (envelope-from dweimer@dweimer.net) Received-SPF: pass (webmail.dweimer.net: authenticated connection) receiver=webmail.dweimer.net; client-ip=10.9.5.1; helo=www.dweimer.net; envelope-from=dweimer@dweimer.net; x-software=spfmilter 2.001 http://www.acme.com/software/spfmilter/ with libspf2-1.2.10; Received: from www.dweimer.net (pfsense.dweimer.me [10.9.5.1]) (authenticated bits=0) by webmail.dweimer.net (8.16.1/8.16.1) with ESMTPSA id 12IFBa0L090904 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Thu, 18 Mar 2021 10:11:37 -0500 (CDT) (envelope-from dweimer@dweimer.net) MIME-Version: 1.0 Date: Thu, 18 Mar 2021 10:11:31 -0500 From: "Dean E. Weimer" To: FreeBSD Stable Subject: 13-RC2 Serial Console not accepting Input Reply-To: dweimer@dweimer.net User-Agent: Roundcube Webmail/1.4.11 Message-ID: <0971335bba5932c5cfac6bd6f7eb1d2f@dweimer.net> X-Sender: dweimer@dweimer.net Organization: dweimer.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4F1VqG6DLJz3psP X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; HAS_REPLYTO(0.00)[dweimer@dweimer.net]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:24.240.198.184/29]; REPLYTO_ADDR_EQ_FROM(0.00)[]; HAS_ORG_HEADER(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[dweimer.net:+]; DMARC_POLICY_ALLOW(-0.50)[dweimer.net,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[24.240.198.186:from]; ASN(0.00)[asn:20115, ipnet:24.240.196.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[dweimer.net:s=2017.01.31]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[24.240.198.186:from:127.0.2.255]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2021 15:11:40 -0000 I upgraded a system from 12.2 to 13-RC2 running on an PC Engines Alix APU1D4 system board. It boots and works fine, however despite console output displaying fine, it will not accept any keyboard input. I have the following in my /boot/loader.conf # Enable Serial Console console="comconsole" comconsole_speed="115200" boot_serial="YES" Am I missing something else that has changed requiring a new option to be set for serial consoles? -- Thanks, Dean E. Weimer http://www.dweimer.net/ From owner-freebsd-stable@freebsd.org Thu Mar 18 15:44:05 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 12FCC57ACE7; Thu, 18 Mar 2021 15:44:05 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [IPv6:2a00:1450:4864:20::131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F1WXh1mdSz3s3Z; Thu, 18 Mar 2021 15:44:03 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-lf1-x131.google.com with SMTP id 75so5291023lfa.2; Thu, 18 Mar 2021 08:44:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Mc/Y6msjubceQPcuTWHm3gAGQ5Nrc2qG26qviHpdOGc=; b=XqRck4m90+5XclJL48mpjxbRBuGbElhjAHop+V/+v2YIERpgtLTBSwNaCnt5dgKvdu JTXOoTCTWSsTrcYOisBI7kSeqrXw77hS1XOrfdF3DoppvJntrlyWjAoPvP9nQ+jWoaFa ye0nMgv6O/Ok3ht8La2dThi42UaIwi1eDURQGWZz9HzjP7uN/kTPr6jgQz+SWtnqcfX0 41e0l5lSs/YGD0korlXqXT/6vshiw68E95wfhxEtXHR/VNd0cQ357fCaNGPDYyAj/HwW e9njZLk541X2Ij5OuL3amTpf5UsUv5QbLp4D/6wWa1E+hpKf5Tdpubd4yPHKTtAlmMyo SHyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Mc/Y6msjubceQPcuTWHm3gAGQ5Nrc2qG26qviHpdOGc=; b=PCJnfnukLMztgBf1PXey2x73g8gfzrVIG3WkfI0VEEPbP4mowobEFtPwkQUccMdru9 ezunB8WhR4W6enlMCnQcqVfREY2tTi7uNNyE2UMrR1UxbZ4d4FgrvuAdm66PtjdoMgj9 dWA+BYFN/26jOXWxmU+z8mhUufmiW5+pq8JlNvAsNbgMJoPSMC9Oz2SU7FlawZWDYWKO e09npp1MxSuZoNjHaMA7EQKzhR8XvdCsG9XJOp3HeVvFnBMn/UxtCIkSd4FFJO9UYoE+ Zw8jUmXcAJs+j+FS+uspgxgjvrKEXoUGI3W2Ex8ncqFGti5Na1WDI1VdiKIExZao/cZK pu0Q== X-Gm-Message-State: AOAM532zIsTOI/Z8TZxVF8tmJoQdbLAjakKEcfX8ewsVYGp1sE3Lxw9I 7Dyjzvctf6cgtp8k6tGqhdrkkMQA/r1L6Dm2yonYucXKoDY= X-Google-Smtp-Source: ABdhPJxQDGdyRRcBnzYyyFL/8Wq2EKBua5Q1LZhLhhs80ZgRHYUNOPMi5e3Mc/bNugtVe/1Sh79kPXELYOvgwAaFCAM= X-Received: by 2002:ac2:43d6:: with SMTP id u22mr6188930lfl.266.1616082241915; Thu, 18 Mar 2021 08:44:01 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:b54e:0:0:0:0:0 with HTTP; Thu, 18 Mar 2021 08:44:00 -0700 (PDT) In-Reply-To: <20210317143307.20beb5fca0814246f2a91e9a@yamagi.org> References: <20210317143307.20beb5fca0814246f2a91e9a@yamagi.org> From: Mateusz Guzik Date: Thu, 18 Mar 2021 16:44:00 +0100 Message-ID: Subject: Re: 13.0-RC2 / 14-CURRENT: Processes getting stuck in vlruwk state To: Yamagi Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4F1WXh1mdSz3s3Z X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=XqRck4m9; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2a00:1450:4864:20::131 as permitted sender) smtp.mailfrom=mjguzik@gmail.com X-Spamd-Result: default: False [-2.09 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.91)[0.910]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::131:from]; TO_DN_SOME(0.00)[]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::131:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::131:from]; NEURAL_HAM_LONG(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current,freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Mailman-Approved-At: Thu, 18 Mar 2021 19:07:06 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2021 15:44:05 -0000 To sum up what happened, Yamagi was kind enough to test several patches and ultimately the issue got solved here https://cgit.freebsd.org/src/commit/?id=e9272225e6bed840b00eef1c817b188c172338ee . The patch also got merged into releng/13.0 On 3/17/21, Yamagi wrote: > Hi, > me and some other users in the ##bsdforen.de IRC channel have the > problem that during Poudriere runs processes getting stuck in the > 'vlruwk' state. > > For me it's fairly reproduceable. The problems begin about 20 to 25 > minutes after I've started poudriere. At first only some ccache > processes hang in the 'vlruwk' state, after another 2 to 3 minutes > nearly everything hangs and the total CPU load drops to about 5%. > When I stop poudriere with ctrl-c it takes another 3 to 5 minutes > until the system recovers. > > First the setup: > * poudriere runs in a bhyve vm on zvol. The host is a 12.2-RELEASE-p2. > The zvol has a 8k blocksize, the guests partition are aligned to 8k. > The guest has only zpool, the pool was created with ashift=13. The > vm has 16 E5-2620 and 16 gigabytes RAM assigned to it. > * poudriere is configured with ccache and ALLOW_MAKE_JOBS=yes. Removing > either of these options lowers the probability of the problem to show > up significantly. > > I've tried several git revisions starting with 14-CURRENT at > 54ac6f721efccdba5a09aa9f38be0a1c4ef6cf14 in the hope that I can find at > least one known to be good revision. No chance, even a kernel build > from 0932ee9fa0d82b2998993b649f9fa4cc95ba77d6 (Wed Sep 2 19:18:27 2020 > +0000) has the problem. The problem isn't reproduceable with > 12.2-RELEASE. > > The kernel stack ('procstat -kk') of a hanging process is: > mi_switch+0x155 sleepq_switch+0x109 sleepq_catch_signals+0x3f1 > sleepq_wait_sig+0x9 _sleep+0x2aa kern_wait6+0x482 sys_wait4+0x7d > amd64_syscall+0x140 fast_syscall_common+0xf8 > > The kernel stack of vnlru is changing, even while the processes are > hanging: > * mi_switch+0x155 sleepq_switch+0x109 sleepq_timedwait+0x4b > _sleep+0x29b vnlru_proc+0xa05 fork_exit+0x80 fork_trampoline+0xe > * fork_exit+0x80 fork_trampoline+0xe > > Since vnlru is accumulating CPU time it looks like it's doing at least > something. As an educated guess I would say that vn_alloc_hard() is > waiting a long time or even forever to allocate new vnodes. > > I can provide more information, I just need to know what. > > > Regards, > Yamagi > > -- > Homepage: https://www.yamagi.org > Github: https://github.com/yamagi > GPG: 0x1D502515 > -- Mateusz Guzik From owner-freebsd-stable@freebsd.org Fri Mar 19 18:26:57 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8C94657DC3B for ; Fri, 19 Mar 2021 18:26:57 +0000 (UTC) (envelope-from fred.ha11@yahoo.com) Received: from sonic315-20.consmr.mail.ne1.yahoo.com (sonic315-20.consmr.mail.ne1.yahoo.com [66.163.190.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2C683c9tz3QCK for ; Fri, 19 Mar 2021 18:26:56 +0000 (UTC) (envelope-from fred.ha11@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1616178414; bh=YenEpuZzAZlaKtVaoaxTMzDCKrA6rRzxlxPL4uZKNYb=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=BzRp7SLZxDjRDacZqBh6Xw09MbSvDdaL5amMEGmByiYZoD8tWel9R1nmig0iO+BPNOcT9/R9D4qQTQ7AJ1hgcO2gT2ejRKjV1sdZM8Q0XqFDy64y9i6kg0sNBaim/j1mCsqEgchOJeQUdJ9m3cNsnSqbsM2gf9WwQhbxNIVxaeaHXmfyMnz8PTu5BTuZg6UqL55v9Is+Z+gj49pxEPWqmyO/IfbxhsbpaBS/wQiAxz78z3KHAZLR96qhaN87812o0+k9KRzMELe5wx1qikCFTrNRovZWvdDiF3NxDpx2GBOCNf7NmstjdjZ0pURKI9Y0wIZUAXi69jzkBYTn2WQLkQ== X-YMail-OSG: ujghexoVM1k9MbNZS2PmSQEDpK.H8WK.32.hrIkcTSinxubJFsPETqlmQkryQir fnHJrGVxd8b5Y2rngEqF9mE2F09Vr0uiX6ra0kBsG4h7AQ0QM.Zau89GvXYGfANeCb.NmUsOUGld Ev.yIrY30vJtcVgRHpz0vZABLwsA_WfPLfeBlgJ.gvl8GogLKfxctvI3Luk9HJPzQexcj7CLMkW9 mOhbLU4oI.cMHX8yQbMgqklNJKmkUhxiMe0LcWvUJe7OcqCMKEhfccJv.9DQMcHMiwDIL88EeZvL cuV8vcXXWNNvQO6.ZMlNXAIP5Eu2mrXuvpjEGkS6893_JSOgIX2bWLpXorS9qTiFdZhg_JOi6fLO hMujLGte3AXD7BZBgn5obpPQ6jSFMpK_DHvVHNrOwZd2qI.SWbA7jFTTT_weKcSWpTbFjudxuVCM SmNtR.AtpKyqLsrcbca3kDYhth5ZI69HGXiVWumS0PoiFny7LPeC4iT6Y0v7x7EfRWLZH64XlVI1 iz7VfaFeAAmZVCVE8T4nBsOCZNpUX1BsrTfwtcVEXSPRyG7rn9VeZw6TZVNjEt7XrnVZIUAb9ycd T2Z2G8dVL848sBuhgYm5wvjfg7LE3.IoeWhVA2S58YxTPG_wzW2aGKrrkqRveXblo6Y94wgSsQEt xUlu6N0NsG406Agiits915kDXh3UMDC4VxD2BniUkrmKNrYPZe0i4wS2odvOx6woTTUYycKm1KHB 1DqCjFUBiyVdRjlB_8tS_Vqr9EvNMm2PMprMFudoVakW8_Yek8vwsJiKaRgfmE40vH.7ryIz3IEu f9_fxkYlAjvEwep4jLUYUpZKhMyGukqcS_2NRVHSxi5C.SUj9CgqWsDrhkFDLPT9LAKb8XFjgp_Y GJLuHnR_jxmyrrupugf6tre7gDCBEcP.id5aR9pzeXxuXQCR_21A7uGqEMr9KLgciym6026gS3Sx XZijF8Y8nX6WsBkSinMq0bMujgwTW0vwnFX1bi0inCyO3WBJdZtuysmex3nBMrf__1H7CIdpp0IZ b._ZRWKFxky9BLCtsKfri3HP07YkFW_LFLBap3xWDq.CvygRlsmyTXA9PxH8tRcsi2FcqMRGo103 bTSoaHOvg0PxO.SUfa2E4JJIAk_3Z1sQF4Fi_H6kinpnMhzSMngzhtHyHYzhxGUjQH23gMURYoJ6 m0jyg9xgPZrttBk07gk44Em7QZHy9vzDDs5yW51i8GALAdLpNF1e0wMOEI_N9OhHVCE92X_cF4Hv OH1fZrgSJmp.ip8bYTedQSccYWLzgI5eS_jtolNsZ9olZnK5tf.2SJtY2TK_ODSrZn.DrEDG_5mX hjRbQXRm6E0g2H8LxRUZd.d81KSiNTq2E41CBsUQbuxz893cLgdLQQM8cS3VXHBsbgxjHe0T87dF v2BvqVU3FM.t05dozt4PaPciGC7GktTQ_VclqQKTsP6UjEcElOoIekn9ojJqsYinD1dfiyInwHRZ T0hwdmTPWlr0m982rrzuUSRqkyDars_l1D0Cx2WKz3BYgxH1_wAQtmP0Y0kGpercg30q4rJSSx2M MU366moG1lm7eo1pjtan5tRGlfc34R6qok6.jUEE4J6iT6yQNsnqwFxPYw0sIzDfY83e2GSSVlgu pdgRBlHLU2ZoFWRKqcVa9T_OYMI5VMrwDnN4QS_Lf2LNaWgTwzdbZfSQIgKOUX5sOuPBAsrQphUv uIOerEv_66LWOzTHMp.8udj7R_RwuWRE8FDhu18hlEjnYWqfl_jEFL3MwKeiFoSwW.ymamdP7i4U vMoa8rq5BkxP3g_LZ7jFqAuPWxub9eyVOSp3mKTdT8Sp1BeyWK6ngtyyDQgMaDGN6oJTQegeStsn i9YO5QX6tYWVAGZlNU3tGO9nK_wOaaDNl.DRiCXbiqhgZKYXdnPCO6zCqGWxyEKsxlQGO8uADJwN OziqBxr1H7O4q5f9fNLemFmgdWHLiIhjwjXllHk7BBSL1MjzyrtFIk28o.40cru3GaLhhRKzzoU6 X5DgQChPan3GDRB_VgbGkIvxrv8ktnejB73UlO.r9oPS0CLR5Eo18G2Ggg0rplg9SCcIkLv8upZb 51aj6liuk4l.vgLJR.EDSK338hg.qor0o4US__qabkt74kiiQlphQQoV11w9iBQz5vqTTKSCrCaY iurawjMjvIUfSPkUuZPymVkYBoaTyq0CBxpC7jDYbmY4VlJPBC8yiqMa_wlh5JoUuDFksY3Ic.JG cZuCXpx1k7PdEFzsPtergYDZEQuV6PkKcHLszfFxSHS979fT.kZlA5YvNLnxqb2xT6RvHOMScFWu KGLR8S0x9aBT3v.NqDvRs87y9WAEx0fAYcYze1C_nWcEu6N9Kw5.fsS0jQgOWe.IA1vUgbQiZ3tX laQAa2XgWvY6q7IrgpNnlo7BZM8kRxMlunopiYfCRunt9conMpMhsgRofuR.WRUDu5.jbJvt_mbq uvRbJ4dBoq7sKMv4H0ye80z8Ag6JbC89O0_2DAZZniZ6XUJk0G05jPOyyosnDR.8.bvuLgJtU1Gs 6OiyL X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.ne1.yahoo.com with HTTP; Fri, 19 Mar 2021 18:26:54 +0000 Date: Fri, 19 Mar 2021 18:26:50 +0000 (UTC) From: Fred Hall To: "freebsd-stable@freebsd.org" Message-ID: <1867238560.3032633.1616178410076@mail.yahoo.com> Subject: ThinkPad X1 Carbon Gen 4 unable to boot 13.0-RC1 MIME-Version: 1.0 References: <1867238560.3032633.1616178410076.ref@mail.yahoo.com> X-Mailer: WebService/1.1.17936 YMailNorrin Mozilla/5.0 (X11; FreeBSD amd64; rv:86.0) Gecko/20100101 Firefox/86.0 X-Rspamd-Queue-Id: 4F2C683c9tz3QCK X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[66.163.190.146:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[66.163.190.146:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[66.163.190.146:from]; TO_DN_EQ_ADDR_ALL(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[66.163.190.146:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2021 18:26:57 -0000 I have a ThinkPad X1 Carbon Gen 4 (20FCS0NB00) which has happily run FreeBS= D 11 and 12, but which can't boot 13.0-RC1 from memstick or via freebsd-upd= ate. In both cases the boot process locks up on the line "hwpstate_intel0: = on cpu0" If running=C2=A0freebsd-update, a work around is to add hint.hwpstate_intel= .0.disabled=3D"1" in loader.conf. See the note under the Eighth Generation = (2020) in https://wiki.freebsd.org/Laptops/Thinkpad_X1_Carbon I was quite surprised to find the lack of support for hwpstate_intel in 13 = when it apparently worked under 11 and 12. Does anyone know the status of h= wpstate_intel on ThinkPads? Cheers,=20 Fred From owner-freebsd-stable@freebsd.org Fri Mar 19 20:08:13 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 59CD55A990B for ; Fri, 19 Mar 2021 20:08:13 +0000 (UTC) (envelope-from "") Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F2FLz1hMFz3pK4 for ; Fri, 19 Mar 2021 20:08:11 +0000 (UTC) (envelope-from "") Received: by mailman.nyi.freebsd.org (Postfix) id 382D55A990A; Fri, 19 Mar 2021 20:08:11 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 37F205A9633 for ; Fri, 19 Mar 2021 20:08:11 +0000 (UTC) (envelope-from "") Received: from securemail-y50.synaq.com (securemail-y50.synaq.com [196.35.198.33]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "*.synaq.com", Issuer "GeoTrust TLS DV RSA Mixed SHA256 2020 CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2FLw64zWz3pMJ for ; Fri, 19 Mar 2021 20:08:08 +0000 (UTC) (envelope-from "") Received: from [172.245.162.177] by securemail-pl-omx8.synaq.com with esmtpsa (TLS1) tls TLS_DHE_RSA_WITH_AES_256_CBC_SHA (Exim 4.93.0.4-30-5edf9c7) id 1lNLKi-0003wA-Ko for stable@FreeBSD.ORG; Fri, 19 Mar 2021 22:02:56 +0200 MIME-Version: 1.0 Subject: Important Message stable@FreeBSD.ORG To: stable@FreeBSD.ORG From: "ANZ Internet Banking" <> Date: Fri, 19 Mar 2021 13:02:47 -0700 X-UNKNOWN-NULL: Yes X-Red-Router: yes X-SYNAQ-Pinpoint-Information: Please contact SYNAQ for more information X-SYNAQ-Pinpoint-ID: 1lNLKi-0003wA-Ko X-SYNAQ-Pinpoint: No virus infections found X-SYNAQ-Pinpoint-SpamCheck: not spam, SpamAssassin (not cached, score=6.109, required 9, autolearn=disabled, ALL_TRUSTED -1.00, FROM_NO_USER 2.60, HTML_IMAGE_RATIO_06 0.00, HTML_MESSAGE 0.00, MISSING_MID 0.14, NULLSENDER 0.01, TVD_PH_BODY_ACCOUNTS_PRE 0.00, URI_GOOGLE_PROXY 1.63, WIKI_IMG 2.72) X-SYNAQ-Pinpoint-SpamScore: ssssss X-Pinpoint-From: X-Rspamd-Queue-Id: 4F2FLw64zWz3pMJ X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of securemail-y50.synaq.com has no SPF policy when checking 196.35.198.33) smtp.helo=securemail-y50.synaq.com X-Spamd-Result: default: False [5.55 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GREYLIST(0.00)[pass,body]; ZERO_FONT(0.10)[1]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[1]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[196.35.198.33:from]; ASN(0.00)[asn:3741, ipnet:196.34.0.0/15, country:ZA]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_NEUTRAL(0.00)[196.35.198.33:from]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[No domain in From header]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[196.35.198.33:from:127.0.2.255]; MANY_INVISIBLE_PARTS(0.05)[1]; MISSING_MID(2.50)[]; NEURAL_SPAM_LONG(1.00)[1.000]; RCVD_IN_DNSWL_NONE(0.00)[196.35.198.33:from]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[stable] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2021 20:08:13 -0000 = = = = = = = = Dear ANZ customer = = Your ANZ Internet Banking access has been violated. We suspected someone o= ther than you with IP 178.210.92*** trying to access your information's. = = Please verify your banking information with us to show that you are not= currently away. You have to verify this as soon as possible to prevent you= r online bank account from getting suspended. = Restore your Access = =A9 Australia and New Zealand Banking Group Limited (ANZ) ABN 11 005 35= 7 522. ACLN and AFSL Number 234527. = = = = = = = =20 From owner-freebsd-stable@freebsd.org Sat Mar 20 01:59:23 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C91045B5B51 for ; Sat, 20 Mar 2021 01:59:23 +0000 (UTC) (envelope-from Mathias.Picker@virtual-earth.de) Received: from www94.your-server.de (www94.your-server.de [213.133.104.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2P8B5knWz4lDq for ; Sat, 20 Mar 2021 01:59:22 +0000 (UTC) (envelope-from Mathias.Picker@virtual-earth.de) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtual-earth.de; s=default_1811; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Message-ID:Date:In-reply-to:Subject:Cc:To:From:References:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=TPnsY++SCmyvlqXSf+b/DeS/HaNI0EsoMgfSKPLAF/4=; b=INFGNL5yDKYKbbP306MajN41ay Gs9l2+W7Hc5OveBQTFgs0LJ722kVJnndeYPS8O+JMOwOFKLWBN3Q+A3VcgEZF5wPKsWZTi6+hjHJp I7XgvDkWk5Lx7qFl4E+mFt+wNXt9W4WjEYMmibnhwuJthijmOmnhAbtmW68j9OIyinkHh4nrDpqi5 tXwuLyIM57A8LTFCzMF1MWsNmHVHYn4ekThl7sTy7MfMOx8n3HERaMznGZKspLwmkcNntbqjoL2nW m3oSBhSTqU/sa7Ud9dNEEaVb8VW22P5vbdZPaEZZbZOxZHu/GdWmDQ4ESg/JMGnqerAhyl9lHtZ5y hBJ5Y0SA==; Received: from sslproxy05.your-server.de ([78.46.172.2]) by www94.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92.3) (envelope-from ) id 1lNQtc-000GYv-Nz; Sat, 20 Mar 2021 02:59:20 +0100 Received: from [2a01:c22:764c:d500:1a1d:eaff:fe16:cc77] (helo=Danton.virtual-earth.de) by sslproxy05.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lNQtc-000Qbq-K2; Sat, 20 Mar 2021 02:59:20 +0100 References: <1867238560.3032633.1616178410076.ref@mail.yahoo.com> <1867238560.3032633.1616178410076@mail.yahoo.com> User-agent: mu4e 1.2.0; emacs 27.1 From: Mathias Picker To: Fred Hall Cc: "freebsd-stable@freebsd.org" Subject: Re: ThinkPad X1 Carbon Gen 4 unable to boot 13.0-RC1 In-reply-to: <1867238560.3032633.1616178410076@mail.yahoo.com> Date: Sat, 20 Mar 2021 02:59:20 +0100 Message-ID: <86czvud1qf.fsf@virtual-earth.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Authenticated-Sender: Mathias.Picker@virtual-earth.de X-Virus-Scanned: Clear (ClamAV 0.102.4/26113/Fri Mar 19 12:14:45 2021) X-Rspamd-Queue-Id: 4F2P8B5knWz4lDq X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=virtual-earth.de header.s=default_1811 header.b=INFGNL5y; dmarc=pass (policy=none) header.from=virtual-earth.de; spf=pass (mx1.freebsd.org: domain of Mathias.Picker@virtual-earth.de designates 213.133.104.94 as permitted sender) smtp.mailfrom=Mathias.Picker@virtual-earth.de X-Spamd-Result: default: False [-3.89 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[virtual-earth.de:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[virtual-earth.de,none]; NEURAL_HAM_SHORT(-0.79)[-0.793]; FREEMAIL_TO(0.00)[yahoo.com]; RCVD_IN_DNSWL_LOW(-0.10)[213.133.104.94:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[213.133.104.94:from]; MID_RHS_MATCH_FROM(0.00)[]; HAS_X_AS(0.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[virtual-earth.de:s=default_1811]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[213.133.104.94:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 01:59:23 -0000 Fred Hall via freebsd-stable writes: > I have a ThinkPad X1 Carbon Gen 4 (20FCS0NB00) which has happily=20 > run FreeBSD 11 and 12, but which can't boot 13.0-RC1 from=20 > memstick or via freebsd-update. In both cases the boot process=20 > locks up on the line "hwpstate_intel0: on=20 > cpu0" > If running freebsd-update, a work around is to add=20 > hint.hwpstate_intel.0.disabled=3D"1" in loader.conf. See the note=20 > under the Eighth Generation (2020) in=20 > https://wiki.freebsd.org/Laptops/Thinkpad_X1_Carbon > I was quite surprised to find the lack of support for=20 > hwpstate_intel in 13 when it apparently worked under 11 and 12.=20 > Does anyone know the status of hwpstate_intel on ThinkPads? I=E2=80=99m running 13 -STABLE from 10 days ago on a Thinkpad Yoga 3rd=20 gen, and hwpstate_intel works fine, never had a problem. mathiasp:~% sysctl dev.hwpstate_intel=20 dev.hwpstate_intel.7.epp: 15 dev.hwpstate_intel.7.%parent: cpu7 dev.hwpstate_intel.7.%pnpinfo:=20 dev.hwpstate_intel.7.%location:=20 dev.hwpstate_intel.7.%driver: hwpstate_intel dev.hwpstate_intel.7.%desc: Intel Speed Shift dev.hwpstate_intel.6.epp: 15 dev.hwpstate_intel.6.%parent: cpu6 dev.hwpstate_intel.6.%pnpinfo:=20 dev.hwpstate_intel.6.%location:=20 dev.hwpstate_intel.6.%driver: hwpstate_intel dev.hwpstate_intel.6.%desc: Intel Speed Shift [snip] The gen3 is using sudo dmesg|grep -i cpu CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (1992.08-MHz=20 K8-class CPU) [snip, snip] mathiasp:~% uname -a FreeBSD Danton 13.0-STABLE FreeBSD 13.0-STABLE #2=20 stable/13-n244845-f21c0366f53: Wed Mar 10 20:53:26 CET 2021=20 root@Danton:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 Cheers, Mathias > Cheers,=20 > Fred > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to=20 > "freebsd-stable-unsubscribe@freebsd.org" --=20 Mathias Picker=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 Gesch=C3=A4ftsf=C3=BChrer Mathias.Picker@virtual-earth.de virtual earth Gesellschaft f=C3=BCr Wissens re/pr=C3=A4 sentation mbH http://www.virtual-earth.de/ HRB126870 support@virtual-earth.de Westendstr. 142 089 / 1250 3943=20=20=20=20=20=20=20=20=20=20=20=20 From owner-freebsd-stable@freebsd.org Sat Mar 20 03:00:45 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 51A655B7858 for ; Sat, 20 Mar 2021 03:00:45 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F2QW110lYz4pXh for ; Sat, 20 Mar 2021 03:00:45 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: by mailman.nyi.freebsd.org (Postfix) id 226345B79CC; Sat, 20 Mar 2021 03:00:45 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 222A35B7A18 for ; Sat, 20 Mar 2021 03:00:45 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from mail02.asahi-net.or.jp (mail02.asahi-net.or.jp [202.224.55.14]) by mx1.freebsd.org (Postfix) with ESMTP id 4F2QVz3dm8z4pHC; Sat, 20 Mar 2021 03:00:42 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from vmware12.advok.com (cpe-184-152-96-96.nj.res.rr.com [184.152.96.96]) (Authenticated sender: NR2Y-OOT) by mail02.asahi-net.or.jp (Postfix) with ESMTPSA id 039D3797E3; Sat, 20 Mar 2021 12:00:40 +0900 (JST) Date: Fri, 19 Mar 2021 23:01:12 -0400 From: Yoshihiro Ota To: Andriy Gapon Cc: stable@freebsd.org Subject: Re: kldload zfs spins the system after upgrading from 12.2 to 13-BETA Message-Id: <20210319230112.e327d1c69197b73244fd89d6@j.email.ne.jp> In-Reply-To: References: <20210306130913.dae1fb546e68bcaec882cdbe@j.email.ne.jp> <9f9db150-ee5d-e74e-7f24-74d1fa687a48@FreeBSD.org> <20210307222441.02755e7396cd329edd15df15@j.email.ne.jp> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; i386-portbld-freebsd12.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4F2QVz3dm8z4pHC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of ota@j.email.ne.jp designates 202.224.55.14 as permitted sender) smtp.mailfrom=ota@j.email.ne.jp X-Spamd-Result: default: False [-2.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:202.224.55.0/24]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; RCVD_NO_TLS_LAST(0.10)[]; RECEIVED_SPAMHAUS_PBL(0.00)[184.152.96.96:received]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[202.224.55.14:from]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:4685, ipnet:202.224.32.0/19, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCVD_IN_DNSWL_LOW(-0.10)[202.224.55.14:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[email.ne.jp]; SPAMHAUS_ZRD(0.00)[202.224.55.14:from:127.0.2.255]; RWL_MAILSPIKE_POSSIBLE(0.00)[202.224.55.14:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 03:00:45 -0000 On Tue, 9 Mar 2021 11:24:53 +0200 Andriy Gapon wrote: > On 08/03/2021 05:24, Yoshihiro Ota wrote: > > On Sun, 7 Mar 2021 00:09:33 +0200 > > Andriy Gapon wrote: > > > >> On 06/03/2021 20:09, Yoshihiro Ota wrote: > >>> Hi all, > >>> > >>> I'm upgrading fron 12.2-RELEASE to 13-BETA/RC one by one. > >>> > >>> After upgrading one in VMWare, 'zfs mount -a' hangs the system. > >>> I don't have boottime zfs mount on nor don't have zfsroot. > >>> I just simply ran install world/kernel and mergemaster. > >> > >> Please use procstat -kk to capture a kernel stack trace of the hung process. > > > > Actually, spining was 'kldload zfs'. > > Console doesn't response but ping and sshd sessions still work. > > procstat output is below. > > In addition, this doesn't happen to systems that I've been following 13-CURRENT > > but rather happen only wiht a system upgraded from 12.2-RELEASE to 13-RC. > > > > > > # procstat -kk 1049 > > PID TID COMM TDNAME KSTACK > > 1049 100215 kldload - spa_init+0xc6 zfs_kmod_init+0x1a > > zfs_modevent+0x34 module_register_init+0x8c linker_load_module+0xaab kern_kldload+0xc1 > > sys_kldload+0x50 syscall+0x17d g_ctx+0xe280bf29 > > > > If you could use kgdb to find out what source code line spa_init+0xc6 > corresponds to that may help to see what's going on. > It look me a while to get kgdb working properly. At last, I got the output. It looks it is spining on a mutex. I have few other machines run the same kernel but they can load zfs.ko. It is only vmware VM that spins with 'kldload zfs'. vmware# kgdb101 /usr/usr/lib/debug/boot/kernel/zfs.ko.debug GNU gdb (GDB) 10.1 [GDB v10.1 for FreeBSD] Copyright (C) 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-portbld-freebsd13.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "helpType "apropos word" to search for commands related to "word"... Reading symbols from zfs.ko.debug... (kgdb) info line *spa_init+0xc6 Line 2345 of "/usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c" starts at address 0x2b0461 and ends at 0x2b0467 . (kgdb) void spa_init(spa_mode_t mode) { mutex_init(&spa_namespace_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&spa_spare_lock, NULL, MUTEX_DEFAULT, NULL); // <- line 2345 Hiro From owner-freebsd-stable@freebsd.org Sat Mar 20 13:24:22 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2CBD65A8539 for ; Sat, 20 Mar 2021 13:24:22 +0000 (UTC) (envelope-from ruben@verweg.com) Received: from erg.verweg.com (erg.verweg.com [IPv6:2a02:898:96::5e8e:f508]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2hLX5Pl6z4Vdm for ; Sat, 20 Mar 2021 13:24:20 +0000 (UTC) (envelope-from ruben@verweg.com) DKIM-Filter: OpenDKIM Filter v2.10.3 erg.verweg.com (unknown-jobid) Received: from [IPv6:2a10:3781:3e9:1:3460:de52:bbd3:ac96] ([IPv6:2a10:3781:3e9:1:3460:de52:bbd3:ac96]) (authenticated bits=0) by erg.verweg.com (8.16.1/8.15.2) with ESMTPSA id 12KDOBq0005733 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sat, 20 Mar 2021 13:24:11 GMT (envelope-from ruben@verweg.com) From: Ruben van Staveren Content-Type: multipart/signed; boundary="Apple-Mail=_D83AAA30-CA8B-427F-AB6F-8471C55D1E24"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Subject: Request: Mount zfs encrypted datasets at boot? Re: FreeBSD 13.0-RC2 Now Available Date: Sat, 20 Mar 2021 14:24:10 +0100 References: <20210313001110.GP92054@FreeBSD.org> To: FreeBSD-STABLE Mailing List In-Reply-To: <20210313001110.GP92054@FreeBSD.org> Message-Id: <817D61E5-1977-46CC-B6E9-D22740B03184@verweg.com> X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Rspamd-Queue-Id: 4F2hLX5Pl6z4Vdm X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ATTACHMENT(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[verweg.com:+]; DMARC_POLICY_ALLOW(-0.50)[verweg.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a02:898:96::5e8e:f508:from]; ASN(0.00)[asn:8283, ipnet:2a02:898::/32, country:NL]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[verweg.com:s=verweg]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a02:898:96::5e8e:f508:from:127.0.2.255]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 13:24:22 -0000 --Apple-Mail=_D83AAA30-CA8B-427F-AB6F-8471C55D1E24 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, > On 13 Mar 2021, at 1:11, Glen Barber wrote: >=20 > The second RC build of the 13.0-RELEASE release cycle is now = available. Might it be interesting to change the zfs mount -a / zfs unmount -a in /etc/rc.d/zfs to zfs mount -al / zfs unmount -au so that filesystems using the openzfs encryption feature can be = automatically mounted? given their keylocation is not =E2=80=9Cprompt" and accessible during = boot. This would make it possible for me to move away from my current dual = pool boot setup, in where the boot pool has the geli keys for my geli = encrypted system pool, which seems to be incompatible with beadm/bectl = handling of things these days=E2=80=A6 The single geli encrypted pool does not work for me as it doesn=E2=80=99t = allow for unattended reboots. Best regards, Ruben --Apple-Mail=_D83AAA30-CA8B-427F-AB6F-8471C55D1E24 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBAgAdFiEEJ5YXTZtFY5bgSLwXiG9r7NR3qT8FAmBV93oACgkQiG9r7NR3 qT84ehAAio941FCMWseGuIeDmEuHsNk8ylg3P4n2Dt7izHEjl/MeZ5KehDGiziId k0oakV3bDBqgRkTNatCLGlBsXJwoUtV6Q1vQ6Gem9JM5WYY9QNJg0NkRv1408da7 kkcHKN3VjMGOQU7MxP9BWuzx8Xx/qQwCu8noNomIO/19UP9WSSrpodlwHe8P8n+q 8zLIucDo/0gn/bvQ9tWKN3oN67afjrvL/gqyZJVnL/w+t2CFCDfl7aSjCqOSDeBi hhHPzuikIuZ5YnzF4WHoUX1766DKhCOI8fo9Z4sQtpHTiI99GRjBSWQEAjpVr9kq jkBhOit6VmYQr7/HyyWkVY5jOFCVErRwDPgn/JIkwObd9Zb8SH8v63RBuCIgRcBx Iuq5A6A9L4lAENvwDwlyH06zV2WgKJrTpqboJBAKn67yk4OH4oRiSZTC7+iajR4e Q2wpBuEW58pSsGsiuNE1xrEfTOiktVtAtaFVyhasIEw4vgf/g7BbbqUIbSt/iJzO ygU8yvOTfYHD6ay1v+iJwWZ6IxYDiQYVvdcWsRFlk8bipIDGr9ehYasE+5GW4GGT dxxLyf9NEEv81munFF8FS4+M+wiU/fAIj3dlU6/M9kZEUoQ/71fRqKNALpTmMXiZ Isnoo7B6NOyo9hgMDKAW059Dr3LYEKiUr7gYVaIOPJgnUVY5yLE= =xsmP -----END PGP SIGNATURE----- --Apple-Mail=_D83AAA30-CA8B-427F-AB6F-8471C55D1E24-- From owner-freebsd-stable@freebsd.org Sat Mar 20 15:35:14 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2863D5AC73D for ; Sat, 20 Mar 2021 15:35:14 +0000 (UTC) (envelope-from freebsd-stable@m.gmane-mx.org) Received: from ciao.gmane.io (ciao.gmane.io [116.202.254.214]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2lFY4fb5z4fR4 for ; Sat, 20 Mar 2021 15:35:13 +0000 (UTC) (envelope-from freebsd-stable@m.gmane-mx.org) Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1lNdYA-0002qa-EY for freebsd-stable@freebsd.org; Sat, 20 Mar 2021 16:30:02 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-stable@freebsd.org From: Fred Subject: Re: ThinkPad X1 Carbon Gen 4 unable to boot 13.0-RC1 Date: Sat, 20 Mar 2021 09:25:57 -0600 Message-ID: References: <1867238560.3032633.1616178410076.ref@mail.yahoo.com> <1867238560.3032633.1616178410076@mail.yahoo.com> <86czvud1qf.fsf@virtual-earth.de> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 In-Reply-To: <86czvud1qf.fsf@virtual-earth.de> Content-Language: en-US X-Rspamd-Queue-Id: 4F2lFY4fb5z4fR4 X-Spamd-Bar: +++ X-Spamd-Result: default: False [3.13 / 15.00]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; ARC_NA(0.00)[]; GREYLIST(0.00)[pass,body]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[116.202.254.214:from]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.956]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[116.202.254.214:from:127.0.2.255]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_SPAM_LONG(0.08)[0.083]; R_SPF_ALLOW(-0.20)[+mx]; DMARC_POLICY_REJECT(2.00)[yahoo.com : SPF not aligned (relaxed), No valid DKIM,reject]; FORGED_SENDER(0.30)[fred.ha11@yahoo.com,freebsd-stable@m.gmane-mx.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:116.202.0.0/16, country:DE]; FROM_NEQ_ENVFROM(0.00)[fred.ha11@yahoo.com,freebsd-stable@m.gmane-mx.org]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 15:35:14 -0000 On 3/19/21 7:59 PM, Mathias Picker wrote: > > Fred Hall via freebsd-stable writes: > >> I have a ThinkPad X1 Carbon Gen 4 (20FCS0NB00) which has happily run >> FreeBSD 11 and 12, but which can't boot 13.0-RC1 from memstick or via >> freebsd-update. In both cases the boot process locks up on the line >> "hwpstate_intel0: on cpu0" >> If running freebsd-update, a work around is to add >> hint.hwpstate_intel.0.disabled="1" in loader.conf. See the note under >> the Eighth Generation (2020) in >> https://wiki.freebsd.org/Laptops/Thinkpad_X1_Carbon >> I was quite surprised to find the lack of support for hwpstate_intel >> in 13 when it apparently worked under 11 and 12. Does anyone know the >> status of hwpstate_intel on ThinkPads? > > I’m running 13 -STABLE from 10 days ago on a Thinkpad Yoga 3rd gen, and > hwpstate_intel works fine, never had a problem. > > mathiasp:~% sysctl dev.hwpstate_intel dev.hwpstate_intel.7.epp: 15 > dev.hwpstate_intel.7.%parent: cpu7 > dev.hwpstate_intel.7.%pnpinfo: dev.hwpstate_intel.7.%location: > dev.hwpstate_intel.7.%driver: hwpstate_intel > dev.hwpstate_intel.7.%desc: Intel Speed Shift > dev.hwpstate_intel.6.epp: 15 > dev.hwpstate_intel.6.%parent: cpu6 > dev.hwpstate_intel.6.%pnpinfo: dev.hwpstate_intel.6.%location: > dev.hwpstate_intel.6.%driver: hwpstate_intel > dev.hwpstate_intel.6.%desc: Intel Speed Shift > [snip] > > The gen3 is using > sudo dmesg|grep -i cpu > CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (1992.08-MHz K8-class CPU) > [snip, snip] > > mathiasp:~% uname -a > FreeBSD Danton 13.0-STABLE FreeBSD 13.0-STABLE #2 > stable/13-n244845-f21c0366f53: Wed Mar 10 20:53:26 CET 2021 > root@Danton:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64 > > > Cheers, > > Mathias Thanks for the feed back. Good to know most people won't encounter the problem. Perhaps it is a bios issue specific to the model. I did update to the latest bios version but that made no difference. I have chosen to rollback to 12.2 as it works perfectly for me. > >> Cheers, Fred >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > From owner-freebsd-stable@freebsd.org Sat Mar 20 15:37:08 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2710C5AC842 for ; Sat, 20 Mar 2021 15:37:08 +0000 (UTC) (envelope-from carlj@peak.org) Received: from smtp.email-protect.gosecure.net (smtp.email-protect.gosecure.net [208.80.202.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "*.email-protect.gosecure.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2lHk1Jmgz4fPs for ; Sat, 20 Mar 2021 15:37:05 +0000 (UTC) (envelope-from carlj@peak.org) Received: from envoy14.neonova.net ([137.118.58.100]) by smtp.email-protect.gosecure.net ({8cc1eff8-78b8-11eb-8141-0fb792f6fe52}) via TCP (outbound) with ESMTP id 20210320153657213_00002462 for ; Sat, 20 Mar 2021 08:36:57 -0700 X-RC-FROM: X-RC-RCPT: Received: from bay.localnet (unknown [199.58.99.76]) (Authenticated sender: carlj@peak.org) by envoy14.neonova.net (Postfix) with ESMTPA id 4F2lHT4GG9z9tKd for ; Sat, 20 Mar 2021 11:36:51 -0400 (EDT) Received: from carlj by bay.localnet with local (Exim 4.94 (FreeBSD)) (envelope-from ) id 1lNdek-0002HH-KW for freebsd-stable@freebsd.org; Sat, 20 Mar 2021 08:36:50 -0700 From: Carl Johnson To: freebsd-stable@freebsd.org Subject: Re: Rasberry Pi 4 has no USB References: <868s786r40.fsf@bay.localnet> <20210228203542.02d8435c12a4da34d59dac71@bidouilliste.com> <86sg5g53k6.fsf@bay.localnet> Date: Sat, 20 Mar 2021 08:36:50 -0700 In-Reply-To: <86sg5g53k6.fsf@bay.localnet> (Carl Johnson's message of "Sun, 28 Feb 2021 12:40:25 -0800") Message-ID: <86v99l25wt.fsf@bay.localnet> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain X-MAG-OUTBOUND: greymail.email-protect.gosecure.net@137.118.58.100/32 X-Rspamd-Queue-Id: 4F2lHk1Jmgz4fPs X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=peak.org; spf=pass (mx1.freebsd.org: domain of carlj@peak.org designates 208.80.202.2 as permitted sender) smtp.mailfrom=carlj@peak.org X-Spamd-Result: default: False [-3.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[208.80.202.2:from]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:208.80.200.0/21]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[208.80.202.2:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[peak.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:208.80.202.0/23, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 15:37:08 -0000 Carl Johnson writes: > Emmanuel Vadot writes: > >> On Sun, 28 Feb 2021 09:26:23 -0800 >> Carl Johnson wrote: >> >>> I have an 8GB RPi 4B that I am trying out, but it has no USB response at >>> all. That means that I can't use a keyboard or mouse, but I can use a >>> serial console and ethernet. I can plug any USB device into any port, >>> but there is nothing logged in /var/messages. Running usbconfig as root >>> just reports 'No device match or lack of permissions'. Running >>> 'dmesg | grep -i usb' reports only the following lines: >>> >>> usb_nop_xceiv0: on ofwbus0 >>> bcm_xhci0: irq 81 at device 0.0 on pci2 >>> usb_needs_explore_all: no devclass >>> >>> Running 'pciconf -lv' shows there is a VL805 USB controller present. I >>> tested this with Linux and everything worked properly. This is a new >>> computer that I bought only a couple of weeks ago. Does anybody have >>> any ideas on what this might be, or what I could do to try to figure it >>> out? >>> >>> Thanks in advance for any ideas. >>> -- >>> Carl Johnson carlj@peak.org >> >> Hello, >> >> This is known, see >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252971#c25 >> >> I wanted to wait that a tag was created on the rpi-firmware github >> repo but it seems that they don't want to so I'll update the port next >> week so RC1 will have the updated firmware. >> >> Cheers, > > Thank you very much for that! I will test that as soon as it comes > out. Just for the record, RC3 now works properly. Thanks to everyone for getting this fix in before the release. -- Carl Johnson carlj@peak.org From owner-freebsd-stable@freebsd.org Sat Mar 20 16:08:56 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3E6D05AD661; Sat, 20 Mar 2021 16:08:56 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2m0S1H3nz4hTv; Sat, 20 Mar 2021 16:08:56 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1616256536; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=iXdjeMoe2Iu2c87e60GHJMP1CbUWgcLlKfRcucwarJw=; b=sJs8EzxwiOVlEIxM51NIp3RpaywRo2D1HxrP8aTd577/O/8wo6rUMZplTp5EY76OlnOlcy gubWcPG8YyaPp1LjZhFzY5NEynZKJyT0ou1VYS1q6Dj4hOuOuO4dTE305OH2cwKtjek1VW 3KzG8wSt9JsCx/zmPs12Ea1CT8x96Qvnw3xUFlJGG3IE14U0GEqOczVTY+9gBOJi4/SAVr NEFLyQv7tw9e0tkahy40F70uIxYPlRsCfq6cHL9Bemp1vfl0lG2ndyfOch3B0hn8Rzs9WM cjiYdRx+c8dg+kXmx2iXNiELt+OL8znLep0n7ZuvqD9NhP3YRrQYG6vyvBd2lA== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id B03339161; Sat, 20 Mar 2021 16:08:55 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Sat, 20 Mar 2021 16:08:53 +0000 From: Glen Barber To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org Cc: FreeBSD Release Engineering Team Subject: FreeBSD 13.0-RC3 Now Available Message-ID: <20210320160853.GL92054@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1616256536; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=iXdjeMoe2Iu2c87e60GHJMP1CbUWgcLlKfRcucwarJw=; b=J63hJr5BrdM6Hkvf+Baeyj3TKp3F1iRuNfLGezI1l5KObenAwPRsKy/S8sMF+FKWewIfbs yfWMwnezXTloePuyGV7SdCH7FnZMIQ6gkKT1r1vUtbgVZ8GnaU/YCa6PjOmvpF5BoaeTo6 dF1wSY0yEuN7sblV38G9roZw2kMo54ncQS/IROtIsEqJTWZgnECT35GmIhwNK0Rlbw0ipN UNus8EcTx38mJhoTNFEckFcXJ5qvou2u8WMkgqsnAz67weP28CS67zDovuTh0viHj1VoNR +yWZvr+QO4jD9p1nVeUf6YEIW+b/p6MwDxNQN3JTtHjKuTkgccmrPDA6y8ZT8w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1616256536; a=rsa-sha256; cv=none; b=cv8PuM+owcIn01T4jqvP2y1RNx+DtG4IeNOmevdAOhTEvFe/r/VhCh107B2CrLeCDiuyov EaiY9q52E3xN2cfQwy94U82z5yP5tFXF8riynd9gQ2I69eYLLvpVcNwZqv7nBkpapwsrNK JnjGyVChUWWQPBNp4qzj5/OfhxtDF3gppFGN6JeMiSMWLII7jtndre2dQM8AseNwyR+YdX RwY4DRCtpiEXj60pKd7TWapuVsit4UO4/+l4BezkBZwWjTW+q36ecps6Iu0fF/1RF+Vk27 sukbVWRbsB0FcLCsUOzyDt6j8CaSkSEFeJYlP0A2RH7c5neihAs/jcXJT6Uiqg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 16:08:56 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 The third RC build of the 13.0-RELEASE release cycle is now available. Installation images are available for: o 13.0-RC3 amd64 GENERIC o 13.0-RC3 i386 GENERIC o 13.0-RC3 powerpc GENERIC o 13.0-RC3 powerpc64 GENERIC64 o 13.0-RC3 powerpc64le GENERIC64LE o 13.0-RC3 powerpcspe MPC85XXSPE o 13.0-RC3 armv6 RPI-B o 13.0-RC3 armv7 GENERICSD o 13.0-RC3 aarch64 GENERIC o 13.0-RC3 aarch64 RPI o 13.0-RC3 aarch64 PINE64 o 13.0-RC3 aarch64 PINE64-LTS o 13.0-RC3 aarch64 PINEBOOK o 13.0-RC3 aarch64 ROCK64 o 13.0-RC3 aarch64 ROCKPRO64 o 13.0-RC3 riscv64 GENERIC o 13.0-RC3 riscv64 GENERICSD Note regarding arm SD card images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root. It is strongly recommended to change the password for both users after gaining access to the system. Installer images and memory stick images are available here: https://download.freebsd.org/ftp/releases/ISO-IMAGES/13.0/ The image checksums follow at the end of this e-mail. If you notice problems you can report them through the Bugzilla PR system or on the -stable mailing list. If you would like to use Git to do a source based update of an existing system, use the "releng/13.0" branch. A summary of changes since 13.0-RC2 includes: o A virtual memory list locking fix has been addressed. o A fix for systems running under VirtualBox has been addressed. o A fix in the service(8) utility has been addressed. o The if_wg(4) pseudo driver has been removed. o A fix to TSO for TCP/IPV6 has been addressed in the vtnet(4) driver. o Several other miscellaneous fixes have been addressed. A list of changes since 12.2-RELEASE is available in the releng/13.0 release notes: https://www.freebsd.org/releases/13.0R/relnotes.html Please note, the release notes page is not yet complete, and will be updated on an ongoing basis as the 13.0-RELEASE cycle progresses. === Virtual Machine Disk Images === VM disk images are available for the amd64, i386, and aarch64 architectures. Disk images may be downloaded from the following URL (or any of the FreeBSD download mirrors): https://download.freebsd.org/ftp/releases/VM-IMAGES/13.0-RC3/ The partition layout is: ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label) ~ 1 GB - freebsd-swap GPT partition type (swapfs GPT label) ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label) The disk images are available in QCOW2, VHD, VMDK, and raw disk image formats. The image download size is approximately 135 MB and 165 MB respectively (amd64/i386), decompressing to a 21 GB sparse image. Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI loader file is needed for qemu-system-aarch64 to be able to boot the virtual machine images. See this page for more information: https://wiki.freebsd.org/arm64/QEMU To boot the VM image, run: % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt \ -bios QEMU_EFI.fd -serial telnet::4444,server -nographic \ -drive if=none,file=VMDISK,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -device virtio-net-device,netdev=net0 \ -netdev user,id=net0 Be sure to replace "VMDISK" with the path to the virtual machine image. BASIC-CI images can be found at: https://download.freebsd.org/ftp/releases/CI-IMAGES/13.0-RC3/ === Amazon EC2 AMI Images === FreeBSD/amd64 EC2 AMIs are available in the following regions: af-south-1 region: ami-0ad55ef28f5927282 eu-north-1 region: ami-0fde22c6026e95a57 ap-south-1 region: ami-06fd5c64b54c3e005 eu-west-3 region: ami-093ba1f2661abd0be eu-west-2 region: ami-0803205f015f5ec14 eu-south-1 region: ami-03e90955c75a8c809 eu-west-1 region: ami-0d9313767b0b818fd ap-northeast-3 region: ami-0f25c989c843d8dbb ap-northeast-2 region: ami-0f27bdd4fd819bb6c me-south-1 region: ami-032ee25ad2976ad49 ap-northeast-1 region: ami-0fb5829e3ff1f1b15 sa-east-1 region: ami-0f66abb1219e00a5c ca-central-1 region: ami-062381d2a77db4a3d ap-east-1 region: ami-0d21456c4a5bd1384 ap-southeast-1 region: ami-0519190624816cc77 ap-southeast-2 region: ami-0805e040cf3167547 eu-central-1 region: ami-0abb1093896437395 us-east-1 region: ami-02f65312a05a1abfd us-east-2 region: ami-023106328f1d2d3c7 us-west-1 region: ami-00802b96b8eec8b49 us-west-2 region: ami-090a96d68eed457a1 FreeBSD/aarch64 EC2 AMIs are available in the following regions: af-south-1 region: ami-04e1b958484d79c53 eu-north-1 region: ami-09784ab80e6a9812f ap-south-1 region: ami-0641ef56dae855387 eu-west-3 region: ami-0cc83c013292eb9ee eu-west-2 region: ami-033fa6488ab12698b eu-south-1 region: ami-02fe3008ea2757fed eu-west-1 region: ami-06fdd3bce845d8c96 ap-northeast-3 region: ami-0afb61493a8eb593d ap-northeast-2 region: ami-0a81e284ff987ba61 me-south-1 region: ami-024551ac2729be872 ap-northeast-1 region: ami-0a77688c2dad3202f sa-east-1 region: ami-03e01af30fabcee9b ca-central-1 region: ami-04fbc9b6d1a72f279 ap-east-1 region: ami-09e4ceed8b9d050fc ap-southeast-1 region: ami-05deb6243327e00dd ap-southeast-2 region: ami-04236f49ca0d42756 eu-central-1 region: ami-0df095df8f0d66ac9 us-east-1 region: ami-04a67aea020cc9cc5 us-east-2 region: ami-0c5f779125f936dab us-west-1 region: ami-03bb5fbea314fedcc us-west-2 region: ami-0b56603a85f1c8449 === Vagrant Images === FreeBSD/amd64 images are available on the Hashicorp Atlas site, and can be installed by running: % vagrant init freebsd/FreeBSD-13.0-RC3 % vagrant up === Upgrading === The freebsd-update(8) utility supports binary upgrades of amd64, i386, and aarch64 systems running earlier FreeBSD releases. Systems running earlier FreeBSD releases can upgrade as follows: # freebsd-update upgrade -r 13.0-RC3 During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly. # freebsd-update install The system must be rebooted with the newly installed kernel before continuing. # shutdown -r now After rebooting, freebsd-update needs to be run again to install the new userland components: # freebsd-update install It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example, FreeBSD 11.x. Alternatively, the user can install misc/compat11x and other compatibility libraries, afterwards the system must be rebooted into the new userland: # shutdown -r now Finally, after rebooting, freebsd-update needs to be run again to remove stale files: # freebsd-update install == ISO CHECKSUMS == o 13.0-RC3 amd64 GENERIC: SHA512 (FreeBSD-13.0-RC3-amd64-bootonly.iso) = 1d8234d83825754fad4c05783196e9e2d4f447c10eeb17a6c548a7ab59b4107d8046d8df8880efc2197fadac4f2b13cd33ffdefb7e4366d8f7a2aa694d56f464 SHA512 (FreeBSD-13.0-RC3-amd64-bootonly.iso.xz) = cae723447e08991ee202c5c43125c5d6430c274fcc55f9334ceabd17a1b5a8077cc2f38124136cfab8ea080b63f3cec08a0aa90263cc357cfad377ad7893e3bb SHA512 (FreeBSD-13.0-RC3-amd64-disc1.iso) = 31b9d9e9488af2f370c9181ed47a23dabbc58853c070d60233db5823c0e80d39d104852e20399c4b755b38eb76cc34da909098a08b6e4f064b9f05077f7a2927 SHA512 (FreeBSD-13.0-RC3-amd64-disc1.iso.xz) = ebe6ee173c283b0f684d1dc772191e6d80a793acde3c8d26f5efb4e9fa90c579d57f5bb6ba3451ce64ccf3f8a7e630f580b343e4e6b07535aab49d9868841cd4 SHA512 (FreeBSD-13.0-RC3-amd64-dvd1.iso) = 5ac4c60e37506c5305bc29df7678a47dfc04150a8826d9389931c54a8f244740f06251edf63bdfc31f9dbcffcec7b2933c77777bed42024e05930145e590ca18 SHA512 (FreeBSD-13.0-RC3-amd64-dvd1.iso.xz) = 542c4b322a51461bec85321babd2791a5c1cb038b7fcd25609040c7a19d449c4978f9e88bd362ba040d60a50db00b0f8cc4b9f5ba881a6502f0f23a6d2db4f8b SHA512 (FreeBSD-13.0-RC3-amd64-memstick.img) = d7a7388a1d8398af245a5bf40a3e8fd2e137a0b6dcdec5d10f0f1bbf834fe2477ef1fc0ba80e7facacc7902bb6983c4fbd928539d6cabad0402d9d89735ac6be SHA512 (FreeBSD-13.0-RC3-amd64-memstick.img.xz) = 564b51529bad0d17eac2ac07d538f20434575a6f9fced0313965f518c35cdbe0d771f1d8440c731de9e38d82b5561c15596243a2b431408391af6828e19909c4 SHA512 (FreeBSD-13.0-RC3-amd64-mini-memstick.img) = 7cc0ea6ea104fdf9938300534757f261bc3992a17bb0de78e59cd6bf7c29de88155d7a2fe0919c3bcd7048d73e9a140dbd3caaae842211e840802a9fbd5c2411 SHA512 (FreeBSD-13.0-RC3-amd64-mini-memstick.img.xz) = 3f6e0b98e969b5e18036ca905a9ad613aeaf804e07395dde9a17f5004bdf1acae35782dd0c1a794c546d7184681873d519eb962859b670b863ba565830d3c5c5 SHA256 (FreeBSD-13.0-RC3-amd64-bootonly.iso) = 61f9ccff52e463ea498ab8e0151c094f2e5410ab78fd9b20cd82ae1aae14256f SHA256 (FreeBSD-13.0-RC3-amd64-bootonly.iso.xz) = 452dc1b7a28b9c84b0777b6ba3069657ba3bbfb51c8fc7be52773e100a1ca2be SHA256 (FreeBSD-13.0-RC3-amd64-disc1.iso) = 45d44170a3079b813d8e0033e1a99f10d0e8ed0ac349aeaf25193a33553c9601 SHA256 (FreeBSD-13.0-RC3-amd64-disc1.iso.xz) = 36424f242f18db36883b7f532545a548e97b61f2107a9fe98e1562b63ac5570a SHA256 (FreeBSD-13.0-RC3-amd64-dvd1.iso) = deb84cd7a5d9804bbabaa6cc8642ad02e537e9dc2eb3b9c801c52fb8b3d79d30 SHA256 (FreeBSD-13.0-RC3-amd64-dvd1.iso.xz) = 4494197d4b746728b6407bda62b7c77af7798aa5562d50aee70ce6fd4f291426 SHA256 (FreeBSD-13.0-RC3-amd64-memstick.img) = cc8162d5ebf79280e8fa41e1345ee985b86351aefdd77a90ad675eabab325c7c SHA256 (FreeBSD-13.0-RC3-amd64-memstick.img.xz) = 865f3873f797aff92e6215ef28d9ae97bb69dfdf7acbeb17d70799ebcf982fd7 SHA256 (FreeBSD-13.0-RC3-amd64-mini-memstick.img) = 0832bc13a555780772903073f1242471e25d3676d37a28ea16364a5d25b817b9 SHA256 (FreeBSD-13.0-RC3-amd64-mini-memstick.img.xz) = d1c6e3d6a382d876981fb3f3f8c4929bfefab2290be279fef41a5a85314f95ab o 13.0-RC3 i386 GENERIC: SHA512 (FreeBSD-13.0-RC3-i386-bootonly.iso) = 87046f2f2636c0a4119d9ea88e611a7ef53d12bdc71563f75179d3b5bba26d36934a186ffe7d452e1d191fe432a76ca53e5f56c08dcd3be249e209566db3ad75 SHA512 (FreeBSD-13.0-RC3-i386-bootonly.iso.xz) = 81d2f40daeabe22d70506da6c4739fbd466380ad50cde08100ce2bc5fefc2cf952752090f003b6349c43c985642741fa222289feb7ac45aab65a0af2e6238e7f SHA512 (FreeBSD-13.0-RC3-i386-disc1.iso) = b9747f5c5062fe75ef1ecbf343a8fd46285061ad23068e381eabdccfc198f9e168c7912fbeb14cd756e8df87d2e8382711390a379d32b3e81063814e32c6f3ba SHA512 (FreeBSD-13.0-RC3-i386-disc1.iso.xz) = 7365cdd37e2edc0f4cefcd045d64ec310ba7a224c45c7f6e08e06f25fe6fe3a2cc68e7b66010a9738da351edc0fc6ba4a690562f641d432cf333e288c024d1f1 SHA512 (FreeBSD-13.0-RC3-i386-dvd1.iso) = f40f3c7d69c580dff2db3c2e67487a42fe98e8cf6d979d82fa88f02561b90860231c52ce669467dc85a91a5a59fe3e86dbe982c01b2b032d27382f2c77b7a9ec SHA512 (FreeBSD-13.0-RC3-i386-dvd1.iso.xz) = d4a0abee3be88156914e5b23e833f14d6dc8bbf96c7d470b61d08041bf597c8e31da23219d74079a05c5cc7d91c70a83e0a06bbdb65d639da2b464149e9408a7 SHA512 (FreeBSD-13.0-RC3-i386-memstick.img) = 9bf6d0cbeccabedf3da57a42b3ba0580c218573d5a77bd36b4e90925b26eaa8a8706df808c289fbdd264683a1a5397efc88ecdd2fa377853c3851489671db832 SHA512 (FreeBSD-13.0-RC3-i386-memstick.img.xz) = ac65933dee3753c4584ad7a7acb9ce7e0d3fe05a83ddd6e3be0da72ac92bc51bc5cd7db836e2775bdb585448713a9cb8dba6c4ad9260f9d14e3b699b89cccd86 SHA512 (FreeBSD-13.0-RC3-i386-mini-memstick.img) = 8167ebfa75fb02ee7eade7c0e91ab1e775de71e457fa3bb773c29ccf3386df3db1b723eafad4586ca43518de3ea85e4268e363f162c348cb54ceea36b8f14fdb SHA512 (FreeBSD-13.0-RC3-i386-mini-memstick.img.xz) = 31064cf16b56bf09593bee6cc4ef4155c6b6e41592e81d8d3593f327fa69cc30af41874af1e1c8d5e0e347e670c5ca2267b7702afcb675e2c286b6be1c6cedc4 SHA256 (FreeBSD-13.0-RC3-i386-bootonly.iso) = ca224697c104eb1ab8819bb45ba1ef8af4815e4e04ce550ee83a21c1953d40b6 SHA256 (FreeBSD-13.0-RC3-i386-bootonly.iso.xz) = 99d26efb2b6be602f9ea62d1efee784a34573a1701674fc1c7cb1275680e7664 SHA256 (FreeBSD-13.0-RC3-i386-disc1.iso) = 0c4b1d061db20df8f615237054c83434ba9178e36419634a91b736fee5b6472e SHA256 (FreeBSD-13.0-RC3-i386-disc1.iso.xz) = 54970c6f228d483ccba5a375ce2efc329894a1c06575c438190356270ab97293 SHA256 (FreeBSD-13.0-RC3-i386-dvd1.iso) = 6c14c841fc824c4f8859d990f07f750f98516436d4091c3b85808fc0f10e3ea6 SHA256 (FreeBSD-13.0-RC3-i386-dvd1.iso.xz) = 283ca6ec1a804397e3802f23df9134812ecaecbcf0a1941aae5cb2ca14b06ea1 SHA256 (FreeBSD-13.0-RC3-i386-memstick.img) = dfb288c6125be214ca70414b0d4c19ca12b8356f99a7cde081bc5604f39ed3b4 SHA256 (FreeBSD-13.0-RC3-i386-memstick.img.xz) = 2c727559a052c6627d0698193596b7e47c6d27d3091c8715e51bfaa22a3fc286 SHA256 (FreeBSD-13.0-RC3-i386-mini-memstick.img) = 24c5e2e612c2b9c0626f623006bd49cf7b0eb53e0c7e792a12e33031223e218c SHA256 (FreeBSD-13.0-RC3-i386-mini-memstick.img.xz) = c0d52746137c951b1ee20fb7a8463c9ceb0143e16812ddb2398224510435df87 o 13.0-RC3 powerpc GENERIC: SHA512 (FreeBSD-13.0-RC3-powerpc-bootonly.iso) = 878c1a32539e9c8483e96d103891a3725d829b46ec4a8132f9312e3785ee640de53592eb67522a330fcc49d09f59772b508f5393d7e3a9b0dc26d754464c31f7 SHA512 (FreeBSD-13.0-RC3-powerpc-bootonly.iso.xz) = 0888f2dfdc6bd6d582c04fb8d920a8638d303744e973956a03f2ed9d7426dc0b28652ac5e6427fde38951f29da3e6fe1bd425ac9ba7bc4c88ee4f04d252dcf16 SHA512 (FreeBSD-13.0-RC3-powerpc-disc1.iso) = d4966f4c534daaf6fef855d98f9a554aaacc18e122309648fee9ea9b2760d676a53d4cf89f58bcb45d942232ca1c7928502226f6d5a998d91d8969a2834f2f86 SHA512 (FreeBSD-13.0-RC3-powerpc-disc1.iso.xz) = 6d049458b316fc899d664f8980cc10b0732ab3e167200736a552bc4be9c323e8087c01698af38a5a95b83f73475da6f165f21f1ec576ad64d0789229ef68cf29 SHA512 (FreeBSD-13.0-RC3-powerpc-dvd1.iso) = 3af2a243d6cadc7b2458a592c6537f2316a3ae2bb62bef7de71ed2ed00c08b55bce802bd9d7a9c8eb348ced139f85b8a56862fdeac3dcb05634c2d83357c980e SHA512 (FreeBSD-13.0-RC3-powerpc-dvd1.iso.xz) = 52a2e33d8a8b7e83b62fb97302b4a16120c58e0d5fe7532aa92f0e9c346f70a0b714ea03c9cb5ec177f8d7261e9983d5365bbeb91d74e3f31f2a9b6daf53f9f2 SHA512 (FreeBSD-13.0-RC3-powerpc-memstick.img) = 0dcdc1762efcae98cb7d16d2294cc85987c39369d1d359beec31c1077b960b51c3985272c3ca86e19989c06724b00dce19cb0efe586f3538a6f30aa9a7350562 SHA512 (FreeBSD-13.0-RC3-powerpc-memstick.img.xz) = 4c96d70729c116b1033a15de5aeaa58c054a80e84405219008abb1fccf04323df52d5f027871426f2c85736b5e4680dafcecf7244d7599ac9712aa49d41a98e6 SHA512 (FreeBSD-13.0-RC3-powerpc-mini-memstick.img) = c7e92c709677a85b795cfd2108489ec0b35fac923ff88628f82ce240b9bc8e79b1ef90921e7120f0002956c6d5f42fb139684e5eaf108befae476a13a0baf1d2 SHA512 (FreeBSD-13.0-RC3-powerpc-mini-memstick.img.xz) = 9d48249c75fc2ad6a526c34117b9318994d93a2cb80814cdd73442239fa93e91c5bad5adedab777fc55f61af0ff196d12501e98fdece70cd838bbcea40ce2b01 SHA256 (FreeBSD-13.0-RC3-powerpc-bootonly.iso) = 4066eaffd07f593ff8917a02ae7a2291fd830a43bad2491f9a4f0dfdcc22a3cc SHA256 (FreeBSD-13.0-RC3-powerpc-bootonly.iso.xz) = fa2081156359d7a9f6c8459cdd1e525fda58a9d4a4a516fa7faf679679bbda39 SHA256 (FreeBSD-13.0-RC3-powerpc-disc1.iso) = 8088574a0589d13221d461548f39a83e23041dc0f689ecc2a03478778e1de374 SHA256 (FreeBSD-13.0-RC3-powerpc-disc1.iso.xz) = 2825be88eb4a9cffc0df728db3ee77bd7c73c6f581fc036049d738126636cea6 SHA256 (FreeBSD-13.0-RC3-powerpc-dvd1.iso) = 13f7ffeac6c820572618b61c2d278b3237fb21e0c19d4fcd8dc9e96fc360c8c5 SHA256 (FreeBSD-13.0-RC3-powerpc-dvd1.iso.xz) = dd3a91f2d6914e3d95d03f39babb5555675938a709b70fb22ea8038b6abb99f4 SHA256 (FreeBSD-13.0-RC3-powerpc-memstick.img) = 828b06ce910823bd4e968f61da26d2169e70a987cb89a75f255678ff685daf80 SHA256 (FreeBSD-13.0-RC3-powerpc-memstick.img.xz) = a662e3ca383b835cdf3f19904ac1b60ad9a39de2f5c9a613f83ab05af1d12e72 SHA256 (FreeBSD-13.0-RC3-powerpc-mini-memstick.img) = 3c811f437b4a76d8cb33f7db1765490058a794725419e27291a929384620b44c SHA256 (FreeBSD-13.0-RC3-powerpc-mini-memstick.img.xz) = 4b8318c51b07e19d30734681e6915093b1ca0cd8f3f8d447cc7ab9b56c7534b0 o 13.0-RC3 powerpc64 GENERIC64: SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-bootonly.iso) = 0be147d614b9fe5ad8f003797422d5dd6d8c459754109e0bf358930bb37fbacdbdd5007341abe44dbf7a6fee191e0b5f82833e9d35466f8e705b9e388e5b60bc SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-bootonly.iso.xz) = 3322c80f3148b744fbea7265913eb6031af4a1d2aa419bfb3deda220db45c81b27954a58612bd8c1d49f04751c1ab70e3110c6329ace023323693fd9f7a6f8f7 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-disc1.iso) = e8450533c3291a90eaa30032ea95793648499e6bf2834c2a1cbce873b7e68ae3c4ed69159097532f6c01b8545806d4707b22df61d406a27119b00ffdb431585e SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-disc1.iso.xz) = 5d04a070e4ecf2fa5098de6b50f9ac16efce353b78692371e15beb3a630a12a5caf199bb1b6163f7847ef073b2d2e3ac9e5fac97c8beb4efb042c7e08f00e357 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-dvd1.iso) = a96d5b09d2abdfa2b13fa2f5e0f4b95a65528769be7929623b1416b5285ec4043122cd491f2b076b66ac4130fdd54f98df5643a4092f185101ca7cdece73417d SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-dvd1.iso.xz) = ae1db436b5e52e27a6f7987b77185cf2f153b7002df757e676f432d1e4a7e5ea8935b9e5b604c4602aa0a2ea7c7ccf2e56250cfdfb3aa73af528ec62d09b0978 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-memstick.img) = ceced804788d55b8eeb7188195ba2b62da551e06185a4c6f6740be38f13cedbdb5a18846f045f91086c1f5ca4783fcd8742834b9ae5df9f6dfebcf777b328cf5 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-memstick.img.xz) = 8c6642873208606f3ce912ecdf519cc89488feaee55ec3228370740ff7c8d3660ccee95cb5c28a4ee40d1da221cfa1cfc1be13e92747e489e14bf2a0832b7e9f SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-mini-memstick.img) = 5bf5c533b9e002a72f23f50381f114b0e024ca304d9c4cd4b88bb8fef8c045faf27a9779f69f5f0b0b1c66a5e06c389c5229b29defbaccce1d5eee4612fc3574 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64-mini-memstick.img.xz) = c0c0fad62772a940d0563638e5ec3b391688b81fa3bee8befb817dd1c689283d508eb9c531dbbc0098ed30c6f6e0a51210e163a39dd73764f4f7d229ee58a08a SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-bootonly.iso) = 9fe607162acf6f3155716ee517cffd2049900c736166858b7f5b0d8ba44c924e SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-bootonly.iso.xz) = 9b3d631f180cc0a1ac88161359c51ff52644a6964d7c6add707f55af1bf03ec8 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-disc1.iso) = dbc5ff4f6949051c5b32231d223a2e58c7ebde74736d21001bd73eef76c5d0f0 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-disc1.iso.xz) = 0b7ce4c0f018457ba296f09dbf24113f73fdf7a5dea7010e111dfb3a0c5c0bf1 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-dvd1.iso) = e89482b35fa7d8256e35891cf45d0e4afbace45078249033a95e26cbb025e3f2 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-dvd1.iso.xz) = d21437dfd71b7e2dc35271195828409185432937bff0f3591969c426c85897e7 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-memstick.img) = 42d54e6aad696e426ef40fe3089a30c7d890ff80829e02d7981810e717f7a93d SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-memstick.img.xz) = 404718b19b82553b9d8b315f4761e69759f30922fe8caffa2abd21e62616e690 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-mini-memstick.img) = 5dcdcdefdd9d6ad348269af00a0ef53fc7bf1f8eeb787aca2a22686ad984f029 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64-mini-memstick.img.xz) = dc3b9ffdb17035ea65b06426e36927701d5aa81f903521a58e93ad4447c2ed1a o 13.0-RC3 powerpc64le GENERIC64LE: SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-bootonly.iso) = 5ff6c54afca6d20585647e59466240c221a77df514f974fc4ce451c2b69189ea4ee3ccd54dafcf2b4a40c8a8740e836ad521150b7fce77bb9d41fd21119d76d4 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-bootonly.iso.xz) = 299521f0f90fb81bd8a7dce33affe5687dfe7b8f10f0ceebbacd973298b50764d8fcc225c37b9b509f64effcaa5352e3263ad28d0c57f7f870d692285011e37e SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-disc1.iso) = bfeccf30ed83129cd25daab391bd85df6ddfcb9d7886919da3c35e9158ce7faf5f37b2cdf9b2bb2d490fb233506ecf2e1c239fc7460da6b9bf44b5abaf1c7f47 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-disc1.iso.xz) = 7039facc741be13af26572ea6ec09d4e60e41b559cad1de218ed77e923a948af26c01d060baada18575aa1da178f9653f1fb2603d78c29ce8837c42fd2fe694a SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-dvd1.iso) = 439fb6fd56c2c091c029b9d00a565d8347530b981bf6cd987cf2b7406c2418c577fbc8ac7399e892673469ad4384910825fc6f1fb83709e74af764f7ca55fd09 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-dvd1.iso.xz) = 14084b02698b352b44a6ee8c204e788aa1f86fc4ee034f8ef2314b0fb62b8361c7dce4d5f6ac2e5beeaf990a16ae925d6da60e858f30e90f27da2610dc72fe82 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-memstick.img) = 1cd897bb5b9a6ad0512dc3e5065f14fdd255947fbbd2c60015d4dc9c742854aafdca316715f1c049238e01c43c2a0ab6a84109c1760ebcba3655f81a386d891a SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-memstick.img.xz) = 0dac6d8de42101ada78ee37b170d957cab549e4d9c2927f34742f384ef4d68b545939a5fd5584398f6c0af2620de4e458f1cde1b5a1e419982679591dbeca7b5 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-mini-memstick.img) = 606946c3bf807d05a4d99873739b78b41df7b1c6fdaf70e7d3eab3b754f9a36c8f13a2d72d63fd0617765cd88aad212418be5b5b9732ad72b707a52b47bd23b1 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpc64le-mini-memstick.img.xz) = dda2eb50a0f18ce509d9bef57ac6ec96cace4b98dc6181d6b11f900c0fddf189ff73cf00a81986e9e3dc0944836074ac41d3e5e5aab9aea93c4da191078711e9 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-bootonly.iso) = 0e506b45827dbdf95b8a93bfb93897b90a829cd34467f79b937a839a3323c6a2 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-bootonly.iso.xz) = aea5f78ebc1c3fb893e1c14a0b7c52700304da5cdb5bd57f8e9707a833bac3c7 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-disc1.iso) = a87b529a6d4dd1901696976d2242ad9eade35cb9836ef3b23352b9b68a8dc408 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-disc1.iso.xz) = 2f6df29467a8323382d963e7589fd6374243c4dc16471e43870d71df4f1bf463 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-dvd1.iso) = e8cd5c1fdc88f03e57f9cf8fee673fa9180845b73ed52ef6546b0d323ab77515 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-dvd1.iso.xz) = c57bae422bc519cba108e712fab8f3cc3c6678d9689f4e8b015db6b9e9cfe2d0 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-memstick.img) = 9de982cecdecec99de537e924c0335dfcc0ae1f324d118df4105ed0b987a378e SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-memstick.img.xz) = 2d734610349d1f7003cd1091bee1d55f26d4415ef5e9e793fb7d10b70e6c23ea SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-mini-memstick.img) = f12c27b660938c5c7c3c4cb0ebefe5843b5061e7fe9f418009dc57c9e25c4436 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpc64le-mini-memstick.img.xz) = a0f207e9d6021303ce8059f9c328a185f4baef8513e01b0512c064169bae5967 o 13.0-RC3 powerpcspe MPC85XXSPE: SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-bootonly.iso) = a0516a1cfa3366793aa7c8d8edbffa67da66bfce8f27d531958d286f6596c972ce66e1489e7390e6d5e4464529c55eab0ef575e81ec4a172d0144e0dd1c35ec6 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-bootonly.iso.xz) = 6212285ca75c6a03d2fdddb8fd62e426f27bc8f90177c14e86e91abd9ed543828b4e7e370ebf9a8d604654c30550be12acc0f84549f4b303f194bf7cdb65dcd2 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-disc1.iso) = d1c8cbc3853e135eded550d94eb629dfd2f48a9736456ecbe6aed248fc496010c5bad85a5a1b0c0150d389b1eb344cff24910cf6e36587e434b475dbf216009d SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-disc1.iso.xz) = 887ccfb78a32033ef1b2b960eba73c13bb84df2e8360d08517b6d60ee6d3ea3abac6002ed555d075115a60bee406d486f0c65fdf7b67a0320eba207dd0126b6a SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-dvd1.iso) = 9628330d91f52fae354d899fd0c11d79d241ff16eec7f239d60a39a1f7a76a4e8777894c33d159c4f8c21315fff589833bbfa59bdafbf8a2b64a6474f26077ed SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-dvd1.iso.xz) = 9102fac6a11ab298d2ebc3b4a65e963c7e3e06722373cfc2afcc005baff0169f7b0f8776d4c1f5fad57c2959d9fde675f411a3233df94d52637650b013fd1148 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-memstick.img) = 3eddd49544f4e90b57ab0f4b968a9bcb1743c7cb9e445ffcffd60faae5e98e688e8f414bba84ef4d2c91805da343dab1ae20f68c35acd280e5008703d21ff028 SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-memstick.img.xz) = 270e41ea166967aa0fd1723c102323421d16709e71b62984d9d1eca0b19bd07415dcad620e384e1122ecaab0101adf4ebd60d470a4cbf2c01f1c217b7034f4be SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-mini-memstick.img) = 36b3b386565fe44b13105dde59623a8a31b63e4553e28b02997ef7f3e6c7551cb2eec16f28bd0aec28e973a01c2385533173065afb1e384a151f9496d49ebb9a SHA512 (FreeBSD-13.0-RC3-powerpc-powerpcspe-mini-memstick.img.xz) = cbb8650a7f5bca3b59cc0029dd08dce9ee95373e90df865eea38b0b49ca7eab82da7581db3f41b80dc3551850511ae21d0d71deffc2b9c2ba3dc991521601426 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-bootonly.iso) = 0f610c5a299f1a15f8543481595e276efa281135225c31663f6da36374a11ddc SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-bootonly.iso.xz) = accb9a7848ecee7f619ffc87ca82ac49b6662daccb5f314a22b55c5482cd73f6 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-disc1.iso) = 3675ab93e9048ec1e93f462eef1f7b146c00a5d03b2d308a5250e5a66a2ed7ed SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-disc1.iso.xz) = 5a85e594991c2abdfa47dc7c908de6e48518c333ff2f1d6c83b488aa31fb4b18 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-dvd1.iso) = 4a195bd5cbb0416cf2640675c753670311326992385ce84fbb19f587260d85ef SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-dvd1.iso.xz) = 8abeb2f0c66f94430a6e8dbe5980d09b01d8daa7a061ef88f57498fdade02f22 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-memstick.img) = 55e582e947c787c68e6b4de2b48ebc2156fc443366517f69233806648a9703c8 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-memstick.img.xz) = 1e99c45c862733843b0ca8f98fbc2119fee0f9fe4c0032096ddde3014143c76e SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-mini-memstick.img) = 45585d171c62a128dd1c035cde8d956a83541e984e2fe6c98f274c7c9a2dfe40 SHA256 (FreeBSD-13.0-RC3-powerpc-powerpcspe-mini-memstick.img.xz) = 13923dfa411c15c70de2dbaf96f9bdb7411f375ce723b3d4883d36f9da3eb097 o 13.0-RC3 armv6 RPI-B: SHA512 (FreeBSD-13.0-RC3-arm-armv6-RPI-B.img.xz) = 3c293abaadf4a6615dcd9be0baf444d32bcfc2e5ba3a13f88f311c8e865a4ec98d9e79999ddd7be721ed862c9c3c1f969763215b00001365582cdac6c2b0124f SHA256 (FreeBSD-13.0-RC3-arm-armv6-RPI-B.img.xz) = c9735ac85f6588a97e7be608aa8297ab6d3d81ce4e0986b7a8380c45b067f6f1 o 13.0-RC3 armv7 GENERICSD: SHA512 (FreeBSD-13.0-RC3-arm-armv7-GENERICSD.img.xz) = 339ed5f99140b1a992e35d4098e722d60700573d118a310554590faf5fbe452fa13462f588c3d2cde60761afbcb8224f308890980f30e39edea05b7a25347b88 SHA256 (FreeBSD-13.0-RC3-arm-armv7-GENERICSD.img.xz) = b8ae46bffdf781be6735c1e11ecbd9644d7ea02265e8cde0ea28abb6c5e28722 o 13.0-RC3 aarch64 GENERIC: SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-bootonly.iso) = 67d54096fab752615056888144658185a38875500a14a4a8316a0b70ee781dac8d3c7f3a9e0170807d022c2eb6a6828487b27cd2a6270d3efd01bc1e097e7eaa SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-bootonly.iso.xz) = 957c8620797f6fbda17b544a3aef0239371bb8dcbcfb0c7d0fc182714a9e53a1604152610736f3f50dba863bfc8da669d688ac9b7eda74cc3f9e4c163d6be4cf SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-disc1.iso) = 5b24cac9a8f497aec0459a2be7e7c5b9ef3b08315049f8bc98f00b6412a8bf62fa7459bfeef8a2aba83b416029f33e1ed1a24bb0556dc4bae087510f9707f0db SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-disc1.iso.xz) = 243edb015184041dd0c0237c55f107c4a052ef008fda5ec2bed641c9c99f74cbdf9a860a61fa871e7f186fded00483b36cb5ffde2787e35d7ea934e3c00f6a82 SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-dvd1.iso) = df621c853d9169683298fa878caf52cec6d585a99f09f01e3b441871b2bb91309c354f6cbd7649eca6a8873b680096b883380fc828b8e2a3b95e34f623af79b2 SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-dvd1.iso.xz) = 21103cd151f4f1f69c60408ee5a4ab2f4bb0e9b31e97711d6ffa68e5db18b52d33c246f7e8a259ef137ce4866710e8752c263bc93faf53c31fcb750697c7a231 SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-memstick.img) = 12f008293d38cc54927b32727ddbd0f5150ae066119cc7f72d465dd3ada9377435481504fdec660b73b2229b0157316c6d409ef5843b145be9e9e8f1784e5f8f SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-memstick.img.xz) = fc6bf7bc26efdb15d5993214b31b71962193d2a9d46a97b0db77cb397840cf284933d497d1338fc15d46bb87a06349df1950d52ba1efcbaf073f3275ec59e4c7 SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-mini-memstick.img) = 5f0a075ae538995fbc171604f107be60498bf8ab215e55d9fdad98069622b06d327f35e82dbe9d6912ecfffffa724e7bd9ecde5b001c49275cee86232df4ec70 SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-mini-memstick.img.xz) = e993796fa8d45171c41e6338083fc9a3cfd00cfc0dba90a129768e6ae958cf3930e5820a42fa275174e951cd14f3c0cf3acc1b439cf2d3e0e56d2b5e8e0f4044 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-bootonly.iso) = ea6cc6df52a265c2ed50d48363f00ab06b63d352a06aab5d70ef169285f35a94 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-bootonly.iso.xz) = c89925ccd1df086a3124cd4c83fc2b76e549c5853e4d4a5c9b04470f4f0445a4 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-disc1.iso) = 36fc92bf421f55453917b7be92bb5e1676da828d7ebb1d3c1ffaa274e1b6e32b SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-disc1.iso.xz) = 9a4f0b05c6691996e3f907a03414ba8f5c8f384671ddfdca3c35595bb5d90dcb SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-dvd1.iso) = 5eadf34065e51108237a01912c352b545bc24069c66ed9d995a9fa91f4178de6 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-dvd1.iso.xz) = bfc2ffb62ded44a6e668ec2db672bb102c3424cb640076ab2fd8ece1d6f07130 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-memstick.img) = 67d832f15eac687f5dd9aa05f5c32a1571d16d401f054506dbb3f9a0e27462ac SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-memstick.img.xz) = 3b1ecefecd189c501924e52da83e54f794bb128afaf3d1f641788685bb03b848 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-mini-memstick.img) = 99ad8bec87c55e2fd7e950ba8e03e89b3017d34c22cc02a9ac3a99fef75f9ea4 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-mini-memstick.img.xz) = 0e95974796aa45bc6d9bbee836a086a514e350b020172f567469a433b85994ca o 13.0-RC3 aarch64 RPI: SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-RPI.img.xz) = 22f6cacbc25ca0861aa434337afda85fdcc1e2ab707298b5058a4d57345b7f17808663b863aef263f3a5fc22fabb95442a39e9605fbd2a348a46179cf08175f7 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-RPI.img.xz) = 75baf7655ae8690918b2813062e9a5b2ca60f9e19df3a874724110f0546bde2f o 13.0-RC3 aarch64 PINE64: SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-PINE64.img.xz) = 362c58670c598ebefcd64fbdca4e8af9490e6244db4dd0f22fac6c2a3b3131be9e451b36fcb70a1544c22df240a2f721cd77cdd99f1c323ad672d0bb95952e4c SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-PINE64.img.xz) = 9f90f695921e922fb6f73a22c287dda9fa1e991fea8389a54633010b7a40b160 o 13.0-RC3 aarch64 PINE64-LTS: SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-PINE64-LTS.img.xz) = dbd28b0cabaf969c13da611ee11960ef5cb49383bdeda5b5187235eb1b261e03b67cc565692c3c553ed60ddbf1f43f0f641837bf97c880adeda548fa5f80a0ad SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-PINE64-LTS.img.xz) = 17fc09dfbdf36901bae32f8959373b9f77706f47420cfd1a2f51c6812f36800a o 13.0-RC3 aarch64 PINEBOOK: SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-PINEBOOK.img.xz) = f5121e29d5f31edcb81edc2598786c9791d87038f8e189dcdbed2cc3286b2c0c8d2fd5305300a868556f2b8b579ed2bd5a6c8b71ae6f2ea0ae79032dc4ed14ca SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-PINEBOOK.img.xz) = 580647b43fb156bbf791c7d7f1dc07759adb090f79f72b31ae0849ea83577e80 o 13.0-RC3 aarch64 ROCK64: SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-ROCK64.img.xz) = 88fac80fd941a073c39aa0f19c166954aff0e9344ae675d2c6d68a476a7795ac9d2c757d1afb95817816f0a71e4cb57fb51024f942092c49e247b6ecf87d0057 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-ROCK64.img.xz) = fdab2d985630f5a1c8cf7c32286cfe586dd8e9d88c66039946d492376f4cb7ef o 13.0-RC3 aarch64 ROCKPRO64: SHA512 (FreeBSD-13.0-RC3-arm64-aarch64-ROCKPRO64.img.xz) = c789f1cbe903afc21564ab31f4bdaf041abc30765d5040ca587cf2388b2843dd34ff3c97ed94c26dac85a4a9cdc68ceca75ca12a7d9551a862788e63b682ea43 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64-ROCKPRO64.img.xz) = 2cc839203a3e4171c4ccde06c711288c89fdc906d60a87795b7f7916612985a4 o 13.0-RC3 riscv64 GENERIC: SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-bootonly.iso) = 0b02d66d0c0a43af7e4e003be6c50f402f07ad030c979f5bb716281aea08d45882af3c586e57a1a4a0e58a4e8f15de1491deee103cb8493b10d811941ddd0306 SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-bootonly.iso.xz) = 4813aefae405cd4e048550c1cacd81568326d6af3945251796d80e27231df8c9c9a1dffb187354bb2736f2b2cc2cb27775504af69cf38a1a3ba7d5571704edf2 SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-disc1.iso) = ea679f983ec9d85ddd984aadd68b379518b6a1f8f931259307386324c7984a9b0239841e2ebbd5d04df95ed6dba45ced961326e9283150c706ad6938714a9f5a SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-disc1.iso.xz) = 76201d8ccd6fe830fbda53917ce458e83f216465410abf2c0add88a9da17eded78013195b579bd6a594c8af939bc551f594db86fbace1a52e63aa845872ff2fc SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-dvd1.iso) = 59fd823239663c9af4f75a26b4aeea193fb5fce79f773feb1d66ed3dd2e0841052a88659e8c3b9f646a6231d50065de468cba0cac971f7f9109e5ffb93cba203 SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-dvd1.iso.xz) = 0b69deec5bbbe756a9fc26a785132dccc9e19cdfde14eff182f37afccf9af8a7277fc771f261604c9c57f4111994c22bbc1c320418a42d95ee1ccccbf792aeb9 SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-memstick.img) = 595760225319f8de599861d319b4db2d93321072c69fb44649498b9ee4258534b1c1433d766c995b6ceb44f5b3ef189b372f685da0f7866d28467d9d772e6511 SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-memstick.img.xz) = decb9311c8d5c146403acf9aa0ffcbd71014cabf890d36a0e0784b2470d690d62465aa6971230cd6270c4c282c16944da90b8c02ba7c695d63168dbf656dc09b SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-mini-memstick.img) = 3db1702bc86ec3934b40889a3b52bd8dc5dd030c69ce96836941e3e2c6cffe755f28a5455cf277c47fe5d86ddb693c6b3c9f2726dd468bf47788ef0970844b72 SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-mini-memstick.img.xz) = 8711aba0c2923de7ed5d1e28f18372e2c89d0a93beec18453907bd877c177e37c539515e373e23c4eec6a56be9bef54f1019baca98d32611f793a55c2f49c24c SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-bootonly.iso) = 5809f78a48ac7d29d1547e95a6cd43bb58666c293dae273ead5f6f336f832f5e SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-bootonly.iso.xz) = 53f38657df7f4de4f1493d54ed31a4996bde294ade0d3346cfbd628a1db676d2 SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-disc1.iso) = ccb710957bd6b93e1adc4ab66fd3a130ed13c7ca09ba59361448a69012b1813f SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-disc1.iso.xz) = 755ab3fe1fa7ccd06f1fb0e753f82127f983f41c00fbd22f53979b3795d12ba5 SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-dvd1.iso) = b7b2836c3d6884ac7d32095724e33fec7584297537df69745a178f44b0097ceb SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-dvd1.iso.xz) = 964e21fa42e8a242edf5907116f0cdd0569edcabcbbdfc6253e70fcc6e121a2e SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-memstick.img) = 3f5ad3a0f2b251fd26884dbbb62bc235db0a168982a1afe9372eb465b8646dca SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-memstick.img.xz) = e4e5ee26ce083db674e036b55c0d9b2ad6bbcd159e64e81afc7021d7ddaf770a SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-mini-memstick.img) = 48bdd9f3afd074fc682e618d1cd42a3e2437a4484b6990bad0922a6a8e32e2e5 SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-mini-memstick.img.xz) = 8d5b8d052301cb897e82eaed1bd0fa51ae53a26003a20ddfc31da1459512ad41 o 13.0-RC3 riscv64 GENERICSD: SHA512 (FreeBSD-13.0-RC3-riscv-riscv64-GENERICSD.img.xz) = 1574597cccceedfb207fc9b9ab76542d40d4bcab7f24cb9400d51c6fb9d935ac5bbf09af556e0e32989d6b556669f4c2607a15261e8c383424f54f0f577ca0b9 SHA256 (FreeBSD-13.0-RC3-riscv-riscv64-GENERICSD.img.xz) = 1e05ea2e8add4cce3331a8072e0e9a6b8c9f69e727f2641b05aad96109b44baf == VM IMAGE CHECKSUMS == o 13.0-RC3 amd64: SHA512 (FreeBSD-13.0-RC3-amd64.qcow2.xz) = 85341a74163a18034ed9982431d551d151312002b9d2c046fe787c1e084bd58a9748262636e8b537626483acd99c4c1c86ea7b77b9dae190919dcb0831709f42 SHA512 (FreeBSD-13.0-RC3-amd64.raw.xz) = df35e633e1b3c431f16236b90048ff2b1edda544a51238cfdacd056f3cf739e7b25dddf31fb2cbc879646c8fcb8feb6e433ecd483ff71ab0294900f9eba127b7 SHA512 (FreeBSD-13.0-RC3-amd64.vhd.xz) = f28716d6638240ffeabff00188449edc8a453041413fd6cc44146ccedbdf2e70d7fa6666eafbffd036c3a4c4c7223ab4e7bc994ec0bb8af1976d2fbabedd919f SHA512 (FreeBSD-13.0-RC3-amd64.vmdk.xz) = 634d6becdfdfffd60ad94825e1ce64459ae3ee32873e25b97def0482d8e4617135a13e0011e1ae336e1f13793db2e2fb424a11ca3013836d81fb9ce245e34ac6 SHA256 (FreeBSD-13.0-RC3-amd64.qcow2.xz) = 1e393b2397cae0b4c942c49cb859f85869bbfb3f7e5a4d2d77e78932cc223278 SHA256 (FreeBSD-13.0-RC3-amd64.raw.xz) = 1335f4d6674de43330c1a54d09efb3ac59219c8950d429a2a86b1e88d3816bfa SHA256 (FreeBSD-13.0-RC3-amd64.vhd.xz) = 9bf03b7276892a3e015dd384bee9f9aa610a6018a8dd15cb5178399ac7d8f151 SHA256 (FreeBSD-13.0-RC3-amd64.vmdk.xz) = 3bfee44fb4aca49f5b40ef678b60acfab14310cd30bfaeee8f39e0a14692339e o 13.0-RC3 i386: SHA512 (FreeBSD-13.0-RC3-i386.qcow2.xz) = d424d8a26e408590a4b01b52fd813b605e39fb729130c38f4d99cfcd3ea673f544ad4d248ec8837a45afdf61c9b6887cdf72e7d0ca7635e47a3771acac2b3cc1 SHA512 (FreeBSD-13.0-RC3-i386.raw.xz) = 23bd63d52e726fcdd57352e496ca5c28af3501e502dc68582c699da8e7b44d0a3e4cea3482730a02a9ba511f9e3f38e7725cb20e215f4e7910882adf4f9ee305 SHA512 (FreeBSD-13.0-RC3-i386.vhd.xz) = 4b59f32f062a71cc27143ea9ab0651f7c4c8c42a3390758b814e886832da473d902b7c6fa57995a54b5e888250d937f687bc55e0bdb653ea4b2660d456477597 SHA512 (FreeBSD-13.0-RC3-i386.vmdk.xz) = 1fa6650ac5f1d87757bbb405b0f5e86a82755b0ee48b1d91d36ea546ea3f963fba43cea47d1f7543a898ce48671e27062263af07909407777f3c33609890d19c SHA256 (FreeBSD-13.0-RC3-i386.qcow2.xz) = ef1d26dcdecb32cc23a616847b0f99eb4bd1784fc33d3bd5efea4556b161bb43 SHA256 (FreeBSD-13.0-RC3-i386.raw.xz) = 405440c1ad1070749e2b77a4c2d2ef2d2aab4bf87c54d3296b1c722a06d2de4e SHA256 (FreeBSD-13.0-RC3-i386.vhd.xz) = 287e0ea5f9597a15cf587020c22326932a2f1ddc0d81c3f101a64b5058985466 SHA256 (FreeBSD-13.0-RC3-i386.vmdk.xz) = b2ecd761ceaaae62064fa03a10cf0a9487a1706eb946d9d1f27ff9f5b2ae7942 o 13.0-RC3 aarch64: SHA512 (FreeBSD-13.0-RC3-arm64-aarch64.qcow2.xz) = a8a4a557b121869bf73d096230b6659709d6f53b4762d30ee7f60245d341550d9e44c9049b93ec3a284a5b9aed07ad23b3922274dfcefdef8947271172cf2821 SHA512 (FreeBSD-13.0-RC3-arm64-aarch64.raw.xz) = 5c285e4f677609cc6abd17bf748135f1d8ffeb1cdd4381de210e9c5d0c81ebc6a9c691ae888a3735c1fcf0e735f893b45ee152c13df8003f7c4711b14822fa8c SHA512 (FreeBSD-13.0-RC3-arm64-aarch64.vhd.xz) = 342d5219dfe82b53a1bea1635c3a0c11ee84e37eebf948a75c6a43d9c26fa5ad5bed07bef42f8dbfe043832358d86df104526fb48317c97a487b445cdd6c30ca SHA512 (FreeBSD-13.0-RC3-arm64-aarch64.vmdk.xz) = 6a794f981f4d4076ec05c9dd8a853f28726003e452ed1f7dc341d1aba462e1b0f23767387331714fb80e567e80fc167accd401c36a588aa08ac8d5885dde331a SHA256 (FreeBSD-13.0-RC3-arm64-aarch64.qcow2.xz) = f0b064ee2f638dbb03bab5df0f13ddfb9755861e4c301667aff0606335337936 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64.raw.xz) = ec3b751a4d994c1bd89dfabad03e3154be744a3d9667b65a09ffd9125451f285 SHA256 (FreeBSD-13.0-RC3-arm64-aarch64.vhd.xz) = 9ea7117736a7e9db12992984199b02aaa72013eea8ec2737058e37605a82c91f SHA256 (FreeBSD-13.0-RC3-arm64-aarch64.vmdk.xz) = e5256d4dd05b22d61803bfe0580580e2c0a953731bed92f8930ec3e29f22c4a6 o 13.0-RC3 riscv64: SHA512 (FreeBSD-13.0-RC3-riscv-riscv64.qcow2.xz) = 76742cb8e1ac50b4ac60fde9593ab7e2e07d0406020496a6c184d185b3a19d96748668bc21c26f976c8c18cea102aba56dd3c4e6388ae4c29f5f5baf175c88de SHA512 (FreeBSD-13.0-RC3-riscv-riscv64.raw.xz) = 13bf67cce78e7c42f4aa6cf9fbfce6dbcba1b5b2191e71e7bac43c831a627a356d8d2f6ec9309fa7d6ed982845ef9158f62a3eca6c96ad98bf1267850d76fd4f SHA512 (FreeBSD-13.0-RC3-riscv-riscv64.vhd.xz) = 0a4a99e4b88a68d1c964ebe68facf9a92072538d6da14cf98d599cf0174ae2ba31b100b4e8708438c0ffeb6f9a43823b63bc79394569575975722a2fafeed556 SHA512 (FreeBSD-13.0-RC3-riscv-riscv64.vmdk.xz) = f1793ba35d5a8f73b76cb2ae6a85ac5c8149ee6ad500889d22c5a2eadd9b083491b0a6eef89c1ece43ee11d27f586fa57ece15ef7f2ebd675b61745aaf0227e3 SHA256 (FreeBSD-13.0-RC3-riscv-riscv64.qcow2.xz) = 65fdee698acf127e18b996cdec9570c443afb0f5c40048d6352de0832e583ebb SHA256 (FreeBSD-13.0-RC3-riscv-riscv64.raw.xz) = 10b51deb348b7fe75a3efdab0bc3e341d4c99189176d3c401fc8a8e20f4903f1 SHA256 (FreeBSD-13.0-RC3-riscv-riscv64.vhd.xz) = e47edf3ca4a9ef0c639acfc522559efbe2c146b187bbd6638930fdb852320528 SHA256 (FreeBSD-13.0-RC3-riscv-riscv64.vmdk.xz) = 598b1b6a2fff8e7513961bfed580573533c235b4c42b5e15161bc1ac33a14651 o 13.0-RC3 amd64 BASIC-CI: SHA512 (FreeBSD-13.0-RC3-amd64-BASIC-CI.raw.xz) = 6a79035a220ef35098953969a8de9cf311c38446e43fec03e121c8e79aff46a8f48bf0184a73c578b00fd801bb4d1aa65652b6799e6d6a59a7025b281c5c3f4d SHA256 (FreeBSD-13.0-RC3-amd64-BASIC-CI.raw.xz) = e93a6d4c8c14c893d7d08abaa584d979ffe4dc06c2eddc586970caf0b52ab86f Regards, Glen Love FreeBSD? Support this and future releases with a donation to the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmBWHhUACgkQAxRYpUeP 4pMdVBAAjFjfzlKn2UaNifLhS/c5oXkZ2nAnd/A3PP5J4BAEURt997vxVVlfQlxF 7iaQCxUiQTtxUGNRnPZDjAu29D/vJN9Jd5rox1hn0nTlDZFVrewJxzBh1lE177Cg 4WTwkc6WslRYxrSdJ127RsiEsiEs0Se7pCdLPJXH5sQtddHb8EP/30btlsirmYqe osywQIacDh2d9mJRWreZuMBKOCFgYKVxye/ze/PEWX3Q1F5duv/igJv0XaXvZ+Vf lLOOYEYqvaVPu+lrLI6eSJEVU3iSjcHpQOAkz9kP5jIpJaXmUdL8pg/IWD/l7jFg AZslB2fKNwNcmvn+i2Q8djKi3bjLo+8WrF09Sj9St9XyBDEw2raWwHQT4lIdmmhk tqiRFYaVn4WBASJfjAImxB3ukE+qTwVbEAftYHL4w/Qoot7Pc2RPvY7dNxTgwA3l tzyX2NjPpEdOBmPlAlsML/FTpZzJfWiXo6pUNi0KRCMSNTN/bJ2BZWrphvErJTU4 j7yMhpvS5jvZzYAkYYX112C+ezRDNY7ywNnHtmxRGfhL0mCEvr5CvJvmXnzLYCgc jfvz/5/Fj0hjWQ9hdP24m++rz7z1T17Nhlywv5D5V5OSD9XOuRwBpf1jCz3TchZe Q3gAUx4bqO1fQhN6h3G4etuaJGbsQWnSMiof09L9MjNjp5of2ew= =YzoF -----END PGP SIGNATURE----- From owner-freebsd-stable@freebsd.org Sat Mar 20 16:26:23 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C232D5AE094 for ; Sat, 20 Mar 2021 16:26:23 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com [IPv6:2607:f8b0:4864:20::32d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2mNZ6dmXz4jgP for ; Sat, 20 Mar 2021 16:26:22 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ot1-x32d.google.com with SMTP id w31-20020a9d36220000b02901f2cbfc9743so11285021otb.7 for ; Sat, 20 Mar 2021 09:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=alsfr4l1ualM+aDM9jbwMVMV7dKoZVS2baePKKKtEL4=; b=YBV2/nzxNI0rx9OuWgtma8OrHkB8MTU0mbzOnn2/V87k6cnIOa7RH09k8SA8zLlouL Jhoa+ZULEUntINqHmPrYy6+FSGpCWgn9iY1smecYB84fsjeTnBNi6ReRgVR3+BgIyFEB dx4lLV9zpmrQNyh7xO2hq4bpemOC3Ki7DEj4K1h+KT+QWn4PoD6eHywMO9HBO2vx3Xd/ 6irng2WNd68ecUaXf4I0nf8y8ztxPdJQFrN/z4wjiWBn2hsCuTY/P3YTQLMwKy3K0XrO 522iguFPG69m8z8B36ZQInLZ1yb3vP8axBjSYuz1Ci+qe0bBV09+h+/ApRHm++1Ch2S+ jFsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=alsfr4l1ualM+aDM9jbwMVMV7dKoZVS2baePKKKtEL4=; b=GGsGKF3zjgCdOpn0qPRv9UDHgRXbdmmTWZ9zrEOfqO7T8Y/gqnos+Ig+FiMHHaxFxs 9L4FzER/255ozhC2beH5ULesT/rqRyr195eN1yoBXhudl2snva/ALdTj8MpT6ZZj7+it p+7l6NiMwuXZx2oIcQ5gx/kJYhtcTD3NCrM6COoCNbe5aM//p/w/uBMxh9bbsEM62Eyw f7s8jxYMnlFKBiKb3YKY19obVTLLZFde8wxXG4L9kKvf64lovwXKxjPTT+jMyvj0y4w/ tj7RLkL0OWyzgENNVhk+zNBvR6Q5VBb7S5LdOmilAu/1er4+XRSJbeWWbOnkuCYGc7Ow CQuA== X-Gm-Message-State: AOAM531f1C+pnZ35fJlU6DGVHJ0JERngTK7dFaoTM+XaMHvvTEZ+L8td GIg/2ZEf+AQzUiShSZoScP3NzIsagx+rmWDu1+rSu5cYJKs= X-Google-Smtp-Source: ABdhPJyvp0uBABBIhkFsCzmKmEsie9p5Tt/c2LuhqHYHynkwj9RzvuX1EKC6a6Qn/yTUapjbyLdoChAzX9sWUA/tUNY= X-Received: by 2002:a9d:7453:: with SMTP id p19mr634555otk.271.1616257581926; Sat, 20 Mar 2021 09:26:21 -0700 (PDT) MIME-Version: 1.0 References: <1867238560.3032633.1616178410076.ref@mail.yahoo.com> <1867238560.3032633.1616178410076@mail.yahoo.com> <86czvud1qf.fsf@virtual-earth.de> In-Reply-To: From: Kevin Oberman Date: Sat, 20 Mar 2021 09:26:06 -0700 Message-ID: Subject: Re: ThinkPad X1 Carbon Gen 4 unable to boot 13.0-RC1 To: Fred Cc: FreeBSD-STABLE Mailing List X-Rspamd-Queue-Id: 4F2mNZ6dmXz4jgP X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=YBV2/nzx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of kob6558@gmail.com designates 2607:f8b0:4864:20::32d as permitted sender) smtp.mailfrom=kob6558@gmail.com X-Spamd-Result: default: False [-3.69 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.987]; FREEMAIL_TO(0.00)[yahoo.com]; FORGED_SENDER(0.30)[rkoberman@gmail.com,kob6558@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::32d:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[rkoberman@gmail.com,kob6558@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::32d:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::32d:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 16:26:23 -0000 On Sat, Mar 20, 2021 at 8:35 AM Fred via freebsd-stable < freebsd-stable@freebsd.org> wrote: > On 3/19/21 7:59 PM, Mathias Picker wrote: > > > > Fred Hall via freebsd-stable writes: > > > >> I have a ThinkPad X1 Carbon Gen 4 (20FCS0NB00) which has happily run > >> FreeBSD 11 and 12, but which can't boot 13.0-RC1 from memstick or via > >> freebsd-update. In both cases the boot process locks up on the line > >> "hwpstate_intel0: on cpu0" > >> If running freebsd-update, a work around is to add > >> hint.hwpstate_intel.0.disabled=3D"1" in loader.conf. See the note unde= r > >> the Eighth Generation (2020) in > >> https://wiki.freebsd.org/Laptops/Thinkpad_X1_Carbon > >> I was quite surprised to find the lack of support for hwpstate_intel > >> in 13 when it apparently worked under 11 and 12. Does anyone know the > >> status of hwpstate_intel on ThinkPads? > > > > I=E2=80=99m running 13 -STABLE from 10 days ago on a Thinkpad Yoga 3rd = gen, and > > hwpstate_intel works fine, never had a problem. > > > > mathiasp:~% sysctl dev.hwpstate_intel dev.hwpstate_intel.7.epp: 15 > > dev.hwpstate_intel.7.%parent: cpu7 > > dev.hwpstate_intel.7.%pnpinfo: dev.hwpstate_intel.7.%location: > > dev.hwpstate_intel.7.%driver: hwpstate_intel > > dev.hwpstate_intel.7.%desc: Intel Speed Shift > > dev.hwpstate_intel.6.epp: 15 > > dev.hwpstate_intel.6.%parent: cpu6 > > dev.hwpstate_intel.6.%pnpinfo: dev.hwpstate_intel.6.%location: > > dev.hwpstate_intel.6.%driver: hwpstate_intel > > dev.hwpstate_intel.6.%desc: Intel Speed Shift > > [snip] > > > > The gen3 is using > > sudo dmesg|grep -i cpu > > CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (1992.08-MHz K8-class CPU= ) > > [snip, snip] > > > > mathiasp:~% uname -a > > FreeBSD Danton 13.0-STABLE FreeBSD 13.0-STABLE #2 > > stable/13-n244845-f21c0366f53: Wed Mar 10 20:53:26 CET 2021 > > root@Danton:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 > > > > > > Cheers, > > > > Mathias > > Thanks for the feed back. Good to know most people won't encounter the > problem. Perhaps it is a bios issue specific to the model. I did update > to the latest bios version but that made no difference. > > I have chosen to rollback to 12.2 as it works perfectly for me. > > > > >> Cheers, Fred > There are two long tickets about this. Take a look at tickets 248659 and 253288 . This problem appeared in 13-current in Jan 2020 and I first saw it on my new Lenovo L15 that summer. It appears specific to Lenovo laptops. It appears that similar issues have been seen with Linux. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Sat Mar 20 16:27:44 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 84B7E5ADFB2 for ; Sat, 20 Mar 2021 16:27:44 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4F2mQ83L1zz4jps for ; Sat, 20 Mar 2021 16:27:44 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id 70AEE5ADF57; Sat, 20 Mar 2021 16:27:44 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 707485AE1A2 for ; Sat, 20 Mar 2021 16:27:44 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from relay12.mail.gandi.net (relay12.mail.gandi.net [217.70.178.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2mQ811zNz4jhZ for ; Sat, 20 Mar 2021 16:27:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from [192.168.0.88] (unknown [195.64.148.76]) (Authenticated sender: andriy.gapon@uabsd.com) by relay12.mail.gandi.net (Postfix) with ESMTPSA id 1C660200003; Sat, 20 Mar 2021 16:27:39 +0000 (UTC) Subject: Re: kldload zfs spins the system after upgrading from 12.2 to 13-BETA To: Yoshihiro Ota Cc: stable@freebsd.org References: <20210306130913.dae1fb546e68bcaec882cdbe@j.email.ne.jp> <9f9db150-ee5d-e74e-7f24-74d1fa687a48@FreeBSD.org> <20210307222441.02755e7396cd329edd15df15@j.email.ne.jp> <20210319230112.e327d1c69197b73244fd89d6@j.email.ne.jp> From: Andriy Gapon Message-ID: <8b11fad4-5b6e-cf4a-0cde-8a8feff9705e@FreeBSD.org> Date: Sat, 20 Mar 2021 18:27:38 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <20210319230112.e327d1c69197b73244fd89d6@j.email.ne.jp> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4F2mQ811zNz4jhZ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 16:27:44 -0000 On 20/03/2021 05:01, Yoshihiro Ota wrote: > On Tue, 9 Mar 2021 11:24:53 +0200 > Andriy Gapon wrote: > >> On 08/03/2021 05:24, Yoshihiro Ota wrote: >>> On Sun, 7 Mar 2021 00:09:33 +0200 >>> Andriy Gapon wrote: >>> >>>> On 06/03/2021 20:09, Yoshihiro Ota wrote: >>>>> Hi all, >>>>> >>>>> I'm upgrading fron 12.2-RELEASE to 13-BETA/RC one by one. >>>>> >>>>> After upgrading one in VMWare, 'zfs mount -a' hangs the system. >>>>> I don't have boottime zfs mount on nor don't have zfsroot. >>>>> I just simply ran install world/kernel and mergemaster. >>>> >>>> Please use procstat -kk to capture a kernel stack trace of the hung process. >>> >>> Actually, spining was 'kldload zfs'. >>> Console doesn't response but ping and sshd sessions still work. >>> procstat output is below. >>> In addition, this doesn't happen to systems that I've been following 13-CURRENT >>> but rather happen only wiht a system upgraded from 12.2-RELEASE to 13-RC. >>> >>> >>> # procstat -kk 1049 >>> PID TID COMM TDNAME KSTACK >>> 1049 100215 kldload - spa_init+0xc6 zfs_kmod_init+0x1a >>> zfs_modevent+0x34 module_register_init+0x8c linker_load_module+0xaab kern_kldload+0xc1 >>> sys_kldload+0x50 syscall+0x17d g_ctx+0xe280bf29 >>> >> >> If you could use kgdb to find out what source code line spa_init+0xc6 >> corresponds to that may help to see what's going on. >> > > It look me a while to get kgdb working properly. > At last, I got the output. > It looks it is spining on a mutex. > > I have few other machines run the same kernel but they can load zfs.ko. > It is only vmware VM that spins with 'kldload zfs'. > > vmware# kgdb101 /usr/usr/lib/debug/boot/kernel/zfs.ko.debug > GNU gdb (GDB) 10.1 [GDB v10.1 for FreeBSD] > Copyright (C) 2020 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > Type "show copying" and "show warranty" for details. > This GDB was configured as "i386-portbld-freebsd13.0". > Type "show configuration" for configuration details. > For bug reporting instructions, please see: > . > Find the GDB manual and other documentation resources online at: > . > > For help, type "helpType "apropos word" to search for commands related to "word"... > Reading symbols from zfs.ko.debug... > (kgdb) info line *spa_init+0xc6 > Line 2345 of "/usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c" > starts at address 0x2b0461 > and ends at 0x2b0467 . > (kgdb) > > void > spa_init(spa_mode_t mode) > { > mutex_init(&spa_namespace_lock, NULL, MUTEX_DEFAULT, NULL); > mutex_init(&spa_spare_lock, NULL, MUTEX_DEFAULT, NULL); // <- line 2345 > It would highly unusual (and unexplainable) for a thread to get stuck here. Are you sure that your source tree matches the binary? Can you disassemble the function to check instructions at and near 0xc6 offset? -- Andriy Gapon From owner-freebsd-stable@freebsd.org Sat Mar 20 22:38:04 2021 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B70715B607A for ; Sat, 20 Mar 2021 22:38:04 +0000 (UTC) (envelope-from freebsd-stable@m.gmane-mx.org) Received: from ciao.gmane.io (ciao.gmane.io [116.202.254.214]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2wdR73qLz3Ks3 for ; Sat, 20 Mar 2021 22:38:03 +0000 (UTC) (envelope-from freebsd-stable@m.gmane-mx.org) Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1lNkEM-00083t-2H for freebsd-stable@freebsd.org; Sat, 20 Mar 2021 23:38:02 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-stable@freebsd.org From: Fred Subject: Re: ThinkPad X1 Carbon Gen 4 unable to boot 13.0-RC1 Date: Sat, 20 Mar 2021 16:37:55 -0600 Message-ID: References: <1867238560.3032633.1616178410076.ref@mail.yahoo.com> <1867238560.3032633.1616178410076@mail.yahoo.com> <86czvud1qf.fsf@virtual-earth.de> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 In-Reply-To: Content-Language: en-US X-Rspamd-Queue-Id: 4F2wdR73qLz3Ks3 X-Spamd-Bar: +++++ X-Spamd-Result: default: False [5.10 / 15.00]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; ARC_NA(0.00)[]; GREYLIST(0.00)[pass,meta]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[116.202.254.214:from]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_MEDIUM(0.99)[0.991]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[116.202.254.214:from:127.0.2.255]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_SPAM_LONG(0.10)[0.105]; R_SPF_ALLOW(-0.20)[+mx]; DMARC_POLICY_REJECT(2.00)[yahoo.com : SPF not aligned (relaxed), No valid DKIM,reject]; FORGED_SENDER(0.30)[fred.ha11@yahoo.com,freebsd-stable@m.gmane-mx.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:116.202.0.0/16, country:DE]; FROM_NEQ_ENVFROM(0.00)[fred.ha11@yahoo.com,freebsd-stable@m.gmane-mx.org]; MAILMAN_DEST(0.00)[freebsd-stable]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2021 22:38:04 -0000 On 3/20/21 10:26 AM, Kevin Oberman wrote: > On Sat, Mar 20, 2021 at 8:35 AM Fred via freebsd-stable < > freebsd-stable@freebsd.org> wrote: > >> On 3/19/21 7:59 PM, Mathias Picker wrote: >>> >>> Fred Hall via freebsd-stable writes: >>> >>>> I have a ThinkPad X1 Carbon Gen 4 (20FCS0NB00) which has happily run >>>> FreeBSD 11 and 12, but which can't boot 13.0-RC1 from memstick or via >>>> freebsd-update. In both cases the boot process locks up on the line >>>> "hwpstate_intel0: on cpu0" >>>> If running freebsd-update, a work around is to add >>>> hint.hwpstate_intel.0.disabled="1" in loader.conf. See the note under >>>> the Eighth Generation (2020) in >>>> https://wiki.freebsd.org/Laptops/Thinkpad_X1_Carbon >>>> I was quite surprised to find the lack of support for hwpstate_intel >>>> in 13 when it apparently worked under 11 and 12. Does anyone know the >>>> status of hwpstate_intel on ThinkPads? >>> >>> I’m running 13 -STABLE from 10 days ago on a Thinkpad Yoga 3rd gen, and >>> hwpstate_intel works fine, never had a problem. >>> >>> mathiasp:~% sysctl dev.hwpstate_intel dev.hwpstate_intel.7.epp: 15 >>> dev.hwpstate_intel.7.%parent: cpu7 >>> dev.hwpstate_intel.7.%pnpinfo: dev.hwpstate_intel.7.%location: >>> dev.hwpstate_intel.7.%driver: hwpstate_intel >>> dev.hwpstate_intel.7.%desc: Intel Speed Shift >>> dev.hwpstate_intel.6.epp: 15 >>> dev.hwpstate_intel.6.%parent: cpu6 >>> dev.hwpstate_intel.6.%pnpinfo: dev.hwpstate_intel.6.%location: >>> dev.hwpstate_intel.6.%driver: hwpstate_intel >>> dev.hwpstate_intel.6.%desc: Intel Speed Shift >>> [snip] >>> >>> The gen3 is using >>> sudo dmesg|grep -i c >>> CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (1992.08-MHz K8-class CPU) >>> [snip, snip] >>> >>> mathiasp:~% uname -a >>> FreeBSD Danton 13.0-STABLE FreeBSD 13.0-STABLE #2 >>> stable/13-n244845-f21c0366f53: Wed Mar 10 20:53:26 CET 2021 >>> root@Danton:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 >>> >>> >>> Cheers, >>> >>> Mathias >> >> Thanks for the feed back. Good to know most people won't encounter the >> problem. Perhaps it is a bios issue specific to the model. I did update >> to the latest bios version but that made no difference. >> >> I have chosen to rollback to 12.2 as it works perfectly for me. >> >>> >>>> Cheers, Fred >> > There are two long tickets about this. Take a look at tickets 248659 > and 253288 > . This problem > appeared in 13-current in Jan 2020 and I first saw it on my new Lenovo L15 > that summer. It appears specific to Lenovo laptops. It appears that similar > issues have been seen with Linux. Thank for the links to the bug reports, it would appear to be the same issue. I tested my wife's X1 Carbon Gen 3 and it worked fine. Perhaps a bios bug with the processor in my 4th gen. CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz (2808.11-MHz K8-class CPU) In the 24 hours I tested 13.0, I also had some X windows failures while waking up from suspend which never happens with 12.2. Oh well, no worries. > -- > Kevin Oberman, Part time kid herder and retired Network Engineer > E-mail: rkoberman@gmail.com > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >