From nobody Fri Sep 1 21:22:23 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RcrbJ4H4Pz4rh0K for ; Fri, 1 Sep 2023 21:22:36 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RcrbH63LGz4kZ8 for ; Fri, 1 Sep 2023 21:22:35 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b="d7G/m9vT"; spf=pass (mx1.freebsd.org: domain of rick.macklem@gmail.com designates 2607:f8b0:4864:20::42c as permitted sender) smtp.mailfrom=rick.macklem@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-68bec3a1c0fso2092117b3a.1 for ; Fri, 01 Sep 2023 14:22:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693603354; x=1694208154; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=NIDFk0GuWcnuwLN5juZM763g7ejPOWiXDh1wPkKD1iQ=; b=d7G/m9vT4T5xt32g8ir6Gu0Fjn2fVHHctlHZs3K7AtZcV5SGBahF9XpO8lK3KM9C/E RP66qRzuuhExoAohDjaXe4acmROUq969JnWDQeu6dHBOK81+oJ5Qwj4ZJMitsw+77skb hwQNWkWDBLi/fGhWWzRewvX3vLrdayubEUbHoxxW0M9y7d9xkMul+/EwfRAQV3qIiHHC duJ3ajkkkKnQW3lWpjOfXUoB1qYglpjCitQ0BM33oDYDsfnbwAs1poFxYdi4Kea6hrIT cDV08mGMhos5qD4jGGfQ+bipZHGUavRgpQA1vHstBIcRpsSlwDupxfLAForhE2/BfV2U TVdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693603354; x=1694208154; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NIDFk0GuWcnuwLN5juZM763g7ejPOWiXDh1wPkKD1iQ=; b=hkfUKgBSKYUNOTGTiDVzYXPGUreA6/9mCPKrNrDVuoOlsO6HbDoJPHj+6NKxwFfCY/ qmdBMUqdojUUJy7IdKWp/Lo6AUIO2t4C12gqlZzNTi98RSgUgxW9SNrpddmDIBMeGQzP 8AEDX3sCU7TdTGfFg4c9GfQwvW8Da+FiGtUhfbdY3TWbIiOQObfnFud1c/wj6uWF8Nu1 Gspj4/pSMBOiZz6cMmyTJJXLY/x0VInGSWct0X3vX4OycWGaN5UVyILaKhvLUdxQEviy uPAKTL26n5yXnzf8UECpGJaeWaSWwzApR4G/WAFQOxMTkkPaUOqbksC+Lzjn1dyOqRJP 8VpQ== X-Gm-Message-State: AOJu0YxbDXSD/u1QLAM/Dwukgndmm0kzongLRof2VWZHUEv10mFRWzpq 7i25jxQzTX846OW9Zy+aRfXupGQyE7BEf8qFenV8QYZGjA== X-Google-Smtp-Source: AGHT+IGDByQCBUaFVCkfKE4tyH9nYgb4SzN4ECBgg5s4tYzeaAUz1PCyiLqrnwQnWeuIEBWsMWfSISftTa97w0TMiAM= X-Received: by 2002:aa7:8881:0:b0:68b:c423:fb20 with SMTP id z1-20020aa78881000000b0068bc423fb20mr4279580pfe.30.1693603354080; Fri, 01 Sep 2023 14:22:34 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <25827.33600.611577.665054@hergotha.csail.mit.edu> <25831.30103.446606.733311@hergotha.csail.mit.edu> <25840.58487.468791.344785@hergotha.csail.mit.edu> In-Reply-To: <25840.58487.468791.344785@hergotha.csail.mit.edu> From: Rick Macklem Date: Fri, 1 Sep 2023 14:22:23 -0700 Message-ID: Subject: Re: Did something change with ZFS and vnode caching? To: Garrett Wollman Cc: freebsd-stable@freebsd.org, Mateusz Guzik Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: -- X-Spamd-Result: default: False [-3.00 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::42c:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TAGGED_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com] X-Rspamd-Queue-Id: 4RcrbH63LGz4kZ8 On Thu, Aug 31, 2023 at 12:05=E2=80=AFPM Garrett Wollman wrote: > > < said: > > > Any suggestions on what we should monitor or try to adjust? I remember you mentioning that you tried increasing kern.maxvnodes but I was wondering if you've tried bumping it way up (like 10X what it currently is)? You could try decreasing the max nfsd threads (--maxthreads command line option for nfsd. That would at least limit the # of vnodes used by the nfsd. rick > > To bring everyone up to speed: earlier this month we upgraded our NFS > servers from 12.4 to 13.2 and found that our backup system was > absolutely destroying NFS performance, which had not happened before. > > With some pointers from mjg@ and the thread relating to ZFS > performance on current@ I built a stable/13 kernel > (b5a5a06fc012d27c6937776bff8469ea465c3873) and installed it on one of > our NFS servers for testing, then removed the band-aid on our backup > system and allowed it to go as parallel as it wanted. > > Unfortunately, we do not control the scheduling of backup jobs, so > it's difficult to tell whether the changes made any difference. Each > backup job does a parallel breadth-first traversal of a given > filesystem, using as many as 150 threads per job (the backup client > auto-scales itself), and we sometimes see as many as eight jobs > running in parallel on one file server. (There are 17, soon to be 18, > file servers.) > > When the performance of NFS's backing store goes to hell, the NFS > server is not able to put back-pressure on the clients hard enough to > stop them from writing, and eventually the server runs out of 4k jumbo > mbufs and crashes. This at least is a known failure mode, going back > a decade. Before it gets to this point, the NFS server also > auto-scales itself, so it's in competition with the backup client over > who can create the most threads and ultimately allocate the most > vnodes. > > Last night, while I was watching, the first dozen or so backups went > fine, with no impact to NFS performance, until the backup server > decided to schedule scans of two, and then three, parallel scans of > filesystems containing about 35 million files each. These tend to > take an hour or four, depending on how much changed data is identified > during the scane, but most of the time it's just sitting in a > readdir()/fstatat() loop with a shared work queue for parallelism. > (That's my interpretation based on its activity; we do not have source > code.) > > Once these scans were underway, I observed the same symptoms as on > releng/13.2, with lots of lock contention and the vnlru process > running almost constantly (95% CPU, so most of a core on this > 20-core/40-thread server). From our monitoring, the server was > recycling about 35k vnodes per second during this period. I wasn't > monitoring these statistics before so I don't have historical > comparisons. My working assumption, such as it is, is that the switch > from OpnSolaris ZFS to OpenZFS in 13.x moved some bottlenecks around > so that the backup client previously got tangled higher up in the ZFS > code and now can put real pressure on the vnode allocator. > > During the hour that the three backup clients were running, I was able > to run mjg@'s dtrace script and generate a flame graph, which is > viewable at . > This just shows what the backup clients themselves are doing, and not > what's going on in the vnlru or nfsd processes. You can ignore all > the umtx stacks since that's just coordination between the threads in > the backup client. > > On the "oncpu" side, the trace captures a lot of time spent spinning > in lock_delay(), although I don't see where the alleged call site > acquires any locks, so there must have been some inlining. On the > "offcpu" side, it's clear that there's still a lot of time spent > sleeping on vnode_list_mtx in the vnode allocation pathway, both > directly from vn_alloc_hard() and also from vnlru_free_impl() after > the mutex is dropped and then needs to be reacquired. > > In ZFS, there's also a substantial number of waits (shown as > sx_xlock_hard stack frames), in both the easy case (a free vnode was > readily available) and the hard case where vn_alloc_hard() calls > vnlru_free_impl() and eventually zfs_inactive() to reclaim a vnode. > Looking into the implementation, I noted that ZFS uses a 64-entry hash > lock for this, and I'm wondering if there's an issue with false > sharing. Can anyone with ZFS experience speak to that? If I > increased ZFS_OBJ_MTX_SZ to 128 or 256, would it be likely to hurt > something else (other than memory usage)? Do we even know that the > low-order 6 bits of ZFS object IDs are actually uniformly distributed? > > -GAWollman > > From nobody Sun Sep 3 03:30:28 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RdcjY4KSRz4s019 for ; Sun, 3 Sep 2023 03:30:41 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RdcjW2qSLz3bsg for ; Sun, 3 Sep 2023 03:30:38 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 153.125.133.21) smtp.mailfrom=junchoon@dec.sakura.ne.jp; dmarc=none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 3833USV8053825 for ; Sun, 3 Sep 2023 12:30:28 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 3 Sep 2023 12:30:28 +0900 From: Tomoaki AOKI To: freebsd-stable@freebsd.org Subject: Is there any plan for ZFS and timerfd updates on stable/14? Message-Id: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: / X-Spamd-Result: default: False [0.56 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-0.99)[-0.990]; MV_CASE(0.50)[]; NEURAL_HAM_MEDIUM(-0.48)[-0.480]; NEURAL_HAM_SHORT(-0.47)[-0.473]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[sakura.ne.jp]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; TO_DN_NONE(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org] X-Rspamd-Queue-Id: 4RdcjW2qSLz3bsg Hi. There are discussions about deadlocks issue of ZFS on freebsd-current ML, starting from [1] last month. IIRC, at least some fixes (candidates?) are merged to main, but not yet to stable/14. Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems to have most of them. So my 1st question is "Is there any plan to import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING releng/14? And one more. timerfd is added at last-minutes BEFORE stable/14 is branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and Differential revision D41600 on Phablicator [5] related to memory leaks and locks. Additionally, splitting out lib32 part to proper place is proposed as D41640 [6]. Both [5] and [6] are accepted but not yet landed. Also, D41641 [7] proposes namespace pollution adjustments. This can be optional? Memory leaks and improper locks can lead system to security issues or deadlocks, so it would be benefical if landed and MFC'ed BEFORE releng/14 branches. Is there any plan to do so? At least, existing deadlocks should be considered as SHOW-STOPPER and resolved. I myself am bitten by several deadlocks on poudriere full builds after upgrading base from stable/13 to stable/14, finally finished with increasing kern.maxvnodes after powercycle on each deadlock and continue. Thanks in advance! [1] https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.html [2] https://cgit.freebsd.org/src/commit/?id=02f534b57f84d6f4f97c337b05b383c8b3aaf18c [3] https://cgit.freebsd.org/src/commit/?id=5eab523053db79b4bd4f926c7d7ac04444d9c1da [4] https://cgit.freebsd.org/src/commit/?id=f4296cfb409a48de00bfa60e76f686c2b031876f [5] https://reviews.freebsd.org/D41600 [6] https://reviews.freebsd.org/D41640 [7] https://reviews.freebsd.org/D41641 -- Tomoaki AOKI From nobody Sun Sep 3 03:36:18 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RdcrL40jgz4s1Gn for ; Sun, 3 Sep 2023 03:36:34 +0000 (UTC) (envelope-from jake@technologyfriends.net) Received: from mail-lj1-x22e.google.com (mail-lj1-x22e.google.com [IPv6:2a00:1450:4864:20::22e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RdcrL20Nwz3dbm for ; Sun, 3 Sep 2023 03:36:34 +0000 (UTC) (envelope-from jake@technologyfriends.net) Authentication-Results: mx1.freebsd.org; none Received: by mail-lj1-x22e.google.com with SMTP id 38308e7fff4ca-2b9d07a8d84so4126921fa.3 for ; Sat, 02 Sep 2023 20:36:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=technologyfriends.net; s=google; t=1693712190; x=1694316990; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=AhUGtu6gQyKXTX1Y/hISoFlcp8IUC44H2u2XRVhQRvs=; b=MJMeGF5OOmQ2+ZFtpZIP+I6Pw2IxzOIqjNmuR0xBpGJZIdlj/vJNWIdjcMPDHhGS39 YGV3F1q4QYpYasJUxJ31/Qdx36oFJKzH8GReva8+76yYOnv//APmDWsa48rQo59/B6ZX /e7KxADAOajxEz2Yubp7V4kt1l0XsbAX5vdUzrNxmBZJuhEJR8zGbznFlPIDbGIZ3CHG A4qCSIvTzml7zdULW2sAmWaKzPcBQPS3Rkcde7ZI3H9v2pZPFKtlzNi+XYgkaFGz/uIe GZ8ZpoMtLW6vkm+dLTgljWZjH8ljoQ7IjvK6nO/b1Y7i5S24/VI1QXH+HC9g1aIVZPRc hqhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693712190; x=1694316990; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AhUGtu6gQyKXTX1Y/hISoFlcp8IUC44H2u2XRVhQRvs=; b=NKzv8tOll0mWI6NWIyh+W6kauRLsF8D2kMuh6oBeSw6HCj5WADcUHJsARjknmsXAP/ pJIzkvlAu+jfaL3JkDUp2EjCkXb80ba01vOP87OuJn3Lt0Bw84YMz2gVXQNS3d9SgGA4 K4CXxRXvd+WLzprTTKyQQOtL8KHP3MD0tnJAgtIP27eI3ye87IHRCyz/G3BigL4iUgVn xkm5y1HI+mSe6W81BCx0nGbEbz4YRx2VAs4MVyl6eOW3AxPrwpSl/mTsEKMPW2S3EZwG MUaimpFFPaY+0OcTiRxjsZrwvSiywmRkjHMmcztqBtg0KiCZowU+0ak2mekXFtfKQKrn onzw== X-Gm-Message-State: AOJu0YxtYItXmqfXL77bxmsfxdic9fsqU6UvVPYeZuHzD4d261csWQpW kO4oJ3lu0MKzLGVj4j8gT1CWKG1nrxJHPWMdKaqaSbJZQAz4ghQbzY0= X-Google-Smtp-Source: AGHT+IFe+yGUEWOET+xe4BoV24Rvv9cLCBooCdulvDFAnRESYIL/fxgQwT52yxomQbZUVaOc61bR5BTaaj/MbiTKA2Y= X-Received: by 2002:a2e:9b5a:0:b0:2bd:1fee:aacf with SMTP id o26-20020a2e9b5a000000b002bd1feeaacfmr4804552ljj.24.1693712189994; Sat, 02 Sep 2023 20:36:29 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> In-Reply-To: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> From: Jake Freeland Date: Sat, 2 Sep 2023 22:36:18 -0500 Message-ID: Subject: Re: Is there any plan for ZFS and timerfd updates on stable/14? To: Tomoaki AOKI , Warner Losh Cc: freebsd-stable@freebsd.org Content-Type: multipart/alternative; boundary="0000000000006fd1bd06046c19c6" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4RdcrL20Nwz3dbm --0000000000006fd1bd06046c19c6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Sep 2, 2023 at 10:31=E2=80=AFPM Tomoaki AOKI wrote: > Hi. > > There are discussions about deadlocks issue of ZFS on freebsd-current > ML, starting from [1] last month. > IIRC, at least some fixes (candidates?) are merged to main, but not yet > to stable/14. > > Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems > to have most of them. So my 1st question is "Is there any plan to > import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING > releng/14? > > And one more. timerfd is added at last-minutes BEFORE stable/14 is > branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and > Differential revision D41600 on Phablicator [5] related to memory leaks > and locks. > Additionally, splitting out lib32 part to proper place is proposed > as D41640 [6]. Both [5] and [6] are accepted but not yet landed. > Also, D41641 [7] proposes namespace pollution adjustments. This can be > optional? > > Memory leaks and improper locks can lead system to security issues or > deadlocks, so it would be benefical if landed and MFC'ed BEFORE > releng/14 branches. > > Is there any plan to do so? At least, existing deadlocks should be > considered as SHOW-STOPPER and resolved. > The plan is to get all of those patches in before releng/14.0, I believe. What are your thoughts, Warner? Thanks, Jake Freeland > > I myself am bitten by several deadlocks on poudriere full builds after > upgrading base from stable/13 to stable/14, finally finished with > increasing kern.maxvnodes after powercycle on each deadlock and > continue. > > > Thanks in advance! > > [1] > https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.htm= l > > [2] > > https://cgit.freebsd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05b383c= 8b3aaf18c > > [3] > > https://cgit.freebsd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d7ac04= 444d9c1da > > [4] > > https://cgit.freebsd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76f686c= 2b031876f > > [5] https://reviews.freebsd.org/D41600 > > [6] https://reviews.freebsd.org/D41640 > > [7] https://reviews.freebsd.org/D41641 > > -- > Tomoaki AOKI > > --0000000000006fd1bd06046c19c6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sat, Sep 2, 2023 at 10:31=E2=80=AFPM T= omoaki AOKI <junchoon@dec.s= akura.ne.jp> wrote:
Hi.

There are discussions about deadlocks issue of ZFS on freebsd-current
ML, starting from [1] last month.
IIRC, at least some fixes (candidates?) are merged to main, but not yet
to stable/14.

Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems
to have most of them. So my 1st question is "Is there any plan to
import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING
releng/14?

And one more. timerfd is added at last-minutes BEFORE stable/14 is
branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and
Differential revision D41600 on Phablicator [5] related to memory leaks
and locks.
Additionally, splitting out lib32 part to proper place is proposed
as D41640 [6].=C2=A0 Both [5] and [6] are accepted but not yet landed.
Also, D41641 [7] proposes namespace pollution adjustments. This can be
optional?

Memory leaks and improper locks can lead system to security issues or
deadlocks, so it would be benefical if landed and MFC'ed BEFORE
releng/14 branches.

Is there any plan to do so? At least, existing deadlocks should be
considered as SHOW-STOPPER and resolved.

The plan is to get all of those patches in before releng/14.0, I believe.=

What are your=C2=A0thoughts, Warner?
Thanks,
Jake Freeland
=C2=A0

I myself am bitten by several deadlocks on poudriere full builds after
upgrading base from stable/13 to stable/14, finally finished with
increasing kern.maxvnodes after powercycle on each deadlock and
continue.


Thanks in advance!

[1]
https://lists.freebsd.org/= archives/freebsd-current/2023-August/004162.html

[2]
https://cgit.freeb= sd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05b383c8b3aaf18c

[3]
https://cgit.freeb= sd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d7ac04444d9c1da

[4]
https://cgit.freeb= sd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76f686c2b031876f

[5] https://reviews.freebsd.org/D41600

[6] https://reviews.freebsd.org/D41640

[7] https://reviews.freebsd.org/D41641

--
Tomoaki AOKI=C2=A0 =C2=A0 <junchoon@dec.sakura.ne.jp>

--0000000000006fd1bd06046c19c6-- From nobody Sun Sep 3 03:38:07 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RdctN6H8dz4s1wK for ; Sun, 3 Sep 2023 03:38:20 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RdctN4H3Fz3fYt for ; Sun, 3 Sep 2023 03:38:20 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-99bdcade7fbso52626766b.1 for ; Sat, 02 Sep 2023 20:38:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1693712299; x=1694317099; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ZIR1m21XX5jEZG3n+Hw1A3UxgtzfM1Ostry87SCfXo8=; b=QGs1ScpBfst+zzHAqhxGwQjl5p5YgWmUYbbJD1xMJfVaj+EHQScMKFsiH04ekoXSHh JlyHdv3tGXkdh35pqyD5wFOWspBLTRSESR+x9d2ZZq1nG/7aukzeEqjBIRY7MntmrunE 9BaBGy+QpA4V00ntnO0pDDawVyKtZyRKMppI98UNVOL3z1pQPXt0U7nJM3e6ZyByv7ze sM5MFfOtHi14yEpbRkv2OqK/rRj+FnihsF6Rd1wjS+hUkg1qiLi0UIMH/2+BGCFF1WTh 6Oz6cokauiX1y6qtXqb3DlGbyu0UGRgG5zHfHplpH0+nX7cT/pr/NhLpkX8J8HdAGnNX 8XBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693712299; x=1694317099; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZIR1m21XX5jEZG3n+Hw1A3UxgtzfM1Ostry87SCfXo8=; b=eHM1vEaLGYlNE2VyCRBLZc7eGu4mGTtkBgmGeKUpQAFDM3iYVZ4qeAkx30HmknykK3 bo7xtRwGM6C+w0AYJNXBpLIw4CXQDbZ/RfU34H7Y0zmMupO6y9jvBYx/VZC5hVmMBSP4 WWXIXnuT0kmN9wk3dIs6EDqvEAw4KaUA3i/dotOcT+HURUEMER3QwTV7EOHb8BBu3kh8 50EYgENitKx5aOPd1m4qexDnsXz8Ere8QqxGeqMbm4ZwJpLzxXM/4oVOsPMx58ROC3Iq EojUmleTczcgBgUFVkNOsuewnubdUYqIOeSkLRpM0rkWHjrv4qeNnt2FQGe6dHQ7EUb3 bNTA== X-Gm-Message-State: AOJu0YzJC4qdKgrsEo2ujF74074bimzA2r5K59uuEuqc5uqNq7YFGpZL /NnWRXSaYtMHROKkN511wZ9SJtmSu1GtpZ3FNDzkQQcBR4sM/sP8 X-Google-Smtp-Source: AGHT+IEK5msK9ayEFskMl+BCs+2iL0E7MiZXLwjs7yH3IhRUqxymi5wzjN02xg6+wM9bjgECgH9NiUYM++uKwHzLcuY= X-Received: by 2002:a17:906:845c:b0:99d:dd43:d427 with SMTP id e28-20020a170906845c00b0099ddd43d427mr4080648ejy.10.1693712298893; Sat, 02 Sep 2023 20:38:18 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> In-Reply-To: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> From: Warner Losh Date: Sat, 2 Sep 2023 21:38:07 -0600 Message-ID: Subject: Re: Is there any plan for ZFS and timerfd updates on stable/14? To: Tomoaki AOKI Cc: FreeBSD-STABLE Mailing List Content-Type: multipart/alternative; boundary="000000000000ed779406046c1f25" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4RdctN4H3Fz3fYt --000000000000ed779406046c1f25 Content-Type: text/plain; charset="UTF-8" On Sat, Sep 2, 2023, 9:30 PM Tomoaki AOKI wrote: > Hi. > > There are discussions about deadlocks issue of ZFS on freebsd-current > ML, starting from [1] last month. > IIRC, at least some fixes (candidates?) are merged to main, but not yet > to stable/14. > > Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems > to have most of them. So my 1st question is "Is there any plan to > import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING > releng/14? > > And one more. timerfd is added at last-minutes BEFORE stable/14 is > branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and > Differential revision D41600 on Phablicator [5] related to memory leaks > and locks. > Additionally, splitting out lib32 part to proper place is proposed > as D41640 [6]. Both [5] and [6] are accepted but not yet landed. > Also, D41641 [7] proposes namespace pollution adjustments. This can be > optional? > > Memory leaks and improper locks can lead system to security issues or > deadlocks, so it would be benefical if landed and MFC'ed BEFORE > releng/14 branches. > Yes. The timerfd ones are being finalized. I plan on committing them Tuesday after the long weekend. Warner Is there any plan to do so? At least, existing deadlocks should be > considered as SHOW-STOPPER and resolved. > > I myself am bitten by several deadlocks on poudriere full builds after > upgrading base from stable/13 to stable/14, finally finished with > increasing kern.maxvnodes after powercycle on each deadlock and > continue. > > > Thanks in advance! > > [1] > https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.html > > [2] > > https://cgit.freebsd.org/src/commit/?id=02f534b57f84d6f4f97c337b05b383c8b3aaf18c > > [3] > > https://cgit.freebsd.org/src/commit/?id=5eab523053db79b4bd4f926c7d7ac04444d9c1da > > [4] > > https://cgit.freebsd.org/src/commit/?id=f4296cfb409a48de00bfa60e76f686c2b031876f > > [5] https://reviews.freebsd.org/D41600 > > [6] https://reviews.freebsd.org/D41640 > > [7] https://reviews.freebsd.org/D41641 > > -- > Tomoaki AOKI > > --000000000000ed779406046c1f25 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Sep 2, 2023, 9:30 PM Tomoaki AOKI <junchoon@dec.sakura.ne.jp> wro= te:
Hi.

There are discussions about deadlocks issue of ZFS on freebsd-current
ML, starting from [1] last month.
IIRC, at least some fixes (candidates?) are merged to main, but not yet
to stable/14.

Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems
to have most of them. So my 1st question is "Is there any plan to
import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING
releng/14?

And one more. timerfd is added at last-minutes BEFORE stable/14 is
branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and
Differential revision D41600 on Phablicator [5] related to memory leaks
and locks.
Additionally, splitting out lib32 part to proper place is proposed
as D41640 [6].=C2=A0 Both [5] and [6] are accepted but not yet landed.
Also, D41641 [7] proposes namespace pollution adjustments. This can be
optional?

Memory leaks and improper locks can lead system to security issues or
deadlocks, so it would be benefical if landed and MFC'ed BEFORE
releng/14 branches.

Yes. The timerfd ones are being finalized.=C2=A0 I plan = on committing them Tuesday after the long weekend.
<= br>
Warner

Is there any plan to do so? At least, existing deadlocks should be
considered as SHOW-STOPPER and resolved.

I myself am bitten by several deadlocks on poudriere full builds after
upgrading base from stable/13 to stable/14, finally finished with
increasing kern.maxvnodes after powercycle on each deadlock and
continue.


Thanks in advance!

[1]
https://lists.f= reebsd.org/archives/freebsd-current/2023-August/004162.html

[2]
https:/= /cgit.freebsd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05b383c8b3aaf18c=

[3]
https:/= /cgit.freebsd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d7ac04444d9c1da=

[4]
https:/= /cgit.freebsd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76f686c2b031876f=

[5] https://reviews.freebsd.org/D41600

[6] https://reviews.freebsd.org/D41640

[7] https://reviews.freebsd.org/D41641

--
Tomoaki AOKI=C2=A0 =C2=A0 <junchoon@dec.sakura.ne.jp>

--000000000000ed779406046c1f25-- From nobody Sun Sep 3 03:40:33 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RdcxC0ptZz4s2Br for ; Sun, 3 Sep 2023 03:40:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RdcxB69Fnz3gc0 for ; Sun, 3 Sep 2023 03:40:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-52a5c0d949eso391241a12.0 for ; Sat, 02 Sep 2023 20:40:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1693712444; x=1694317244; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=nj94eJuFapgwFcuv2O43EqPL0exKHeDNTYRB0qfZJuM=; b=Zk1e7TgWBa9wdBZbar5mrwDNXZ7t+FYNkp7EJ4kdX4bZCeKDRyEV14Z42PtvrmG0ZM jkuZoMHS2cmW7V60RyWIMTlP5OixDdhmLota4NTipb3PfuxX4nghJRp9sIexWYaPnqQx uUqJ3feFZpkkYHsqtfwnJA5uKnZO9tESkgM5/+Lk2Wea2aQcQqA6vSl+OmGhei6jNeaR OmAlEfIEecz0fX+cLGJn7H73XdwhunY+bYCLihS8mWaw6ZEWNxJ/3W8qvB8WgLs1M9nd SccZ4xuQIpxnBpzau2haEXN/DCKBqPncaa6c0BBVEcm8puHWGmbyn4+t5+KzYYhBQP4+ GN9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693712444; x=1694317244; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nj94eJuFapgwFcuv2O43EqPL0exKHeDNTYRB0qfZJuM=; b=GogqzoAYM+cpkEYQOyDAD7+LoCHikHHcpFG94Ilze2ruo3H7nQZsB59rdvKafWe3Dt FzTZ8AGnIlfCRH06KL0GwQ0nw1MFnuRHoxGLyKkp9eY4jUEXJq/an39v0MazWTinJfHk PPFwIh3xG1z7IyeqbwI/Tt7g19pKrdeplpZle4s4zdb27j0z2LElCi0O8Lxasb3bUfBr 5zHUZ+tnaRdThpVnSSjajvYYwAvH8m5pQFiKv0yOSQG9OtX92om21bK+kYhebVQlsTjc m7eOH4w/NzwHHAAUcPE1JCUBNLW4myNDjNhNKYQh4yEXasjWKz7vuwtaZKtQVLiXQ+1O zkyw== X-Gm-Message-State: AOJu0YyXGEsM87mJbwCzy2WOWODCzl394K+f99LFZTg0DbMxARZFpV1E bB/weZ8DPQc/TlNjHZlig8OrlaGn0iDyKYR5JQXVJg== X-Google-Smtp-Source: AGHT+IGLdnEQhInTV98dMfS7fY0thrTlldiZWwAjVCukzKM4HsZgPnnfGlenn7EW/U/ltsWqOB/De1HOO/s5BqDdEZY= X-Received: by 2002:a17:906:10dc:b0:9a1:ec3d:9004 with SMTP id v28-20020a17090610dc00b009a1ec3d9004mr5264541ejv.9.1693712444547; Sat, 02 Sep 2023 20:40:44 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> In-Reply-To: From: Warner Losh Date: Sat, 2 Sep 2023 21:40:33 -0600 Message-ID: Subject: Re: Is there any plan for ZFS and timerfd updates on stable/14? To: Jake Freeland Cc: Tomoaki AOKI , FreeBSD-STABLE Mailing List Content-Type: multipart/alternative; boundary="0000000000009bf61206046c2822" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4RdcxB69Fnz3gc0 --0000000000009bf61206046c2822 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Sep 2, 2023, 9:36 PM Jake Freeland wrote: > On Sat, Sep 2, 2023 at 10:31=E2=80=AFPM Tomoaki AOKI > wrote: > >> Hi. >> >> There are discussions about deadlocks issue of ZFS on freebsd-current >> ML, starting from [1] last month. >> IIRC, at least some fixes (candidates?) are merged to main, but not yet >> to stable/14. >> >> Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems >> to have most of them. So my 1st question is "Is there any plan to >> import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING >> releng/14? >> >> And one more. timerfd is added at last-minutes BEFORE stable/14 is >> branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and >> Differential revision D41600 on Phablicator [5] related to memory leaks >> and locks. >> Additionally, splitting out lib32 part to proper place is proposed >> as D41640 [6]. Both [5] and [6] are accepted but not yet landed. >> Also, D41641 [7] proposes namespace pollution adjustments. This can be >> optional? >> >> Memory leaks and improper locks can lead system to security issues or >> deadlocks, so it would be benefical if landed and MFC'ed BEFORE >> releng/14 branches. >> >> Is there any plan to do so? At least, existing deadlocks should be >> considered as SHOW-STOPPER and resolved. >> > > The plan is to get all of those patches in before releng/14.0, I believe. > > What are your thoughts, Warner? > Sounds like the reviews are done or nearly so. I've not had time to look closely to be sure... I'd planned on making time Tuesday morning. Warner > Thanks, > Jake Freeland > > >> >> I myself am bitten by several deadlocks on poudriere full builds after >> upgrading base from stable/13 to stable/14, finally finished with >> increasing kern.maxvnodes after powercycle on each deadlock and >> continue. >> >> >> Thanks in advance! >> >> [1] >> https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.ht= ml >> >> [2] >> >> https://cgit.freebsd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05b383= c8b3aaf18c >> >> [3] >> >> https://cgit.freebsd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d7ac0= 4444d9c1da >> >> [4] >> >> https://cgit.freebsd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76f686= c2b031876f >> >> [5] https://reviews.freebsd.org/D41600 >> >> [6] https://reviews.freebsd.org/D41640 >> >> [7] https://reviews.freebsd.org/D41641 >> >> -- >> Tomoaki AOKI >> >> --0000000000009bf61206046c2822 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Sep 2, 2023, 9:36 PM Jake Freeland <jake@technologyfriends.net> w= rote:
On Sat, Sep 2, 2023 at 10:31=E2=80=AFPM Tomoaki AOKI <juncho= on@dec.sakura.ne.jp> wrote:
Hi.

There are discussions about deadlocks issue of ZFS on freebsd-current
ML, starting from [1] last month.
IIRC, at least some fixes (candidates?) are merged to main, but not yet
to stable/14.

Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems
to have most of them. So my 1st question is "Is there any plan to
import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING
releng/14?

And one more. timerfd is added at last-minutes BEFORE stable/14 is
branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and
Differential revision D41600 on Phablicator [5] related to memory leaks
and locks.
Additionally, splitting out lib32 part to proper place is proposed
as D41640 [6].=C2=A0 Both [5] and [6] are accepted but not yet landed.
Also, D41641 [7] proposes namespace pollution adjustments. This can be
optional?

Memory leaks and improper locks can lead system to security issues or
deadlocks, so it would be benefical if landed and MFC'ed BEFORE
releng/14 branches.

Is there any plan to do so? At least, existing deadlocks should be
considered as SHOW-STOPPER and resolved.

The plan is to get all of those patches in before releng/14.0, I believe.=

What are your=C2=A0thoughts, Warner?
<= /div>

Sounds like the reviews are done or nearly so. I've not had time to lo= ok closely to be sure... I'd planned on making time Tuesday morning.=C2= =A0

Warner


<= /div>
Thanks,
Jake Freeland
=C2=A0

I myself am bitten by several deadlocks on poudriere full builds after
upgrading base from stable/13 to stable/14, finally finished with
increasing kern.maxvnodes after powercycle on each deadlock and
continue.


Thanks in advance!

[1]
https://lists.f= reebsd.org/archives/freebsd-current/2023-August/004162.html

[2]
https:/= /cgit.freebsd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05b383c8b3aaf18c=

[3]
https:/= /cgit.freebsd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d7ac04444d9c1da=

[4]
https:/= /cgit.freebsd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76f686c2b031876f=

[5] https://reviews.freebsd.org/D41600

[6] https://reviews.freebsd.org/D41640

[7] https://reviews.freebsd.org/D41641

--
Tomoaki AOKI=C2=A0 =C2=A0 <junchoon@dec.sakura.ne.jp>

--0000000000009bf61206046c2822-- From nobody Sun Sep 3 03:47:53 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rdd5f4crjz4s3ts for ; Sun, 3 Sep 2023 03:48:06 +0000 (UTC) (envelope-from jake@technologyfriends.net) Received: from mail-lj1-x22a.google.com (mail-lj1-x22a.google.com [IPv6:2a00:1450:4864:20::22a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rdd5f1p4Jz4FZ7 for ; Sun, 3 Sep 2023 03:48:06 +0000 (UTC) (envelope-from jake@technologyfriends.net) Authentication-Results: mx1.freebsd.org; none Received: by mail-lj1-x22a.google.com with SMTP id 38308e7fff4ca-2bb9a063f26so4379791fa.2 for ; Sat, 02 Sep 2023 20:48:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=technologyfriends.net; s=google; t=1693712884; x=1694317684; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=oGmwaX0jdaJ1v/MYpyfS1ZQAYN5uiSXRctE3ymcFnZI=; b=anHlnEHbw7MXWYav4Olil//o8cIybjmoycM9XekshZlKO+AKYh3sAr4rJkGTgKMhYt lhX+D7hpBoakkKWCG10XtBeyJ+OoRKGARuCh/Q7vKY++P2i1C5FCo2X4QhvoeZldLzJ0 Kv6j+YLBSi/R0YP1dT152R6jaWViD5SQK3PR5wK+7ufs9ZeOIQFgDrXgE2PzI41zd+s8 KJA90llh+VkUtghDHIrAQ2Dz3RpYhEGWI0h2YfIj8UWIDsw/G6OzSw9Bbpqk0IThNu+P PiMmblAQ9AFIdGGbwePo6boaBpyeo+aQkVOfJ+cBPsyboFpjL+rmN1uvq+1GdlFhqj2w +Xbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693712884; x=1694317684; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=oGmwaX0jdaJ1v/MYpyfS1ZQAYN5uiSXRctE3ymcFnZI=; b=cTZ3RSTUEB+xxiwxHt5+xqcy79NpksNJBDDRjyzbUYsPQ/QJaE0oIZ9SNvwd++PQmM 6Rv/p10nSRIgGK3i40rt+5penb+y9QGTIfWlc4jFTtJexMabiKl+FAeF4+5oRPnkhImp xZQCNe252WHF79fvzB69exJvqIZ8h9bI9pX2FVQ5aEmCa8B/x4Y/rCztZPsvK/S1F/zI /lrSqH/MQDVtLsm+GdDxAU1qcjnAbQ8XSr60WAurngo4qGlx59M0q5dBlVTJ5l6pHFnG JawO2PHBvXcX2OZXyQdGqbNXM7GUqZSoFkkKCiLGMuBp0QB94WgSBIAps8TxotLCVbeO PEMw== X-Gm-Message-State: AOJu0YyVFPnPBq0R3iOTYIt7AIXnQZHGDE8jEXoNUZBSwqmgEyEmPtjO CBVkbwJdBSeaxw462eMJ7mFx8yQ+4UUp+lAHb1R1IOPBJnp9Zwo9du4= X-Google-Smtp-Source: AGHT+IE/iJ7JJb2XtJvTxldWK/4SE4nXs8u2pKkPN4yLeRhoFb5w+YilLuGcI8HNVcLzsDFQiFsPu9so96EvOJ5c0Kw= X-Received: by 2002:a2e:8610:0:b0:2b7:31a:9d7c with SMTP id a16-20020a2e8610000000b002b7031a9d7cmr4397939lji.33.1693712884492; Sat, 02 Sep 2023 20:48:04 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> In-Reply-To: From: Jake Freeland Date: Sat, 2 Sep 2023 22:47:53 -0500 Message-ID: Subject: Re: Is there any plan for ZFS and timerfd updates on stable/14? To: Warner Losh Cc: Tomoaki AOKI , FreeBSD-STABLE Mailing List Content-Type: multipart/alternative; boundary="000000000000d4ffdf06046c423d" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4Rdd5f1p4Jz4FZ7 --000000000000d4ffdf06046c423d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Sep 2, 2023 at 10:40=E2=80=AFPM Warner Losh wrote: > > > On Sat, Sep 2, 2023, 9:36 PM Jake Freeland > wrote: > >> On Sat, Sep 2, 2023 at 10:31=E2=80=AFPM Tomoaki AOKI >> wrote: >> >>> Hi. >>> >>> There are discussions about deadlocks issue of ZFS on freebsd-current >>> ML, starting from [1] last month. >>> IIRC, at least some fixes (candidates?) are merged to main, but not yet >>> to stable/14. >>> >>> Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems >>> to have most of them. So my 1st question is "Is there any plan to >>> import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING >>> releng/14? >>> >>> And one more. timerfd is added at last-minutes BEFORE stable/14 is >>> branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and >>> Differential revision D41600 on Phablicator [5] related to memory leaks >>> and locks. >>> Additionally, splitting out lib32 part to proper place is proposed >>> as D41640 [6]. Both [5] and [6] are accepted but not yet landed. >>> Also, D41641 [7] proposes namespace pollution adjustments. This can be >>> optional? >>> >>> Memory leaks and improper locks can lead system to security issues or >>> deadlocks, so it would be benefical if landed and MFC'ed BEFORE >>> releng/14 branches. >>> >>> Is there any plan to do so? At least, existing deadlocks should be >>> considered as SHOW-STOPPER and resolved. >>> >> >> The plan is to get all of those patches in before releng/14.0, I believe= . >> >> What are your thoughts, Warner? >> > > Sounds like the reviews are done or nearly so. I've not had time to look > closely to be sure... I'd planned on making time Tuesday morning. > Yes. All reviews are good to go. Jake Freeland > > Warner > > >> Thanks, >> Jake Freeland >> >> >>> >>> I myself am bitten by several deadlocks on poudriere full builds after >>> upgrading base from stable/13 to stable/14, finally finished with >>> increasing kern.maxvnodes after powercycle on each deadlock and >>> continue. >>> >>> >>> Thanks in advance! >>> >>> [1] >>> >>> https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.h= tml >>> >>> [2] >>> >>> https://cgit.freebsd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05b38= 3c8b3aaf18c >>> >>> [3] >>> >>> https://cgit.freebsd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d7ac= 04444d9c1da >>> >>> [4] >>> >>> https://cgit.freebsd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76f68= 6c2b031876f >>> >>> [5] https://reviews.freebsd.org/D41600 >>> >>> [6] https://reviews.freebsd.org/D41640 >>> >>> [7] https://reviews.freebsd.org/D41641 >>> >>> -- >>> Tomoaki AOKI >>> >>> --000000000000d4ffdf06046c423d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sat, Sep 2, 2023 at 10:40=E2=80=AFPM W= arner Losh <imp@bsdimp.com> wro= te:


On Sat, Sep 2, 2023, 9:36 PM Jake Freel= and <jak= e@technologyfriends.net> wrote:
On Sat, Sep 2, 202= 3 at 10:31=E2=80=AFPM Tomoaki AOKI <junchoon@dec.sakura.ne.jp= > wrote:
Hi.

There are discussions about deadlocks issue of ZFS on freebsd-current
ML, starting from [1] last month.
IIRC, at least some fixes (candidates?) are merged to main, but not yet
to stable/14.

Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems
to have most of them. So my 1st question is "Is there any plan to
import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING
releng/14?

And one more. timerfd is added at last-minutes BEFORE stable/14 is
branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and
Differential revision D41600 on Phablicator [5] related to memory leaks
and locks.
Additionally, splitting out lib32 part to proper place is proposed
as D41640 [6].=C2=A0 Both [5] and [6] are accepted but not yet landed.
Also, D41641 [7] proposes namespace pollution adjustments. This can be
optional?

Memory leaks and improper locks can lead system to security issues or
deadlocks, so it would be benefical if landed and MFC'ed BEFORE
releng/14 branches.

Is there any plan to do so? At least, existing deadlocks should be
considered as SHOW-STOPPER and resolved.

The plan is to get all of those patches in before releng/14.0, I believe.=

What are your=C2=A0thoughts, Warner?
<= /div>

Sounds like the reviews are done or nearly so. I've not had time to lo= ok closely to be sure... I'd planned on making time Tuesday morning.=C2= =A0

Yes. All reviews are good t= o go.

Jake Freeland
=C2=A0

Warner


=
Thanks,
Jake Freeland
=C2=A0

I myself am bitten by several deadlocks on poudriere full builds after
upgrading base from stable/13 to stable/14, finally finished with
increasing kern.maxvnodes after powercycle on each deadlock and
continue.


Thanks in advance!

[1]
https://lists.f= reebsd.org/archives/freebsd-current/2023-August/004162.html

[2]
https:/= /cgit.freebsd.org/src/commit/?id=3D02f534b57f84d6f4f97c337b05b383c8b3aaf18c=

[3]
https:/= /cgit.freebsd.org/src/commit/?id=3D5eab523053db79b4bd4f926c7d7ac04444d9c1da=

[4]
https:/= /cgit.freebsd.org/src/commit/?id=3Df4296cfb409a48de00bfa60e76f686c2b031876f=

[5] https://reviews.freebsd.org/D41600

[6] https://reviews.freebsd.org/D41640

[7] https://reviews.freebsd.org/D41641

--
Tomoaki AOKI=C2=A0 =C2=A0 <junchoon@dec.sakura.ne.jp>

--000000000000d4ffdf06046c423d-- From ml@ft-c.de Sun Sep 3 03:54:53 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RddFY5pnqz4s4vN for ; Sun, 3 Sep 2023 03:54:57 +0000 (UTC) (envelope-from ml@ft-c.de) Received: from einhorn-mail-out.in-berlin.de (einhorn.in-berlin.de [192.109.42.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.in-berlin.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RddFX5h4Sz4HLQ for ; Sun, 3 Sep 2023 03:54:56 +0000 (UTC) (envelope-from ml@ft-c.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of ml@ft-c.de designates 192.109.42.8 as permitted sender) smtp.mailfrom=ml@ft-c.de; dmarc=none X-Envelope-From: ml@ft-c.de X-Envelope-To: Received: from authenticated.user (localhost [127.0.0.1]) by einhorn.in-berlin.de with ESMTPSA id 3833sruk076064 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Sun, 3 Sep 2023 05:54:54 +0200 Message-ID: Subject: postgresql pg_curl bug? From: ft Reply-To: ml@ft-c.de To: stable@FreeBSD.org Date: Sun, 03 Sep 2023 05:54:53 +0200 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.44.4 FreeBSD GNOME Team List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 X-Spamd-Bar: / X-Spamd-Result: default: False [-0.08 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_MEDIUM(0.92)[0.923]; NEURAL_HAM_SHORT(-0.60)[-0.601]; R_SPF_ALLOW(-0.20)[+ip4:192.109.42.0/24]; RWL_MAILSPIKE_GOOD(-0.10)[192.109.42.8:from]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[192.109.42.8:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; HAS_REPLYTO(0.00)[ml@ft-c.de]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[ft-c.de]; ARC_NA(0.00)[]; ASN(0.00)[asn:29670, ipnet:192.109.42.0/24, country:DE]; MLMMJ_DEST(0.00)[stable@FreeBSD.org]; TO_DN_NONE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4RddFX5h4Sz4HLQ Hello, I have install postgresql15 in jail. I need the postgresql extension pg_curl. cd pg_curl-2.1.1/ gmake install cc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after- statement -Werror=3Dvla -Werror=3Dunguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno- strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno- compound-token-split-by-macro -O2 -pipe -fstack-protector-strong -fno- strict-aliasing -fPIC -DPIC -I. -I./ - I/usr/local/include/postgresql/server - I/usr/local/include/postgresql/internal -I/usr/local/include - I/usr/local/include -I/usr/local/include -c -o pg_curl.o pg_curl.c cc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after- statement -Werror=3Dvla -Werror=3Dunguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno- strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno- compound-token-split-by-macro -O2 -pipe -fstack-protector-strong -fno- strict-aliasing -fPIC -DPIC -shared -o pg_curl.so pg_curl.o - L/usr/local/lib -L/usr/local/lib -lpthread -L/usr/local/lib -fstack- protector-strong -L/usr/local/lib -Wl,--as-needed -Wl,- R'/usr/local/lib' -lcurl /bin/mkdir -p '/usr/local/lib/postgresql' /bin/mkdir -p '/usr/local/share/postgresql/extension' /bin/mkdir -p '/usr/local/share/postgresql/extension' /usr/bin/install -c -m 755 pg_curl.so '/usr/local/lib/postgresql/pg_curl.so' /usr/bin/install -c -m 644 .//pg_curl.control '/usr/local/share/postgresql/extension/' /usr/bin/install -c -m 644 .//pg_curl--1.0--2.0.sql .//pg_curl--1.0.sql .//pg_curl--2.0--2.1.sql .//pg_curl--2.0.sql .//pg_curl--2.1.sql=20 '/usr/local/share/postgresql/extension/' find /usr -name "*pg_curl*" /usr/local/share/postgresql/extension/pg_curl--2.0.sql /usr/local/share/postgresql/extension/pg_curl.control /usr/local/share/postgresql/extension/pg_curl--2.1.sql /usr/local/share/postgresql/extension/pg_curl--2.0--2.1.sql /usr/local/share/postgresql/extension/pg_curl--1.0.sql /usr/local/share/postgresql/extension/pg_curl--1.0--2.0.sql In postgresql, the command: create extension pg_curl; get the errormessage: ftc=3D# create extension pg_curl; FEHLER: konnte Bibliothek =C2=BB/usr/local/lib/postgresql/pg_curl.so=C2=AB= nicht laden: /usr/local/lib/libidn2.so.0: Undefined symbol "strversc mp@FBSD_1.7" libidn2 is installed: ls -l /usr/local/lib/libidn2* root wheel 281986 Feb 9 2023 /usr/local/lib/libidn2.a root wheel 16 Feb 9 2023 /usr/local/lib/libidn2.so -> libidn2.so.0.3.8 root wheel 16 Feb 9 2023 /usr/local/lib/libidn2.so.0 -> libidn2.so.0.3.8 root wheel 202256 Feb 9 2023 /usr/local/lib/libidn2.so.0.3.8 uname -a FreeBSD ftc 13.2-RELEASE-p2 FreeBSD 13.2-RELEASE-p2 GENERIC amd64 Do you need more information? What should I do? Franz From nobody Sun Sep 3 04:30:51 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rdf394bq5z4sFdG for ; Sun, 3 Sep 2023 04:31:01 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rdf3917S0z4QS3 for ; Sun, 3 Sep 2023 04:31:00 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 3834Upsa061466; Sun, 3 Sep 2023 13:30:52 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 3 Sep 2023 13:30:51 +0900 From: Tomoaki AOKI To: Warner Losh Cc: FreeBSD-STABLE Mailing List Subject: Re: Is there any plan for ZFS and timerfd updates on stable/14? Message-Id: <20230903133051.b9f03bf1a471bcca1b4cd523@dec.sakura.ne.jp> In-Reply-To: References: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] X-Rspamd-Queue-Id: 4Rdf3917S0z4QS3 On Sat, 2 Sep 2023 21:38:07 -0600 Warner Losh wrote: > On Sat, Sep 2, 2023, 9:30 PM Tomoaki AOKI wrote: > > > Hi. > > > > There are discussions about deadlocks issue of ZFS on freebsd-current > > ML, starting from [1] last month. > > IIRC, at least some fixes (candidates?) are merged to main, but not yet > > to stable/14. > > > > Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems > > to have most of them. So my 1st question is "Is there any plan to > > import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING > > releng/14? > > > > And one more. timerfd is added at last-minutes BEFORE stable/14 is > > branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and > > Differential revision D41600 on Phablicator [5] related to memory leaks > > and locks. > > Additionally, splitting out lib32 part to proper place is proposed > > as D41640 [6]. Both [5] and [6] are accepted but not yet landed. > > Also, D41641 [7] proposes namespace pollution adjustments. This can be > > optional? > > > > Memory leaks and improper locks can lead system to security issues or > > deadlocks, so it would be benefical if landed and MFC'ed BEFORE > > releng/14 branches. > > > > Yes. The timerfd ones are being finalized. I plan on committing them > Tuesday after the long weekend. > > Warner Glad to know. Thanks! For ZFS-related bits, maybe I should wait for answers from mm@ or mav@. Regards. > > Is there any plan to do so? At least, existing deadlocks should be > > considered as SHOW-STOPPER and resolved. > > > > I myself am bitten by several deadlocks on poudriere full builds after > > upgrading base from stable/13 to stable/14, finally finished with > > increasing kern.maxvnodes after powercycle on each deadlock and > > continue. > > > > > > Thanks in advance! > > > > [1] > > https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.html > > > > [2] > > > > https://cgit.freebsd.org/src/commit/?id=02f534b57f84d6f4f97c337b05b383c8b3aaf18c > > > > [3] > > > > https://cgit.freebsd.org/src/commit/?id=5eab523053db79b4bd4f926c7d7ac04444d9c1da > > > > [4] > > > > https://cgit.freebsd.org/src/commit/?id=f4296cfb409a48de00bfa60e76f686c2b031876f > > > > [5] https://reviews.freebsd.org/D41600 > > > > [6] https://reviews.freebsd.org/D41640 > > > > [7] https://reviews.freebsd.org/D41641 > > > > -- > > Tomoaki AOKI -- Tomoaki AOKI From nobody Sun Sep 3 04:33:28 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rdf654Mdfz4sG91 for ; Sun, 3 Sep 2023 04:33:33 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rdf651BBPz4S7w for ; Sun, 3 Sep 2023 04:33:32 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 3834XSDI061826; Sun, 3 Sep 2023 13:33:28 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 3 Sep 2023 13:33:28 +0900 From: Tomoaki AOKI To: Jake Freeland Cc: Warner Losh , FreeBSD-STABLE Mailing List Subject: Re: Is there any plan for ZFS and timerfd updates on stable/14? Message-Id: <20230903133328.54577b85b097da319ecde4ba@dec.sakura.ne.jp> In-Reply-To: References: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] X-Rspamd-Queue-Id: 4Rdf651BBPz4S7w On Sat, 2 Sep 2023 22:47:53 -0500 Jake Freeland wrote: > On Sat, Sep 2, 2023 at 10:40 PM Warner Losh wrote: > > > > > > > On Sat, Sep 2, 2023, 9:36 PM Jake Freeland > > wrote: > > > >> On Sat, Sep 2, 2023 at 10:31 PM Tomoaki AOKI > >> wrote: > >> > >>> Hi. > >>> > >>> There are discussions about deadlocks issue of ZFS on freebsd-current > >>> ML, starting from [1] last month. > >>> IIRC, at least some fixes (candidates?) are merged to main, but not yet > >>> to stable/14. > >>> > >>> Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems > >>> to have most of them. So my 1st question is "Is there any plan to > >>> import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING > >>> releng/14? > >>> > >>> And one more. timerfd is added at last-minutes BEFORE stable/14 is > >>> branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and > >>> Differential revision D41600 on Phablicator [5] related to memory leaks > >>> and locks. > >>> Additionally, splitting out lib32 part to proper place is proposed > >>> as D41640 [6]. Both [5] and [6] are accepted but not yet landed. > >>> Also, D41641 [7] proposes namespace pollution adjustments. This can be > >>> optional? > >>> > >>> Memory leaks and improper locks can lead system to security issues or > >>> deadlocks, so it would be benefical if landed and MFC'ed BEFORE > >>> releng/14 branches. > >>> > >>> Is there any plan to do so? At least, existing deadlocks should be > >>> considered as SHOW-STOPPER and resolved. > >>> > >> > >> The plan is to get all of those patches in before releng/14.0, I believe. > >> > >> What are your thoughts, Warner? > >> > > > > Sounds like the reviews are done or nearly so. I've not had time to look > > closely to be sure... I'd planned on making time Tuesday morning. > > > > Yes. All reviews are good to go. > > Jake Freeland Glad to know. Thanks! Looking forward to see them landed / MFC'ed before releng/14 branches. Regards. > > > > > > Warner > > > > > >> Thanks, > >> Jake Freeland > >> > >> > >>> > >>> I myself am bitten by several deadlocks on poudriere full builds after > >>> upgrading base from stable/13 to stable/14, finally finished with > >>> increasing kern.maxvnodes after powercycle on each deadlock and > >>> continue. > >>> > >>> > >>> Thanks in advance! > >>> > >>> [1] > >>> > >>> https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.html > >>> > >>> [2] > >>> > >>> https://cgit.freebsd.org/src/commit/?id=02f534b57f84d6f4f97c337b05b383c8b3aaf18c > >>> > >>> [3] > >>> > >>> https://cgit.freebsd.org/src/commit/?id=5eab523053db79b4bd4f926c7d7ac04444d9c1da > >>> > >>> [4] > >>> > >>> https://cgit.freebsd.org/src/commit/?id=f4296cfb409a48de00bfa60e76f686c2b031876f > >>> > >>> [5] https://reviews.freebsd.org/D41600 > >>> > >>> [6] https://reviews.freebsd.org/D41640 > >>> > >>> [7] https://reviews.freebsd.org/D41641 > >>> > >>> -- > >>> Tomoaki AOKI -- Tomoaki AOKI From nobody Sun Sep 3 10:51:22 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RdpVM1F7Bz4rf0J for ; Sun, 3 Sep 2023 10:51:39 +0000 (UTC) (envelope-from herbert@gojira.at) Received: from mout-u-204.mailbox.org (mout-u-204.mailbox.org [80.241.59.204]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RdpVK117Fz4Yxr for ; Sun, 3 Sep 2023 10:51:36 +0000 (UTC) (envelope-from herbert@gojira.at) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gojira.at header.s=MBO0001 header.b=e6DkfCJN; spf=pass (mx1.freebsd.org: domain of herbert@gojira.at designates 80.241.59.204 as permitted sender) smtp.mailfrom=herbert@gojira.at; dmarc=none Received: from smtp202.mailbox.org (smtp202.mailbox.org [10.196.197.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-u-204.mailbox.org (Postfix) with ESMTPS id 4RdpVC3WV6z9sSf for ; Sun, 3 Sep 2023 12:51:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gojira.at; s=MBO0001; t=1693738291; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=SaV/Yc+jWtgqbRdv33gDjWEqDWMtkoOTYkubG0Wsyl8=; b=e6DkfCJN6nQVNA80zlKjgQfbFpXPmXpBX17EhA7rNCjQKJpGatvy6bTVWpG/bE/ew0R2uN UqJe8r7tUzZsfvK0umvCZtm2uP9SasHeChIb5u3uv3OnArZ/WYlqAQsoWTt6S/Dk6RphC0 4VRZAfSN1ByKYS7lSZ87AHEnkV2pkSP/Ivec5pHWGyCkDsD8/mnMpW5+4kWIIfkWDCAJrz KsaU+bJ3nD+uR1g7SZiJCIrc6uV7wk56ppPt7t1of+1KT1taTpok8NN0Y212W3ZwxRK/pN 2Hxacccvetj7DOzouxliffMfxSTtyt8L17JZIbkBTuO3EVtE+K1ts6GnVafXsA== Date: Sun, 03 Sep 2023 12:51:22 +0200 Message-ID: <87il8rttqt.wl-herbert@gojira.at> From: "Herbert J. Skuhra" To: stable@FreeBSD.org Subject: Re: net/mpd5 on stable/14 - COMPAT_FREEBSD12 required? In-Reply-To: <7b59195f-9902-629e-79e3-8a3ecc2ea161@grosbein.net> References: <871qfkr747.wl-herbert@gojira.at> <87zg28prqx.wl-herbert@gojira.at> <87h6ofkvym.wl-herbert@gojira.at> <87fs3yxvkf.wl-herbert@gojira.at> <7b59195f-9902-629e-79e3-8a3ecc2ea161@grosbein.net> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-Spamd-Bar: - X-Spamd-Result: default: False [-1.60 / 15.00]; MID_CONTAINS_FROM(1.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; R_DKIM_ALLOW(-0.20)[gojira.at:s=MBO0001]; R_SPF_ALLOW(-0.20)[+ip4:80.241.56.0/21]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[80.241.59.204:from]; MLMMJ_DEST(0.00)[stable@FreeBSD.org]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:199118, ipnet:80.241.56.0/21, country:DE]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; DMARC_NA(0.00)[gojira.at]; BLOCKLISTDE_FAIL(0.00)[80.241.59.204:server fail]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[gojira.at:+]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4RdpVK117Fz4Yxr On Fri, 01 Sep 2023 08:44:50 +0200, Eugene Grosbein wrote: > > 01.09.2023 13:21, Herbert J. Skuhra wrote: > > > On Thu, 31 Aug 2023 18:39:13 +0200, "Herbert J. Skuhra" wrote: > >> > >> On Wed, 30 Aug 2023 15:46:46 +0200, "Herbert J. Skuhra" wrote: > >>> > >>> On Wed, 30 Aug 2023 15:29:28 +0200, "Herbert J. Skuhra" wrote: > >>>> > >>>> Hi, > >>>> > >>>> after updating from stable/13 to (main and later to) stable/14 > >>>> net/mpd5 only connects to my ISP (VDSL with VLAN) if I enable "options > >>>> COMPAT_FREEBSD12" in my kernel. Without I get only: > >>>> > >>>> Aug 30 15:08:06 gw mpd[59876]: [L1] PPPoE connection timeout after 9 seconds > >>>> Aug 30 15:08:06 gw mpd[59876]: [L1] Link: DOWN event > >>>> Aug 30 15:08:06 gw mpd[59876]: [L1] LCP: Down event > >>>> Aug 30 15:08:06 gw mpd[59876]: [L1] Link: reconnection attempt 1 in 7 seconds > >>>> Aug 30 15:08:14 gw mpd[59876]: [L1] Link: reconnection attempt 1 > >>>> Aug 30 15:08:14 gw mpd[59876]: [L1] PPPoE: Connecting to '' > >>>> Aug 30 15:08:23 gw mpd[59876]: [L1] PPPoE connection timeout after 9 seconds > >>>> Aug 30 15:08:23 gw mpd[59876]: [L1] Link: DOWN event > >>>> Aug 30 15:08:23 gw mpd[59876]: [L1] LCP: Down event > >>>> [..] > >>>> > >>>> I guess the problem is: > >>>> > >>>> Without "COMPAT_FREEBSD12" vlanproto = 0x0000. If I run "ifconfig > >>>> vlan31 vlanproto 802.1q" mpd5 works. > >>> > >>> Maybe this commit is related: > >>> > >>> commit afbb64f1d85b7d8c2938031c3567946b5d10da4f > >>> Author: Alexander V. Chernikov > >>> Date: Sun Apr 11 17:47:03 2021 +0100 > >>> > >>> Fix vlan creation for the older ifconfig(8) binaries. > >>> > >>> Reported by: allanjude > >>> MFC after: immediately > >>> > >>> ? > >> > >> On stable/13 (GENERIC without COMPAT_FREEBSD12): > >> > >> # ifconfig vlan99 create > >> ==> vlanproto: 0x0000 > >> # ifconfig vlan99 vlan 99 vlandev igb0 > >> ==> vlanproto: 802.1q > >> # ifconfig vlan99 destroy > >> # ifconfig vlan99 create vlan 99 vlandev igb0 > >> ==> vlanproto: 802.1q > >> > >> On stable/14 or main (GENERIC without COMPAT_FREEBSD12): > >> > >> # ifconfig vlan99 create > >> ==> vlanproto 0x0000 > >> # ifconfig vlan99 vlan 99 vlandev igb0 > >> ==> vlanproto 0x0000# > >> # ifconfig vlan99 create vlan 99 vlandev igb0 > >> ==> vlanproto: 802.1q > >> > >> I guess that's why the following lines in my /etc/rc.conf no longer > >> work on stable/14 or main (without COMPAT_FREEBSD12): > >> > >> cloned_interfaces="vlan99" > >> ifconfig_vlan99="vlan 99 vlandev igb0" > >> > >> I have to replace the ifconfig line with: > >> > >> ifconfig_vlan99="vlan 99 vlandev igb0 vlanproto 802.1q" > >> > >> It also works if I remove the following lines in sys/net/if_vlan.c: > >> > >> 1110 #ifdef COMPAT_FREEBSD12 > >> 1113 #endif > >> > >> 2265 #ifdef COMPAT_FREEBSD12 > >> 2268 #endif > > > > Bug or user error? > > Seems to me as a regression in 14. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273539 -- Herbert From nobody Sun Sep 3 13:37:29 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rdt9v1NtKz4rWrj for ; Sun, 3 Sep 2023 13:37:39 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rdt9s07DHz3c9j for ; Sun, 3 Sep 2023 13:37:36 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 153.125.133.21) smtp.mailfrom=junchoon@dec.sakura.ne.jp; dmarc=none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 383DbTpA033190; Sun, 3 Sep 2023 22:37:30 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 3 Sep 2023 22:37:29 +0900 From: Tomoaki AOKI To: FreeBSD-STABLE Mailing List Cc: Warner Losh , Jake Freeland Subject: Re: Is there any plan for ZFS and timerfd updates on stable/14? Message-Id: <20230903223729.f97f8f9aebe4db561b1c63bc@dec.sakura.ne.jp> In-Reply-To: <20230903133328.54577b85b097da319ecde4ba@dec.sakura.ne.jp> References: <20230903123028.4ffceb705824f86d2efc21e3@dec.sakura.ne.jp> <20230903133328.54577b85b097da319ecde4ba@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spamd-Bar: / X-Spamd-Result: default: False [-0.49 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.993]; MV_CASE(0.50)[]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[sakura.ne.jp]; BLOCKLISTDE_FAIL(0.00)[153.125.133.21:server fail,123.1.88.210:server fail]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; HAS_ORG_HEADER(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org] X-Rspamd-Queue-Id: 4Rdt9s07DHz3c9j On Sun, 3 Sep 2023 13:33:28 +0900 Tomoaki AOKI wrote: > On Sat, 2 Sep 2023 22:47:53 -0500 > Jake Freeland wrote: > > > On Sat, Sep 2, 2023 at 10:40 PM Warner Losh wrote: > > > > > > > > > > > On Sat, Sep 2, 2023, 9:36 PM Jake Freeland > > > wrote: > > > > > >> On Sat, Sep 2, 2023 at 10:31 PM Tomoaki AOKI > > >> wrote: > > >> > > >>> Hi. > > >>> > > >>> There are discussions about deadlocks issue of ZFS on freebsd-current > > >>> ML, starting from [1] last month. > > >>> IIRC, at least some fixes (candidates?) are merged to main, but not yet > > >>> to stable/14. > > >>> > > >>> Upcoming (aleready released? or still rc3?) OpenZFS 2.2-release seems > > >>> to have most of them. So my 1st question is "Is there any plan to > > >>> import vendor/openzfs/zfs-2.2-release into stable/14 BEFORE BRANCHING > > >>> releng/14? > > >>> > > >>> And one more. timerfd is added at last-minutes BEFORE stable/14 is > > >>> branched, and already have not-yet-MFC'ed fixes [2], [3], [4] and > > >>> Differential revision D41600 on Phablicator [5] related to memory leaks > > >>> and locks. > > >>> Additionally, splitting out lib32 part to proper place is proposed > > >>> as D41640 [6]. Both [5] and [6] are accepted but not yet landed. > > >>> Also, D41641 [7] proposes namespace pollution adjustments. This can be > > >>> optional? > > >>> > > >>> Memory leaks and improper locks can lead system to security issues or > > >>> deadlocks, so it would be benefical if landed and MFC'ed BEFORE > > >>> releng/14 branches. > > >>> > > >>> Is there any plan to do so? At least, existing deadlocks should be > > >>> considered as SHOW-STOPPER and resolved. > > >>> > > >> > > >> The plan is to get all of those patches in before releng/14.0, I believe. > > >> > > >> What are your thoughts, Warner? > > >> > > > > > > Sounds like the reviews are done or nearly so. I've not had time to look > > > closely to be sure... I'd planned on making time Tuesday morning. > > > > > > > Yes. All reviews are good to go. > > > > Jake Freeland > > Glad to know. Thanks! > Looking forward to see them landed / MFC'ed before releng/14 branches. > > Regards. And glad to see OpenZFS 2.2-release is imported to stable/14 today as commit f789381671a3f97496d496d8f996a67eaa8b1a33 by mm@. :-) > > > > > > > > > > > Warner > > > > > > > > >> Thanks, > > >> Jake Freeland > > >> > > >> > > >>> > > >>> I myself am bitten by several deadlocks on poudriere full builds after > > >>> upgrading base from stable/13 to stable/14, finally finished with > > >>> increasing kern.maxvnodes after powercycle on each deadlock and > > >>> continue. > > >>> > > >>> > > >>> Thanks in advance! > > >>> > > >>> [1] > > >>> > > >>> https://lists.freebsd.org/archives/freebsd-current/2023-August/004162.html > > >>> > > >>> [2] > > >>> > > >>> https://cgit.freebsd.org/src/commit/?id=02f534b57f84d6f4f97c337b05b383c8b3aaf18c > > >>> > > >>> [3] > > >>> > > >>> https://cgit.freebsd.org/src/commit/?id=5eab523053db79b4bd4f926c7d7ac04444d9c1da > > >>> > > >>> [4] > > >>> > > >>> https://cgit.freebsd.org/src/commit/?id=f4296cfb409a48de00bfa60e76f686c2b031876f > > >>> > > >>> [5] https://reviews.freebsd.org/D41600 > > >>> > > >>> [6] https://reviews.freebsd.org/D41640 > > >>> > > >>> [7] https://reviews.freebsd.org/D41641 > > >>> > > >>> -- > > >>> Tomoaki AOKI > > -- > Tomoaki AOKI -- Tomoaki AOKI From nobody Tue Sep 5 00:47:56 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rfn172Kbxz4sYGT for ; Tue, 5 Sep 2023 00:48:11 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rfn171qCrz4c1K for ; Tue, 5 Sep 2023 00:48:11 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1693874891; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=y3vJ/mreyZDCsElqUfa++/6B1Jb09sXGRJPgcKtbMio=; b=K8/anbGbYclT4ZolbFcmHNEPB4wl+COaN5d/ju6DYs+MdnU8ynxw4o/tFOnhhF6wF4yzGc wW78OT2LzWVwZIFu/ZsdCgRcakiXiXXlxJb/LLw4OhLS08TFwbFtF9dh4kFPmC2fAiOcKL 4RO98/lJ34Xb99GApnoW5t6vZ3Z4Jk4dep915tjW3OeG+XM8QBHTFl+dnG02kBvPtdXDQ/ xBfDdf6mGtfOoMIydqlkw2xznVlB7fD+++Q9DPW4D8EW0FXEDCgSnAkDnQlD9qbvspZh38 Sxdk5JdYN8eP7pf/g3sUSVVYy/tZVy/yd1n77D42/fCjRwSc7v0xH9jFEI6WVg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1693874891; a=rsa-sha256; cv=none; b=Jr70H3jy//I8qG8eGfgX97ZR+xRw0+ygKRr2hykEz5cin8uzIhcGXtky3NQORpK7kt+nS6 Y5z53KZ4Myeu9cTb/pXf72W9/Gxe489y1LZwkX02wu1d3Sm++iZW1j/T3LjUM/dv9xgna6 uNz7TjLPSfU/LfFIp8PSZ+hxyhjRNHbTKwlcetK1o167Ohzeqrz7FifEkfS+RqAuUPHP5H PBXN/9oqAynKj6ad/R1UbujEClooi322OVaSg57o6oOL+hSd5SnmWGubJzOT0zCaObGtjL GyTWvJYmwF3DsbecMrEw36FrnjEocxLgljqKAco9EF9VgPOQjTwJtmQmi8F4FA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1693874891; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=y3vJ/mreyZDCsElqUfa++/6B1Jb09sXGRJPgcKtbMio=; b=dy48AG3QutguhYlRk1RYsJAhhAdcbkeYiQ50unRPknFmGF0IKBqshkOj5dCwUMpVITA663 7+XTT+8488NzvlyU8TtQiJCbP+tjWU+oN+x9MmxtMKX6Bd4fKsy0+Dg7BC9n3yw/xpEMme /a8bT7N7nz2xTdUle3GYJ1dzARSmtmKopvMdpxbxPf5+7ln+rWv1fN+KRnKf1+wfGd8lqY 0FKjT2KmtapxcQ4bPJzLx1htD5gsFZsqbbFm6qtYO+Z3xfzEEJy8gq5ediQDxczBhcB/8O as4BUjAWzbl0gywR+dHfsjhqxsW+iB1ulUhdjCSFkDdCIx44q7RU+yoChsV+/Q== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Rfn162zlmz11m0 for ; Tue, 5 Sep 2023 00:48:10 +0000 (UTC) (envelope-from zlei@FreeBSD.org) From: Zhenlei Huang Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.4\)) Subject: vm_fault_lookup: fault on nofault entry, while loading module cc_cubic with GENERIC-KASAN Message-Id: <7088027B-74F7-4622-95CF-DF2CEBBB8A6E@FreeBSD.org> Date: Tue, 5 Sep 2023 08:47:56 +0800 To: stable@freebsd.org X-Mailer: Apple Mail (2.3696.120.41.1.4) Hi, Observed unexpected kernel panic while loading modules. I have a test VM installed 13.2. I have `cc_cubic_load=3DYES` in /boot/loader.conf to test cc_cubic. Recently I installed custom=20 GENERIC-DEBUG kernel (current/15) to test some features and the kernel panics during early boot. To narrow down the root cause, I tested stable/14 (d6fec2dacf80) with stock kernel config GENERIC-KASAN, it still panics. I can reliably repeat the panic with `options KASAN`. I'm not familiar with KASAN, so post here. Steps to repeat: On current/15 or stable/14 built with `options KASAN`. # kldload cc_cubic interface cubic.2 already present in the KLD 'kernel'! panic: vm_fault_lookup: fault on nofault entry, addr: 0xfffffe0061b0f000 cpuid =3D 1 time =3D 1693873182 KDB: stack backtrace: #0 0xffffffff813419b3 at kdb_backtrace+0x103 #1 0xffffffff81287ced at vpanic+0x1fd #2 0xffffffff81287ae5 at panic+0xb5 #3 0xffffffff819b1db0 at vm_fault+0x2e80 #4 0xffffffff819aedff at vm_fault_trap+0xdf #5 0xffffffff81c27c38 at trap_pfault+0x378 #6 0xffffffff81c2696b at trap+0x4db #7 0xffffffff81be4c08 at calltrap+0x8 Uptime: 23s Dumping 162 out of 951 = MB:..10%..20%..30%..40%..50%..60%..70%..79%..89%..99% Dump complete Some informations that may help: loaded modules: root@:~ # kldstat=20 Id Refs Address Size Name 1 11 0xffffffff80200000 34b4cd8 kernel 2 1 0xffffffff83e19000 7208 intpm.ko 3 1 0xffffffff83e21000 39a8 smbus.ko 4 1 0xffffffff83e25000 cd10 vmci.ko 5 1 0xffffffff83e32000 3428 mac_ntpd.ko part of dmesg: ---<>--- Copyright (c) 1992-2023 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.0-ALPHA4 amd64 1400097 #0 stable/14-n265029-d6fec2dacf80: Mon = Sep 4 16:32:22 CST 2023 = zlei@:/usr/obj/home/zlei/freebsd-src-stable14/amd64.amd64/sys/GENERIC-KASA= N amd64 FreeBSD clang version 16.0.6 (https://github.com/llvm/llvm-project.git = llvmorg-16.0.6-0-g7cbf1a259152) VT(vga): text 80x25 CPU: Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz (2700.00-MHz K8-class CPU) Origin=3D"GenuineIntel" Id=3D0x306d4 Family=3D0x6 Model=3D0x3d = Stepping=3D4 = Features=3D0xf83fbff = Features2=3D0xfffa3203 AMD Features=3D0x2c100800 AMD Features2=3D0x121 Structured Extended = Features=3D0x1c27ab Structured Extended = Features3=3D0xbc000400 XSAVE Features=3D0x1 IA32_ARCH_CAPS=3D0xc TSC: P-state invariant Hypervisor: Origin =3D "VMwareVMware" real memory =3D 1073741824 (1024 MB) avail memory =3D 801873920 (764 MB) Best regards, Zhenlei From nobody Tue Sep 5 17:36:49 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RgCPJ4BdLz4sNYB for ; Tue, 5 Sep 2023 17:37:08 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RgCPJ1D2dz3LsG for ; Tue, 5 Sep 2023 17:37:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1693935425; bh=WizMlhbA8XmNvf+TVUUV6QvjrMPRw5nIVZYX6eVFDu4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=MmW2HPbnOjHrbWleAMhPZ5cskSHz/1yNQASwW3Aj0RpLgXSLfBsxgg1NHPiYcX6oCnkExUBp8H9LmeiKDrhul0i5Amz7To/PXnSyLsgdwtjQTr3kPbyCZK4XFb0e0FjxSDjazpOG7rVG9Q3vVCFU8qf/LXfZQHv+gk7s1Dll+Bi6BuQiUz95IvDebJUhr17yJNSzqFDDRBdSOQmlLR+qfhy/mRFbjHOEy30mUFoye/gzG4eCxAiXq4kYk/j3hRD+1CD1yJKzr2oiJbIxlpqJzI738cAAHalFJhRjobTYS45UDlDQPa6WGtfNzluqFbwbrylTvrZtFFTx+iBsgkMQxA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1693935425; bh=G5g+mGjVM9SO93/xK7USM6Dp59z5BN02gUOOPXSLXzB=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=S1bB4R2D9SBjiYdjF836xDxlCuMoa8s8LvK4oM4CwAM2V1BVGX4kmb/nMUXLy/fcl2eO9Mfq0zTcjYQDBYAcvKeJCVE3j4H57VYDYtvVAKp9b/5IzUgQ/0XLpUhsIOC0c81j40SFRKqwL5/SK3pgoWS22hKet+MOapSkDWJYy+uIiBG9rFMqtxdrvNyJQ3FXrfywSxNg8YU7v5s5Y1Iuhi2ZjujAbOdQR4fhqmXJns15iumnfXpUj74+pLd6PPclhGUTbqXFiWg2ZbQt0wTm2pxGFYGXZZCMiOCGQYXR5GPzQDYJEO4bWB82kwUtbeKHngMuqSXOIZBhdQ1gSLfikA== X-YMail-OSG: Nh1i.m4VM1lrJJ1nfrRZiVOy3x8HVK2kTJ8SQkG9HhtMj3W74uhAh5tO5EXqDJh hn62Vx7yJQwFAGzniUOQTBj2SEgQZ10BBr2NsAZ8fyNmPOojKR2EuRNXN.UcH5D_sraIxIl8g1Og U2oo.UD0uPnremwsgR41yRbfQVzgRuBAxN6UXDKpB0.ncB0XN2h0SVMi6ZEsdIgtihMnjLjTkJ5j k6p.VHAvFTzg8TD3oq0jOuS270eq_w9OCdAb9e7VfaKc1GdB_y0Ut4wdklfK.H30qsY9m2jLLNpX jo2c4ehr9etSErirfAOJKNO.CTJIYAryfyhhJy1fCXwDCNriWFE0BiC7zBTECgYQ.jKFJGeO1zcq FGLw4kj_f56AD3FSB2cHdu3WZSbD0IPS9xBIgcwxH4ZfDEIL2nFI_GstON.OzBF3qxwaZX_AKFUR 8aJfRMv6.XPgAJV.sG90UFD0bL7VJW5QdKxNZVWhXWc1qrsf2_MzfYt73s2E_V3spZfhsoWlZEcK Ls11_HwotjJoe2aDbUj1KYpaqLHr28FjKibxqTglTnFj53C1t8PFZKPKg0C.a4tw3RIoDNoZEEp8 lejKHIrNVLExLdhr_Bvt5_PE8Ae.mf3X6XyOLU2z3pd6puUHJtZv54KE84cDilNG0Befl0XqsSAg IbpubTxIUnn37ho8ykRtMp9n7s0P2.GYOYFfTB5HuyEg4ywpj3Lw4191S1SPvRB8xw_4ug5hZbdR sGAW8ZX4_BeGt2Xzz_xHNP1a8t9w4.4kUJOYOn2Vw5t9oEZ90lb2vgQPBLjoA3H8ONKJu9d4k4iH Kbj5TbRxdlzQ15.HqiXtHX6yNhI5D0oAl2PwXXgUy0zo74uPAITVGKZwD1GHb1hPoq7QbsEzBqQ. Gqa5xgWiH_l9Xz.SHHEnoNww8b47G9TQdfEnB9h9WyFKTEabfJNQPzyeu.okLUx4myNe1gxud7GQ nb1EgIMBOATO9jIIGDAqvaDsDWXv94EqG1CQWnQYGmQw2NgScZ1a0k3QU7Vr2dUwNPMuqCcyDjAB FXgTRMfmGhA3_9zFx5rhzwFo01WHbtN_H3zB5a2UNXg55RqrzOrL8M.p9B6QTFcL0NurYP6.rALD 3jGPzmz2wsgVfEU1YZIRAciRl9N.Tdky0_YdnVUKpfKmzfeQfEYy81pxizEy9uGR4CIi4tH8OcEW 9uEZWIzSGOpGGn1D6HWLjb4taTesyIp3QH.2kcuClhsk63nOD9YUsvWMKiMfkXJoDgj01u0mEIg0 mSSdoUNL3k_3XRsfx3mRdEIdI_8FsDFP8BY60_CmaoPxGN4eOxH5ZuT7MewHtIRhJsCXXSTd_lP6 QXBK3tHXfZ7KA4pYKAwgRi_mLDWE3kRnKpzvnr_FlULVI.uAOTufjcUSyXvDD9psUR68IpKJQ9vZ UCAcB7.b8TFSN3oA.iRZisniJZNP2RcPYQCJN15eNQWdUQZ24caeOerSv87xiaCNPFtP8n5B3Er5 GyJYr_KdIIQnmn3MFCPZUbGaE4wnc5To8BDfQapvfo9PRG.D3sFXbDJXx_Syde67QHbXKotmazRc EbLzwHxsgZde5XqhI5TSZtbQJgZpWue8Z6hqAvw6bYS7yJDiWi_ukjal_Y9e9gYV56mzumU80NoJ QuJ1hBxpQW9_kdEHJcrdFEPmE914BH_OpfFlUmB1iA1VfE6VVGaJgV0SthbMUgUUWVfG2WK0ZRj. JBG4lNyb8Lhkc8_fRSr5RBAbaidmhH_XIYTa7w6HP68OZnCzAs8TTuzZVMbdq98f.tUSqbW_yerC hnCEmjIIcRRYVurkuYQavUbdsTUhiD87akl9yRWWSV0ypwILsdLeLPNT0j0DQlO_dv7zXcdnDk40 4FlcgjCE5dCTpl25UNbvz8uYOYzDDtlF.HnuCSixW6.r2yJWeozd92eIoe_fz7fx36rhQS8J_V8z 7NyAkFWqcQquBtBlGZpzOZdwV7zrbdPEeY5tgLiOJ2cgBwNWbxJjpjTRrZNpMSm1iwkKJ8MW0irO jyt0SVHdQNtvoMjYmEGhvS6RMu9Yo42QDQCmzL1AY_pcYDvo.fM3bAkQktgPjCEVJF6HhG3lQb8V f41MiMB3CiOuZj.caW2znV.5UxUiV9oI5jAlImvojPw4ZTY_A7snoXmKUUWdTCoM7uUEfQUSbdni OcNplOtQKW14hWUMBBb8.oS_IaRHb1x9JY.MHHTBpcx2f9HSIP.mB6IgDa2igLEx83nNpZdiZeMu RX.Ju2rs6GYz51VL_LA9dDVillcTwpwOee25t_iNWKKUdpUglTiVZUlbUxYYUy85nuCEN.pRSwup 2sgZv8M9UqMyR X-Sonic-MF: X-Sonic-ID: 42a91997-168d-4032-9144-e546820e5c76 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Tue, 5 Sep 2023 17:37:05 +0000 Received: by hermes--production-bf1-865889d799-7vf9r (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 500fd4fe3208a4db60ddc2bdb0062e20; Tue, 05 Sep 2023 17:37:02 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Possible regression in main causing poor performance From: Mark Millard In-Reply-To: <20230905155812.D746B71@slippy.cwsent.com> Date: Tue, 5 Sep 2023 10:36:49 -0700 Cc: Glen Barber , Current FreeBSD , FreeBSD-STABLE Mailing List , freebsd-arch , Martin Matuska Content-Transfer-Encoding: quoted-printable Message-Id: References: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> <1B47E578-693D-4690-A577-947E8C9140B5@yahoo.com> <1C3E6B34-2937-46C6-862E-5B988B8A3A43@yahoo.com> <20230830184426.GM1219@FreeBSD.org> <20230830204406.24FDC7E@slippy.cwsent.com> <20230905155812.D746B71@slippy.cwsent.com> To: Cy Schubert , Alexander Motin X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4RgCPJ1D2dz3LsG On Sep 5, 2023, at 08:58, Cy Schubert wrote: > In message <20230830204406.24FDC7E@slippy.cwsent.com>, Cy Schubert = writes: >> In message <20230830184426.GM1219@FreeBSD.org>, Glen Barber writes: >>>=20 >>>=20 >>> On Mon, Aug 28, 2023 at 06:06:09PM -0700, Mark Millard wrote: >>>> Has any more been learned about this? Is it still an issue? >>>> =3D20 >>>=20 >>> I rebooted the machine before the ALPHA3 builds with no other = changes, >>> and the overall times for 14.x builds went back to normal. I do not >>> like to experiment with builders during a release cycle, but as we = are >>> going to have 15.x snapshots available moving forward, I will not = reboot >>> that machine next week in hopes to get some useful data. >>>=20 >>> If my memory serves correctly, mm@ has a pending ZFS import from >>> upstream for both main and stable/14 pending. Whether or not that = will >>> resolve any issue here, I do not know. >>=20 >> Two of my poudriere builder machines have experienced different = panics=20 >> since the ZFS import two days ago. The problems have been documented = on the=20 >> -current list. >=20 > Just an update. >=20 > The three pull requests amotin@ pointed to did resolve all my = problems. A=20 > subsequent update which included the latest ZFS commits worked just as=20= > well, without any new regressions. AFAIAC this problem has been = resolved. >=20 > The random email corruptions have also been resolved. >=20 >=20 > --=20 > Cheers, > Cy Schubert > FreeBSD UNIX: Web: https://FreeBSD.org > NTP: Web: https://nwtime.org >=20 > e^(i*pi)+1=3D0 >=20 >=20 >=20 >=20 > =C2=9C9O8 The just-above quoted line looks like a corruption to me. Otherwise, I'm just reporting more evidence from separate testing on amd64 . . . I will say that my separate-install/boot environment 10hr, 6366 port->package poudriere bulk -a prefix test of: # uname -apKU FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #118 = main-n265152-f49d6f583e9d-dirty: Mon Sep 4 14:26:56 PDT 2023 = root@amd64_ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.a= md64/sys/GENERIC-NODBG amd64 amd64 1500000 1500000 did not show any deadlocks. The only oddity that I've noticed is the 1 extra message shown in: . . . [00:03:25] [32] [00:00:00] Builder starting [00:03:43] [01] [00:00:18] Finished print/indexinfo | indexinfo-0.3.1: = Success [00:03:43] [01] [00:00:00] Building devel/gettext-runtime | = gettext-runtime-0.22_1 [00:05:20] [01] [00:01:37] Finished devel/gettext-runtime | = gettext-runtime-0.22_1: Success 23/.p/cleaning/rdeps/gettext-runtime-0.22_1/chemtool-1.6.14_4 copy: open = failed: No such file or directory [00:05:23] [01] [00:00:00] Building devel/gmake | gmake-4.3_2 [00:05:55] [02] [00:02:30] Builder started . . . I'm comfortable moving my normal environments forward to include this latest import of openzfs. The effort established a separate environment set up for doing testing of jumping to/past an openzfs import(s) in main. Too many recent imports have dangerous-to-the-file-system and/or had deadlocking issues for me to simply update to include them without first testing on separate media that does not have to stay operational. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Sep 5 17:55:33 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RgCpd66MBz4sXgN; Tue, 5 Sep 2023 17:55:37 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta001.cacentral1.a.cloudfilter.net (omta001.cacentral1.a.cloudfilter.net [3.97.99.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RgCpd3rCfz3TSw; Tue, 5 Sep 2023 17:55:37 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Authentication-Results: mx1.freebsd.org; none Received: from shw-obgw-4002a.ext.cloudfilter.net ([10.228.9.250]) by cmsmtp with ESMTP id dY0fqQ9DcLAoIdaH1qWzl3; Tue, 05 Sep 2023 17:55:35 +0000 Received: from spqr.komquats.com ([70.66.152.170]) by cmsmtp with ESMTPA id daGzqMvznyAOedaH0qdblx; Tue, 05 Sep 2023 17:55:35 +0000 X-Authority-Analysis: v=2.4 cv=e5oV9Il/ c=1 sm=1 tr=0 ts=64f76b97 a=y8EK/9tc/U6QY+pUhnbtgQ==:117 a=y8EK/9tc/U6QY+pUhnbtgQ==:17 a=kj9zAlcOel0A:10 a=zNV7Rl7Rt7sA:10 a=CjxXgO3LAAAA:8 a=YxBL1-UpAAAA:8 a=VxmjJ2MpAAAA:8 a=6I5d2MoRAAAA:8 a=EkcXrb_YAAAA:8 a=3rs0Dif0S-5JwpwNab0A:9 a=CjuIK1q_8ugA:10 a=UJ0tAi3fqDAA:10 a=26XcPLqRqocA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=7gXAzLPJhVmCkEl4_tsf:22 a=IjZwj45LgO3ly-622nXo:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id 7F9C7350; Tue, 5 Sep 2023 10:55:33 -0700 (PDT) Received: by slippy.cwsent.com (Postfix, from userid 1000) id 7589E1A8; Tue, 5 Sep 2023 10:55:33 -0700 (PDT) X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.8+dev Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Mark Millard cc: Cy Schubert , Alexander Motin , Glen Barber , Current FreeBSD , FreeBSD-STABLE Mailing List , freebsd-arch , Martin Matuska Subject: Re: Possible regression in main causing poor performance In-reply-to: References: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> <1B47E578-693D-4690-A577-947E8C9140B5@yahoo.com> <1C3E6B34-2937-46C6-862E-5B988B8A3A43@yahoo.com> <20230830184426.GM1219@FreeBSD.org> <20230830204406.24FDC7E@slippy.cwsent.com> <20230905155812.D746B71@slippy.cwsent.com> Comments: In-reply-to Mark Millard message dated "Tue, 05 Sep 2023 10:36:49 -0700." List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 05 Sep 2023 10:55:33 -0700 Message-Id: <20230905175533.7589E1A8@slippy.cwsent.com> X-CMAE-Envelope: MS4xfMFnni6gXWmeGiOT8uBYC/30mLL7MIb0STyhxhMf+Dzs/JF0ePlmym7eSlnpOSMnTB3r8RtCkqJ6r+mZJqmMjtlKLOL2KZKr3pSAJ+aYQRdKb8DLSHk9 HIfq/R2axOlQ3pBETDstKv/W9nY1pLC24iSJ7nUXNTlPKjOkYO4u8L08a3SJzJSbdp8cpQlfFrh9njfgwzgRdEB4ktlO5MnMGJlaPK2PQxpEYpJ9f7ZTGe9r hX/Y0mlfkBgJqKGp5uWzjSaS3TvyT4uuE8tF43cD2hhyrE4xwVxzIVsIChO2WMuDaKN/pnZuK92xTHVYfpg4YA/CBl9xGsmhNPEIvFHt0bsF5vteeSptQsoq h/X3CNC9dajejmmWYtCyOiPZ3gB9OhyYhfFxKU+qBFUcN2xJVhg= X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US] X-Rspamd-Queue-Id: 4RgCpd3rCfz3TSw In message , Mark Millard write s: > On Sep 5, 2023, at 08:58, Cy Schubert wrote: > > > In message <20230830204406.24FDC7E@slippy.cwsent.com>, Cy Schubert = > writes: > >> In message <20230830184426.GM1219@FreeBSD.org>, Glen Barber writes: > >>>=20 > >>>=20 > >>> On Mon, Aug 28, 2023 at 06:06:09PM -0700, Mark Millard wrote: > >>>> Has any more been learned about this? Is it still an issue? > >>>> =3D20 > >>>=20 > >>> I rebooted the machine before the ALPHA3 builds with no other = > changes, > >>> and the overall times for 14.x builds went back to normal. I do not > >>> like to experiment with builders during a release cycle, but as we = > are > >>> going to have 15.x snapshots available moving forward, I will not = > reboot > >>> that machine next week in hopes to get some useful data. > >>>=20 > >>> If my memory serves correctly, mm@ has a pending ZFS import from > >>> upstream for both main and stable/14 pending. Whether or not that = > will > >>> resolve any issue here, I do not know. > >>=20 > >> Two of my poudriere builder machines have experienced different = > panics=20 > >> since the ZFS import two days ago. The problems have been documented = > on the=20 > >> -current list. > >=20 > > Just an update. > >=20 > > The three pull requests amotin@ pointed to did resolve all my = > problems. A=20 > > subsequent update which included the latest ZFS commits worked just as=20= > > > well, without any new regressions. AFAIAC this problem has been = > resolved. > >=20 > > The random email corruptions have also been resolved. > >=20 > >=20 > > --=20 > > Cheers, > > Cy Schubert > > FreeBSD UNIX: Web: https://FreeBSD.org > > NTP: Web: https://nwtime.org > >=20 > > e^(i*pi)+1=3D0 > >=20 > >=20 > >=20 > >=20 > > =C2=9C9O8 > > The just-above quoted line looks like a corruption to me. > Hmm. Just to rule out that a build of the exmh2 and nmh-devel packages might have been corrupt, I've rebuilt the two and will continue to monitor. This email was sent by a rebuilt exmh2 and nmh-devel. -- Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=0 From nobody Wed Sep 6 18:14:10 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rgr9r22RWz4sk3F for ; Wed, 6 Sep 2023 18:14:24 +0000 (UTC) (envelope-from augustnee2@proton.me) Received: from mail-40135.protonmail.ch (mail-40135.protonmail.ch [185.70.40.135]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rgr9p1pLnz4NLV for ; Wed, 6 Sep 2023 18:14:21 +0000 (UTC) (envelope-from augustnee2@proton.me) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=proton.me header.s=protonmail header.b=D+FjQRnR; spf=pass (mx1.freebsd.org: domain of augustnee2@proton.me designates 185.70.40.135 as permitted sender) smtp.mailfrom=augustnee2@proton.me; dmarc=pass (policy=quarantine) header.from=proton.me Date: Wed, 06 Sep 2023 18:14:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1694024058; x=1694283258; bh=Jt01AQqDmZgjpRSvGOHJzl+o7uAluYxYvfm64Pv1C8c=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=D+FjQRnRHbzhPfyBzKDHzYoLwMKqTrg8cjgTrNgUuLOObpm8h3nI+GfERm5EJsPbW ezrnPjAFuywCabCYsQPySrVnpD6UmpHO9s2PZm+uU8aMJAkzQtfS5qXLa1LsLK+deA 7uIg4kySCCEU93B8r6g0im5zJJFA6ww4rpyOLVWs3hCjbcx/ChcFZSOm2tiCwDXsMv HEZtDUTZVyOXPoB5ywGDPYULzVJ+SdtthtaTvTmK5MgeQv1n7+PQ3ysm7mTJHHciID IGpuGBoaYCIsHF2zKKeuhNozNcyeLI6GcwNVvHTI8xWODK/eb5Pbib/vkjU2YSOtvn qkS972EzSyPZA== To: "freebsd-stable@FreeBSD.org" From: augustnee2@proton.me Subject: Can't load i915kms Message-ID: <-772HML8UxXnbdWdYtRoMDD2FbmLQOMkrVNbqAuRSQJOyicRZxtzQLRLPc3_s2AlaD5fM_Z3JELrSUbUzRlY4_ueoRZgP6hQxiySi_G9CgY=@proton.me> Feedback-ID: 77259124:user:proton List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_NUdxBP1lfVk0x3ZzS9VJW8wwMrS6iVf5PbnOTyzJdtA" X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.62 / 15.00]; MIME_BASE64_TEXT_BOGUS(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.79)[-0.789]; NEURAL_HAM_LONG(-0.53)[-0.534]; DMARC_POLICY_ALLOW(-0.50)[proton.me,quarantine]; RWL_MAILSPIKE_EXCELLENT(-0.40)[185.70.40.135:from]; R_DKIM_ALLOW(-0.20)[proton.me:s=protonmail]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; MIME_BASE64_TEXT(0.10)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_EQ_ADDR_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@FreeBSD.org]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[proton.me:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_NO_DN(0.00)[]; HAS_PHPMAILER_SIG(0.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Rgr9p1pLnz4NLV This is a multi-part message in MIME format. --b1_NUdxBP1lfVk0x3ZzS9VJW8wwMrS6iVf5PbnOTyzJdtA Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 SGkhIEkgaGF2ZSBhIHByb2JsZW0gd2l0aCBsb2FkaW5nIHRoZSBHUFUgZHJpdmVyIGludG8gdGhl IGtlcm5lbCBhZnRlciB1cGdyYWRpbmcgZnJvbSAxNC1BTFBIQTIuIFN0ZXBzIEkgbWFkZSB0byB1 cGdyYWRlIG15IHN5c3RlbToKJCBybSAtcmYgL3Vzci9zcmMKJCBnaXQgY2xvbmUgLWIgc3RhYmxl LzE0IGh0dHBzOi8vZ2l0LmZyZWVic2Qub3JnL3NyYy5naXQgL3Vzci9zcmMKJCBtYWtlIGJ1aWxk d29ybGQga2VybmVsCiQgcmVib290CkluIHNpbmdsZSB1c2VyIG1vZGU6CiQgbW91bnQgLXUgLwok IHpmcyBtb3VudCB6cm9vdC91c3Ivc3JjCiQgZXRjdXBkYXRlIC1wCigkIGV0Y3VwZGF0ZSByZXNv bHZlKQokIG1ha2UgaW5zdGFsbHdvcmxkCiQgZXRjdXBkYXRlIC1CICYmIHJlYm9vdApJIHRyaWVk IHRvIHJlaW5zdGFsbCBhbGwgbXkgcGFja2FnZXMgKGluY2x1ZGluZyBwa2cpIGFuZCBpbnN0YWxs IGRybS01MTUta21vZCBmcm9tIHBvcnRzLCBidXQgbm90aGluZyBjaGFuZ2VkLgpCYXNpYyBpbmZv IGFib3V0IHRoZSBzeXN0ZW06IGh0dHBzOi8vcGFzdGViaW4uY29tL2c3WVJxWUJLCi92YXIvbG9n L21lc3NhZ2VzOgpTZXAgNSAxNDo1OToxOCBic2RsYXAga2VybmVsOiBkcm1uMDogPGRybW4+IG9u IHZnYXBjaTAKU2VwIDUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDogdmdhcGNpMDogY2hpbGQgZHJt bjAgcmVxdWVzdGVkIHBjaV9lbmFibGVfaW8KU2VwIDUgMTQ6NTk6MTggYnNkbGFwIHN5c2xvZ2Q6 IGxhc3QgbWVzc2FnZSByZXBlYXRlZCAxIHRpbWVzClNlcCA1IDE0OjU5OjE4IGJzZGxhcCBrZXJu ZWw6IFtkcm1dIFVuYWJsZSB0byBjcmVhdGUgYSBwcml2YXRlIHRtcGZzIG1vdW50LCBodWdlcGFn ZSBzdXBwb3J0IHdpbGwgYmUgZGlzYWJsZWQoLTE5KS4KU2VwIDUgMTQ6NTk6MTggYnNkbGFwIGtl cm5lbDogW2RybV0gR290IHN0b2xlbiBtZW1vcnkgYmFzZSAweGNkODAwMDAwLCBzaXplIDB4MjAw MDAwMApTZXAgNSAxNDo1OToxOCBic2RsYXAga2VybmVsOiB2Z2FwY2kwOiBhdHRlbXB0aW5nIHRv IGFsbG9jYXRlIDEgTVNJIHZlY3RvcnMgKDEgc3VwcG9ydGVkKQpTZXAgNSAxNDo1OToxOCBic2Rs YXAga2VybmVsOiBtc2k6IHJvdXRpbmcgTVNJIElSUSAxMzQgdG8gbG9jYWwgQVBJQyAwIHZlY3Rv ciA1MApTZXAgNSAxNDo1OToxOCBic2RsYXAga2VybmVsOiB2Z2FwY2kwOiB1c2luZyBJUlEgMTM0 IGZvciBNU0kKU2VwIDUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDogc2tsX2RtY192ZXIxXzI3LmJp bjogY291bGQgbm90IGxvYWQgZmlybXdhcmUgaW1hZ2UsIGVycm9yIDIKU2VwIDUgMTQ6NTk6MTgg YnNkbGFwIGtlcm5lbDogaTkxNS9za2xfZG1jX3ZlcjFfMjcuYmluOiBjb3VsZCBub3QgbG9hZCBm aXJtd2FyZSBpbWFnZSwgZXJyb3IgMgpTZXAgNSAxNDo1OToxOCBic2RsYXAga2VybmVsOiBpOTE1 L3NrbF9kbWNfdmVyMV8yNy5iaW46IGNvdWxkIG5vdCBsb2FkIGZpcm13YXJlIGltYWdlLCBlcnJv ciAyClNlcCA1IDE0OjU5OjE4IGJzZGxhcCBrZXJuZWw6IGRybW4wOiBBZGRpbmcgaTJjIGFkYXB0 ZXIgaTkxNSBnbWJ1cyBkcGMKU2VwIDUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDogbGtwaV9paWMw OiA8TGludXhLUEkgSTJDPiBvbiBkcm1uMApTZXAgNSAxNDo1OToxOCBic2RsYXAga2VybmVsOiBp aWNidXMyOiA8UGhpbGlwcyBJMkMgYnVzPiBvbiBsa3BpX2lpYzAKU2VwIDUgMTQ6NTk6MTggYnNk bGFwIGtlcm5lbDogaWljMjogPEkyQyBnZW5lcmljIEkvTz4gb24gaWljYnVzMgpTZXAgNSAxNDo1 OToxOCBic2RsYXAga2VybmVsOiBkcm1uMDogQWRkaW5nIGkyYyBhZGFwdGVyIGk5MTUgZ21idXMg ZHBiClNlcCA1IDE0OjU5OjE4IGJzZGxhcCBrZXJuZWw6IGxrcGlfaWljMTogPExpbnV4S1BJIEky Qz4KU2VwIDUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDogb24gZHJtbjBTZXAgNSAxNDo1OToxOCBi c2RsYXAga2VybmVsOiBpaWNidXMzOiA8UGhpbGlwcyBJMkMgYnVzPiBvbiBsa3BpX2lpYzEKLi4u ClNlcCA2IDE1OjU1OjQ0IGJzZGxhcCBrZXJuZWw6IGlpYzA6IDxJMkMgZ2VuZXJpYyBJL08+IG9u IGlpY2J1czAKU2VwIDYgMTU6NTU6NDQgYnNkbGFwIGtlcm5lbDogaWljMTogPEkyQyBnZW5lcmlj IEkvTz4gb24gaWljYnVzMQpTZXAgNiAxNTo1NTo0NCBic2RsYXAga2VybmVsOiBkcm1uMDogPGRy bW4+IG9uIHZnYXBjaTAKU2VwIDYgMTU6NTU6NDQgYnNkbGFwIGtlcm5lbDogdmdhcGNpMDogY2hp bGQgZHJtbjAgcmVxdWVzdGVkIHBjaV9lbmFibGVfaW8KU2VwIDYgMTU6NTU6NDQgYnNkbGFwIHN5 c2xvZ2Q6IGxhc3QgbWVzc2FnZSByZXBlYXRlZCAxIHRpbWVzClNlcCA2IDE1OjU1OjQ0IGJzZGxh cCBrZXJuZWw6IFtkcm1dIFVuYWJsZSB0byBjcmVhdGUgYSBwcml2YXRlIHRtcGZzIG1vdW50LCBo dWdlcGFnZSBzdXBwb3J0IHdpbGwgYmUgZGlzYWJsZWQoLTE5KS5TZXAgNiAxNTo1NTo0NCBic2Rs YXAga2VybmVsOiBbZHJtXSBHb3Qgc3RvbGVuIG1lbW9yeSBiYXNlIDB4Y2Q4MDAwMDAsIHNpemUg MHgyMDAwMDAwCgpXaGF0IGNhbiBiZSB0aGUgcHJvYmxlbT8gVGhhbmsgeW91IQ== --b1_NUdxBP1lfVk0x3ZzS9VJW8wwMrS6iVf5PbnOTyzJdtA Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0 cHg7Ij5IaSEgSSBoYXZlIGEgcHJvYmxlbSB3aXRoIGxvYWRpbmcgdGhlIEdQVSBkcml2ZXIgaW50 byB0aGUga2VybmVsIGFmdGVyIHVwZ3JhZGluZyBmcm9tIDE0LUFMUEhBMi4gU3RlcHMgSSBtYWRl IHRvIHVwZ3JhZGUgbXkgc3lzdGVtOjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlh bCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+JCBybSAtcmYgL3Vzci9zcmM8YnI+PC9k aXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6 IDE0cHg7Ij4kIGdpdCBjbG9uZSAtYiBzdGFibGUvMTQgPGEgaHJlZj0iaHR0cHM6Ly9naXQuZnJl ZWJzZC5vcmcvc3JjLmdpdCI+aHR0cHM6Ly9naXQuZnJlZWJzZC5vcmcvc3JjLmdpdDwvYT4gL3Vz ci9zcmM8L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZv bnQtc2l6ZTogMTRweDsiPiQgbWFrZSBidWlsZHdvcmxkIGtlcm5lbDwvZGl2PjxkaXYgc3R5bGU9 ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+JCByZWJv b3Q8L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQt c2l6ZTogMTRweDsiPkluIHNpbmdsZSB1c2VyIG1vZGU6PC9kaXY+PGRpdiBzdHlsZT0iZm9udC1m YW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7Ij4kIG1vdW50IC11IC88 L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6 ZTogMTRweDsiPiQgemZzIG1vdW50IHpyb290L3Vzci9zcmM8YnI+PC9kaXY+PGRpdiBzdHlsZT0i Zm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7Ij4kIGV0Y3Vw ZGF0ZSAtcDxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2Vy aWY7IGZvbnQtc2l6ZTogMTRweDsiPigkIGV0Y3VwZGF0ZSByZXNvbHZlKTwvZGl2PjxkaXYgc3R5 bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+JCBt YWtlIGluc3RhbGx3b3JsZDwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fu cy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+JCBldGN1cGRhdGUgLUIgJmFtcDsmYW1wOyByZWJv b3Q8L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQt c2l6ZTogMTRweDsiPkkgdHJpZWQgdG8gcmVpbnN0YWxsIGFsbCBteSBwYWNrYWdlcyAoaW5jbHVk aW5nIHBrZykgYW5kIGluc3RhbGwgZHJtLTUxNS1rbW9kIGZyb20gcG9ydHMsIGJ1dCBub3RoaW5n IGNoYW5nZWQuPC9kaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlm OyBmb250LXNpemU6IDE0cHg7Ij48Yj5CYXNpYyBpbmZvIGFib3V0IHRoZSBzeXN0ZW08L2I+OiA8 c3Bhbj48YSB0YXJnZXQ9Il9ibGFuayIgcmVsPSJub3JlZmVycmVyIG5vZm9sbG93IG5vb3BlbmVy IiBocmVmPSJodHRwczovL3Bhc3RlYmluLmNvbS9nN1lScVlCSyI+aHR0cHM6Ly9wYXN0ZWJpbi5j b20vZzdZUnFZQks8L2E+PC9zcGFuPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTog QXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPjxiPi92YXIvbG9nL21lc3NhZ2Vz PC9iPjo8L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZv bnQtc2l6ZTogMTRweDsiPjxzcGFuPlNlcCAmbmJzcDs1IDE0OjU5OjE4IGJzZGxhcCBrZXJuZWw6 IGRybW4wOiAmbHQ7ZHJtbiZndDsgb24gdmdhcGNpMDwvc3Bhbj48ZGl2PjxzcGFuPlNlcCAmbmJz cDs1IDE0OjU5OjE4IGJzZGxhcCBrZXJuZWw6IHZnYXBjaTA6IGNoaWxkIGRybW4wIHJlcXVlc3Rl ZCBwY2lfZW5hYmxlX2lvPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+U2VwICZuYnNwOzUgMTQ6NTk6 MTggYnNkbGFwIHN5c2xvZ2Q6IGxhc3QgbWVzc2FnZSByZXBlYXRlZCAxIHRpbWVzPC9zcGFuPjwv ZGl2PjxkaXY+PHNwYW4+U2VwICZuYnNwOzUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDogW2RybV0g VW5hYmxlIHRvIGNyZWF0ZSBhIHByaXZhdGUgdG1wZnMgbW91bnQsIGh1Z2VwYWdlIHN1cHBvcnQg d2lsbCBiZSBkaXNhYmxlZCgtMTkpLjwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPlNlcCAmbmJzcDs1 IDE0OjU5OjE4IGJzZGxhcCBrZXJuZWw6IFtkcm1dIEdvdCBzdG9sZW4gbWVtb3J5IGJhc2UgMHhj ZDgwMDAwMCwgc2l6ZSAweDIwMDAwMDA8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj5TZXAgJm5ic3A7 NSAxNDo1OToxOCBic2RsYXAga2VybmVsOiB2Z2FwY2kwOiBhdHRlbXB0aW5nIHRvIGFsbG9jYXRl IDEgTVNJIHZlY3RvcnMgKDEgc3VwcG9ydGVkKTwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPlNlcCAm bmJzcDs1IDE0OjU5OjE4IGJzZGxhcCBrZXJuZWw6IG1zaTogcm91dGluZyBNU0kgSVJRIDEzNCB0 byBsb2NhbCBBUElDIDAgdmVjdG9yIDUwPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+U2VwICZuYnNw OzUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDogdmdhcGNpMDogdXNpbmcgSVJRIDEzNCBmb3IgTVNJ PC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+U2VwICZuYnNwOzUgMTQ6NTk6MTggYnNkbGFwIGtlcm5l bDogc2tsX2RtY192ZXIxXzI3LmJpbjogY291bGQgbm90IGxvYWQgZmlybXdhcmUgaW1hZ2UsIGVy cm9yIDI8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj5TZXAgJm5ic3A7NSAxNDo1OToxOCBic2RsYXAg a2VybmVsOiBpOTE1L3NrbF9kbWNfdmVyMV8yNy5iaW46IGNvdWxkIG5vdCBsb2FkIGZpcm13YXJl IGltYWdlLCBlcnJvciAyPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+U2VwICZuYnNwOzUgMTQ6NTk6 MTggYnNkbGFwIGtlcm5lbDogaTkxNS9za2xfZG1jX3ZlcjFfMjcuYmluOiBjb3VsZCBub3QgbG9h ZCBmaXJtd2FyZSBpbWFnZSwgZXJyb3IgMjwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPlNlcCAmbmJz cDs1IDE0OjU5OjE4IGJzZGxhcCBrZXJuZWw6IGRybW4wOiBBZGRpbmcgaTJjIGFkYXB0ZXIgaTkx NSBnbWJ1cyBkcGM8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj5TZXAgJm5ic3A7NSAxNDo1OToxOCBi c2RsYXAga2VybmVsOiBsa3BpX2lpYzA6ICZsdDtMaW51eEtQSSBJMkMmZ3Q7IG9uIGRybW4wPC9z cGFuPjwvZGl2PjxkaXY+PHNwYW4+U2VwICZuYnNwOzUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDog aWljYnVzMjogJmx0O1BoaWxpcHMgSTJDIGJ1cyZndDsgb24gbGtwaV9paWMwPC9zcGFuPjwvZGl2 PjxkaXY+PHNwYW4+U2VwICZuYnNwOzUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDogaWljMjogJmx0 O0kyQyBnZW5lcmljIEkvTyZndDsgb24gaWljYnVzMjwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuPlNl cCAmbmJzcDs1IDE0OjU5OjE4IGJzZGxhcCBrZXJuZWw6IGRybW4wOiBBZGRpbmcgaTJjIGFkYXB0 ZXIgaTkxNSBnbWJ1cyBkcGI8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj5TZXAgJm5ic3A7NSAxNDo1 OToxOCBic2RsYXAga2VybmVsOiBsa3BpX2lpYzE6ICZsdDtMaW51eEtQSSBJMkMmZ3Q7PC9zcGFu PjwvZGl2PjxkaXY+PHNwYW4+U2VwICZuYnNwOzUgMTQ6NTk6MTggYnNkbGFwIGtlcm5lbDogJm5i c3A7b24gZHJtbjA8L3NwYW4+PC9kaXY+PHNwYW4+U2VwICZuYnNwOzUgMTQ6NTk6MTggYnNkbGFw IGtlcm5lbDogaWljYnVzMzogJmx0O1BoaWxpcHMgSTJDIGJ1cyZndDsgb24gbGtwaV9paWMxPC9z cGFuPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9u dC1zaXplOiAxNHB4OyI+PHNwYW4+Li4uPGJyPjwvc3Bhbj48L2Rpdj48ZGl2IHN0eWxlPSJmb250 LWZhbWlseTogQXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPjxzcGFuPjxzcGFu PlNlcCA2IDE1OjU1OjQ0IGJzZGxhcCBrZXJuZWw6IGlpYzA6ICZsdDtJMkMgZ2VuZXJpYyBJL08m Z3Q7IG9uIGlpY2J1czA8L3NwYW4+PGRpdj48c3Bhbj5TZXAgNiAxNTo1NTo0NCBic2RsYXAga2Vy bmVsOiBpaWMxOiAmbHQ7STJDIGdlbmVyaWMgSS9PJmd0OyBvbiBpaWNidXMxPC9zcGFuPjwvZGl2 PjxkaXY+PHNwYW4+U2VwIDYgMTU6NTU6NDQgYnNkbGFwIGtlcm5lbDogZHJtbjA6ICZsdDtkcm1u Jmd0OyBvbiB2Z2FwY2kwPC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4+U2VwIDYgMTU6NTU6NDQgYnNk bGFwIGtlcm5lbDogdmdhcGNpMDogY2hpbGQgZHJtbjAgcmVxdWVzdGVkIHBjaV9lbmFibGVfaW88 L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj5TZXAgNiAxNTo1NTo0NCBic2RsYXAgc3lzbG9nZDogbGFz dCBtZXNzYWdlIHJlcGVhdGVkIDEgdGltZXM8L3NwYW4+PC9kaXY+PGRpdj48c3Bhbj5TZXAgNiAx NTo1NTo0NCBic2RsYXAga2VybmVsOiBbZHJtXSBVbmFibGUgdG8gY3JlYXRlIGEgcHJpdmF0ZSB0 bXBmcyBtb3VudCwgaHVnZXBhZ2Ugc3VwcG9ydCB3aWxsIGJlIGRpc2FibGVkKC0xOSkuPC9zcGFu PjwvZGl2PjxzcGFuPlNlcCA2IDE1OjU1OjQ0IGJzZGxhcCBrZXJuZWw6IFtkcm1dIEdvdCBzdG9s ZW4gbWVtb3J5IGJhc2UgMHhjZDgwMDAwMCwgc2l6ZSAweDIwMDAwMDA8L3NwYW4+PC9zcGFuPjwv ZGl2PjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXpl OiAxNHB4OyI+PHNwYW4+PHNwYW4+PGJyPjwvc3Bhbj48L3NwYW4+PC9kaXY+PGRpdiBzdHlsZT0i Zm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7Ij48c3Bhbj48 c3Bhbj5XaGF0IGNhbiBiZSB0aGUgcHJvYmxlbT8gVGhhbmsgeW91ITxicj48L3NwYW4+PC9zcGFu PjwvZGl2Pg== --b1_NUdxBP1lfVk0x3ZzS9VJW8wwMrS6iVf5PbnOTyzJdtA-- From nobody Thu Sep 7 18:02:16 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RhRsl1T49z4sdhM for ; Thu, 7 Sep 2023 18:02:35 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-24.consmr.mail.gq1.yahoo.com (sonic304-24.consmr.mail.gq1.yahoo.com [98.137.68.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RhRsk0bKcz4Zlh for ; Thu, 7 Sep 2023 18:02:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=Glf949Nr; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694109752; bh=Ih+MlzrpWBXPVdNwiryi22KM451jCkvPg8y25cp8J6o=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=Glf949Nr0sIxWyo4rozXxTuKpt0NoudR+DuEgVQwGYRfzobnovEcDbV14ynEkuGhJETiGDGyZDk/6Ei4JZiJcSDbKOKGG/jYrF1QiPhU68XVkfYFDlD61FZg/FK4hCyXLn2/2hihPT9JfY9Id56f/Qw9D80yalcEAKXaRMZG0TjiEgku1/ZpycHiK+ZE57n2qF8TO5lZWfCR4Rc7UztSqmpI517+LH03znYX1fkV49PYSNJgYi7CQCAwqkSwYDkzDCtQZdBfdl6llzzq5nXGqGBLG1W4KWPdDUoylrBhlOkUSmFDMo9zinc+nh2KH2GNMJor+aiOfXoYTfV2iDDtpQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694109752; bh=KCQozqplxVlQXODdOyUvdxbD2gSdfsWFOjRt1CYMy07=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=fs39eQeOSEgCWxFhCvJhPJfT+VLaUDyQUzjGhUQDVSnwYiSzXI/KUYzTXtG5AgvP3g+XK+XcdlyEVWeAZvYl22c/2UIkCrDdnSLX34zKr3GR8nXYz5805j2uoe3gsXyHVw99q6dmexsMH2H9gXeWUfF7ymZ4o+F47Sdv/RgnVl/i05Sruz91JoGZCPihFHS/PWjDgP1pKW9FQqE2fMSc6yLzokgoQjzWnLBaqi1SoV//X99spEEHTPSZs5HXSvt8rjwRmnyNnEPuUS9WxbQAsmgG09aYz7EWgydTeR3Mweu372v0KlOh7Rj9Fw75B5gDnEcw39dIDg5qC0QRJhGnsQ== X-YMail-OSG: .Tyg1gIVM1khRcV2KHv13ukxelWzwvssG_98l6l_3.TBNNydQj53Bwi4BkaONfM kMrEPkttVtTFsEr6DK8tnbwBV5rZIHa5bkjQHTsZRvFholY5MZ4aUZyKgfwQZV5pRer2ljQ4H4jb MyDkoRdFUAUhomIJpbxv4KvbuDx_CoJZGR36OdC4iUGRC85TMhxMu6xA.XTvjUuXcMWTriy9ZzLm JhAqnO4RmQ1aIn38aV3qgD1g1XA_9VOTU0Ets08dJrtbkUKftstx_vzXgohei4X.uAj0ZqJzCLSj 0FWZ8nvwhbrw3zBLR9Zn58iIdwTxReXzWrgYLcoqbWkFcikOiQR2TsXWXFDFvl._oSFLRSVtl2WD d2_jl8rY4LGfknn7P6MCDlIVMvAsSMb.LF0B4.U06PEcX3lM17Qtkk9O0gYfX80b2V_kDvc4fje6 hAWncIlluQpB_SuH0HI51OOh.HxoW6KH59u6TixTmOqmDfu0E52PvWVRe_RHY0t0d3hA5c9qXdlx 64UcK_N1UmORRoJDS_UW50yMz_E.O.u8LWzlhB_0CakCFk02ta2HX0gA1EwtGuoALMP8OrCVuv35 OfGyx.aHWMfGaItN2FyT8KA85wLT8wByA5w67UhtTqq4E.5sYaVU7xlxK_tpkQnbFKNnTxM7Y_TC NaN3sSUH5lOmNostOoW5LuvMc_QYFVjV5ZgaWs1jdSgVSCQ2t41bn3b292CKmzukYdAOzMk8F0B9 XTmh9ArLJYneJqnq4lhzeALy.rfVF9hHuGhzD7A7SS0I7XD8FINByAPuo0uOCtRR5URAfhXM0iv8 .j.jcRqKB7z3eyJMeXb1Qri3vGyx2K4kW1dU64y8xcccZtWEubqIz_..1Xb0SGRYhPjOccBlxZUU _26lSAodpBjBBBld2Tnc4HnPoeX25MW5QRljkhAd.xmZdACBNMFyk1CFa0t4LzVcvWdSNiQ6y1L5 HqiJSOP5KHCUwRUUrSXOhp31M8Yb8YSMD6xehhmGCXhQafdo7eEhfATWaE9embrW46_u7SVuEhd_ Mq1aPEdsgFcfFmI7rbdJE8pJynMvlCXfkmFqq_gvwH7JZvuenyfxtsz9p94Ez_A5rQSfyLdDmtdK c7CUi_jet9xuAwsiYFFuTQXs8ylT8gVzZQJN_f.928B_4zeY7geE1c4DOQO.JF1w1OjyplsLWHue uKM5hzYM5Argk20YW.z2vdLeM.Cpes_aAZW6lLx5kOxCSoOJB0K7dVeodFcV2ORMevfNvomVBa3B 0Iqo8POHkNBkPPSs0xcDZE4aQBZ5O6TtVWhS1W_PpZuv.f_UWVKSD8e4e.wCz7WNU9PAfVyyVZpL gn6dxJ50ldzhAe0XlikfqibxoWEkBpLeHHD7WaRlWogZWaZpPcF82kfrbAcMOxpFtVmOSpfBncjR 7EWKlCZHK.77GVVcwH8y1R1HAbMe6rnERnSglRW53uolQaPthm4iq6MxO.4iJ5YmLExc9KIW3Ddp 43YDOoVmWjwOI86pCHhxJpzJlHgoocfZ7B5xZw6Cmm9JZjfzxVGC_Yg1_q5GB_Umg9UfdKNKzKj7 Kjg5fJ1hdC1UBrktwuk80pA0YDVx5yqkWBSnAJ3g5gSPFXMC0IhxuU06gtYhz3uxOBky3u_MpVEy oDm3TrEyqh.xLcU4DuiFLrSgvegzfY24VYac1ufu6YAhe4lLhvuRDYElB9pnBl07yZkFdteiUNtV NPXJtv8ZBc.CLV4AmOn7UKS2C2g6LLIOFb.HnkU5meUygTLqg2lDOXJCnTtUJmsVeg8p2.IEHlnV H612Haiiny1wcrV4c9Eb6Y8l9.GnUyNR3g4SJdpbwfgshf4ZF09X0r3BsyrBjP267ZHrVms94ryZ 6luZd4N2zLpHn7RXuK4jnepzCbh7fKbn0LFGCyApJ6IbiCKAg7.RWUy2IiOUzLX6VpJiSSulRKfZ QNwVNIrTXyTfJBxZ3v1Ushc08azxfqLbYqRISzxu424p9ICXUFgf9P3YhDvh3xDMN0ey0pnlmGqg u.08nhv.MSU7t0mZssxoWNvvta5vdYByIyOxWwIRhMfo0aXEXFz3oAwhB2dwS482ZhSiMVhoBPkR .mgPgHDPjHvDJR3ilmbIfpaM.xeo0jbg7ULc9MYFI_1xcbY4PtWPRRvFiVRfiuk8CvdA58gkGrZm 1UEAUvNG2XykeODMI2wJd68Z85_de2GfnWjH7PckAMkw.JhYp99r1fUdAIA4C5gEWEsF5KXuot_N kRKtujNRGwd1aSMi2fyjXxLWydk0d8zOLImGI2OWOEJz3tNkcDVWPjhsfjyTVDWIBr6cN.fzRld5 mGvhCqmNtDwY- X-Sonic-MF: X-Sonic-ID: cbb56a35-0796-410c-a00c-e1178b0d7069 Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Thu, 7 Sep 2023 18:02:32 +0000 Received: by hermes--production-bf1-865889d799-7x4p2 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID d34197bc64bbc6e9ed481dc514c0e25f; Thu, 07 Sep 2023 18:02:29 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics Message-Id: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> Date: Thu, 7 Sep 2023 11:02:16 -0700 Cc: Glen Barber To: Current FreeBSD , FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3731.700.6) References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C.ref@yahoo.com> X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; BLOCKLISTDE_FAIL(0.00)[98.137.68.205:server fail]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.205:from]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.205:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RhRsk0bKcz4Zlh I was requested to do a test with vfs.zfs.bclone_enabled=3D1 and the bulk -a build paniced (having stored 128 *.pkg files in .building/ first): # more /var/crash/core.txt.3 . . . Unread portion of the kernel message buffer: panic: Solaris(panic): zfs: accessing past end of object 422/1108c16 = (size=3D2560 access=3D2560+2560) cpuid =3D 15 time =3D 1694103674 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe0352758590 vpanic() at vpanic+0x132/frame 0xfffffe03527586c0 panic() at panic+0x43/frame 0xfffffe0352758720 vcmn_err() at vcmn_err+0xeb/frame 0xfffffe0352758850 zfs_panic_recover() at zfs_panic_recover+0x59/frame 0xfffffe03527588b0 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0x97/frame = 0xfffffe0352758960 dmu_brt_clone() at dmu_brt_clone+0x61/frame 0xfffffe03527589f0 zfs_clone_range() at zfs_clone_range+0xa6a/frame 0xfffffe0352758bc0 zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x1ae/frame = 0xfffffe0352758c40 vn_copy_file_range() at vn_copy_file_range+0x11e/frame = 0xfffffe0352758ce0 kern_copy_file_range() at kern_copy_file_range+0x338/frame = 0xfffffe0352758db0 sys_copy_file_range() at sys_copy_file_range+0x78/frame = 0xfffffe0352758e00 amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe0352758f30 fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe0352758f30 --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D = 0x1ce4506d155a, rsp =3D 0x1ce44ec71e88, rbp =3D 0x1ce44ec72320 --- KDB: enter: panic __curthread () at /usr/main-src/sys/amd64/include/pcpu_aux.h:57 57 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" = (offsetof(struct pcpu, (kgdb) #0 __curthread () at = /usr/main-src/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=3Dtextdump@entry=3D0) at /usr/main-src/sys/kern/kern_shutdown.c:405 #2 0xffffffff804a442a in db_dump (dummy=3D, = dummy2=3D, dummy3=3D, dummy4=3D) at /usr/main-src/sys/ddb/db_command.c:591 #3 0xffffffff804a422d in db_command (last_cmdp=3D, = cmd_table=3D, dopager=3Dtrue) at /usr/main-src/sys/ddb/db_command.c:504 #4 0xffffffff804a3eed in db_command_loop () at /usr/main-src/sys/ddb/db_command.c:551 #5 0xffffffff804a7876 in db_trap (type=3D, = code=3D) at /usr/main-src/sys/ddb/db_main.c:268 #6 0xffffffff80bb9e57 in kdb_trap (type=3Dtype@entry=3D3, = code=3Dcode@entry=3D0, tf=3Dtf@entry=3D0xfffffe03527584d0) at = /usr/main-src/sys/kern/subr_kdb.c:790 #7 0xffffffff8104ad3d in trap (frame=3D0xfffffe03527584d0) at /usr/main-src/sys/amd64/amd64/trap.c:608 #8 #9 kdb_enter (why=3D, msg=3D) at /usr/main-src/sys/kern/subr_kdb.c:556 #10 0xffffffff80b6aab3 in vpanic (fmt=3D0xffffffff82be52d6 "%s%s", = ap=3Dap@entry=3D0xfffffe0352758700) at /usr/main-src/sys/kern/kern_shutdown.c:958 #11 0xffffffff80b6a943 in panic ( fmt=3D0xffffffff820aa2e8 = "\312C$\201\377\377\377\377") at /usr/main-src/sys/kern/kern_shutdown.c:894 #12 0xffffffff82993c5b in vcmn_err (ce=3D, = fmt=3D0xffffffff82bfdd1f "zfs: accessing past end of object %llx/%llx = (size=3D%u access=3D%llu+%llu)", adx=3D0xfffffe0352758890) at = /usr/main-src/sys/contrib/openzfs/module/os/freebsd/spl/spl_cmn_err.c:60 #13 0xffffffff82a84d69 in zfs_panic_recover ( fmt=3D0x12 ) at /usr/main-src/sys/contrib/openzfs/module/zfs/spa_misc.c:1594 #14 0xffffffff829f8e27 in dmu_buf_hold_array_by_dnode = (dn=3D0xfffff813dfc48978, offset=3Doffset@entry=3D2560, = length=3Dlength@entry=3D2560, read=3Dread@entry=3D0, = tag=3D0xffffffff82bd8175, numbufsp=3Dnumbufsp@entry=3D0xfffffe03527589bc, = dbpp=3D0xfffffe03527589c0, flags=3D0) at /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:543 #15 0xffffffff829fc6a1 in dmu_buf_hold_array (os=3D, = object=3D, read=3D0, numbufsp=3D0xfffffe03527589bc, = dbpp=3D0xfffffe03527589c0, offset=3D, length=3D, tag=3D) at /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:654 #16 dmu_brt_clone (os=3Dos@entry=3D0xfffff8010ae0e000, object=3D, offset=3Doffset@entry=3D2560, length=3Dlength@entry=3D2560, = tx=3Dtx@entry=3D0xfffff81aaeb6e100, = bps=3Dbps@entry=3D0xfffffe0595931000, nbps=3D1, replay=3D0) at = /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:2301 #17 0xffffffff82b4440a in zfs_clone_range (inzp=3D0xfffff8100054c910, = inoffp=3D0xfffff81910c3c7c8, outzp=3D0xfffff80fb3233000, = outoffp=3D0xfffff819860a2c78, lenp=3Dlenp@entry=3D0xfffffe0352758c00, = cr=3D0xfffff80e32335200) at /usr/main-src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1302 #18 0xffffffff829b4ece in zfs_freebsd_copy_file_range = (ap=3D0xfffffe0352758c58) at = /usr/main-src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:629= 4 #19 0xffffffff80c7160e in VOP_COPY_FILE_RANGE (invp=3D, = inoffp=3D0x40, outvp=3D0xfffffe03527581d0, = outoffp=3D0xffffffff811e6eb7, lenp=3D0x0, flags=3D0, = incred=3D0xfffff80e32335200, outcred=3D0x0, = fsizetd=3D0xfffffe03586c0720) at ./vnode_if.h:2381 #20 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff8095e1e8000, = inoffp=3D0x40, inoffp@entry=3D0xfffff81910c3c7c8, = outvp=3D0xfffffe03527581d0, outvp@entry=3D0xfffff805d6107380, = outoffp=3D0xffffffff811e6eb7, outoffp@entry=3D0xfffff819860a2c78, = lenp=3D0x0, lenp@entry=3D0xfffffe0352758d50, flags=3Dflags@entry=3D0,= incred=3D0xfffff80e32335200, outcred=3D0xfffff80e32335200, = fsize_td=3D0xfffffe03586c0720) at = /usr/main-src/sys/kern/vfs_vnops.c:3085 #21 0xffffffff80c6b998 in kern_copy_file_range ( td=3Dtd@entry=3D0xfffffe03586c0720, infd=3D, = inoffp=3D0xfffff81910c3c7c8, inoffp@entry=3D0x0, outfd=3D, outoffp=3D0xfffff819860a2c78, outoffp@entry=3D0x0, = len=3D9223372036854775807, flags=3D0) at = /usr/main-src/sys/kern/vfs_syscalls.c:4971 #22 0xffffffff80c6bab8 in sys_copy_file_range (td=3D0xfffffe03586c0720, = uap=3D0xfffffe03586c0b20) at = /usr/main-src/sys/kern/vfs_syscalls.c:5009 #23 0xffffffff8104bab9 in syscallenter (td=3D0xfffffe03586c0720) at /usr/main-src/sys/amd64/amd64/../../kern/subr_syscall.c:187 #24 amd64_syscall (td=3D0xfffffe03586c0720, traced=3D0) at /usr/main-src/sys/amd64/amd64/trap.c:1197 #25 #26 0x00001ce4506d155a in ?? () Backtrace stopped: Cannot access memory at address 0x1ce44ec71e88 (kgdb)=20 Context details follow. Absent a openzfs-2.2 in: ls -C1 /usr/share/zfs/compatibility.d/openzfs-2.* /usr/share/zfs/compatibility.d/openzfs-2.0-freebsd /usr/share/zfs/compatibility.d/openzfs-2.0-linux /usr/share/zfs/compatibility.d/openzfs-2.1-freebsd /usr/share/zfs/compatibility.d/openzfs-2.1-linux I have copied: /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2 over to: # ls -C1 /etc/zfs/compatibility.d/* /etc/zfs/compatibility.d/openzfs-2.2 and used it: # zpool get compatibility zamd64 NAME PROPERTY VALUE SOURCE zamd64 compatibility openzfs-2.2 local For reference: # zpool upgrade This system supports ZFS pool feature flags. All pools are formatted using feature flags. Some supported features are not enabled on the following pools. Once a feature is enabled the pool may become incompatible with software that does not support the feature. See zpool-features(7) for details. Note that the pool 'compatibility' feature can be used to inhibit feature upgrades. POOL FEATURE --------------- zamd64 redaction_list_spill which agrees with openzfs-2.2 . I did: # sysctl vfs.zfs.bclone_enabled=3D1 vfs.zfs.bclone_enabled: 0 -> 1 I also made a snapshot: zamd64@before-bclone-test and I then made a checkpoint. These were establshed just after the above enable. I then did a: zpool trim -w zamd64 The poudriere bulk command was: poudriere bulk -jmain-amd64-bulk_a -a where main-amd64-bulk_a has nothing prebuilt. USE_TMPFS=3Dno is in use. No form of ALLOW_MAKE_JOBS is in use. It is a 32 builder context (32 hardware threads). For reference: # uname -apKU FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #118 = main-n265152-f49d6f583e9d-dirty: Mon Sep 4 14:26:56 PDT 2023 = root@amd64_ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.a= md64/sys/GENERIC-NODBG amd64 amd64 1500000 1500000 I'll note that with openzfs-2.1-freebsd compatibility I'd previously let such a bulk -a run for about 10 hr and it had reached 6366 port->package builds. Prior to that I'd done shorter experiments with default zpool features (no explicit compatibility constraint) but vfs.zfs.bclone_enabled=3D0 and I'd had no problems. (I have a separate M.2 boot media just for such experiments and can reconstruct its content at will.) All these have been based on the same personal main-n265152-f49d6f583e9d-dirty system build. Unfortunately, no appropriate snapshot of main was available to avoid my personal context being involved for the system build used. Similarly, the snapshot(s) of stable/14 predate: Sun, 03 Sep 2023 . . . git: f789381671a3 - stable/14 - zfs: merge openzfs/zfs@32949f256 = (zfs-2.2-release) into stable/14 that has required fixes for other issues. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Sep 7 18:17:22 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RhSC36g6wz4smHY for ; Thu, 7 Sep 2023 18:17:35 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RhSC260V0z3CKb for ; Thu, 7 Sep 2023 18:17:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=IH7HgLw3; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694110653; bh=5sFN3xXQaexLhfTaeL9ndfEG/M0UHb4VYVr0PE4Pt4I=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=IH7HgLw3JFhGvwXbS7xHYsEdLt1Y2UIWgGiXpHZLERRuNVaGeZXBj6SYYLPNSeXPUS/3eBVto81OkhejsYjW8lR6gQCdR+GzSTk1oBLTkDNSxrl+bgeS57OVRkQcZe1kp8AdFqcc9R1Gr7JSXFY7RIRdfUWYXGoILokrdit4KnB1sqPK5gFV1vh5rw5qvnYQ7Pks1KzCbnV3FhudZdnhYK3QqylCGLV8E5aPKoBN/O+ouIzi7hEuXpZ5lqo8GVFQINAX5o6LMsagpBNe1YvgLMjU7dSx6yQ8nxjUZ3YyqAJXqeIGWFwnNNzrOx2o24Iw4qNOwMSqYLx28fosIIZ96g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694110653; bh=48ftdgRrBHwrA8tgzroxr04zOUqAX6sejUQjV+r7hfn=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Lq9WOW0pbaUSmNIlAh/5whtMoMLrq8SejzvPHZPzV/NzA8QNpp5rjtqxfm7xgEucrui0sgwKJw7AVj8ROxWQ7zwZaU+ZGKflFTOv4jtC2iYpjBATeRkXDs2I1fWoBPwudBdl2g0IxmIp6HYLKVtGmJUnPoTFIa3CljQFR/JLL0+6WdFYrtOvuTmD77i/Wlv0juqGAu24ziB+eR+OxXMYPdmgQztOlNEDToWyznZ2S+tLeTcr+Qb2nK755Z+U8dr5QKmEPH3YDCZE4VzltsewkzHBS6EbIMt1OcPVxF+gpj0dEYe5ywIpme4bXTZyGWa8dnoSAisnUXQsX6Ee+c9Jsg== X-YMail-OSG: HAfkR1QVM1lNSKyNVzBjMBEn_ELh3PpL1S.SOJ3HrSL.776qmYtASIcsISCWUox TSkb8O1qjKp8hn.dSBx81XsQTGPl2OyfUNhzmaPjfmkgAKolez7fqgk22HFyW5wnj5qtlJ2nl7Pi LAd3HF0VIxI6lXv9O7q9mpEZwobTWvdc.gYLz_UmDs_kgcrTmpHENEk8CfG.OUtDGiMhXEQ0vdX6 XjKJbSf6BoP1ffUoEKKqa995lgqcl.NRyyVw.0Zh2S8_Vz8j2k2F7U5AG7yEs8YlRz_x.gCIQqZQ dGwli6AUrlnPrqp7Vu0hTNzrdf_6uXAKvNAO19h7ZDOQIvuVx2u.QBsjUFC6aEpE7o866S4Esq0Q zrvS7ySUznI8rM5e_NdbA0RRfHAxGpJmC1IwC7rhxwQfgJn_79jm0JACpJxbnjsqm5Igo1zebUNA Fu1q8fhjg.YmvBvHunuYBp.BauuQS6kl7LB9KXE2o_.E1mQFLxLq978GSEXXBCurCa7wC2uLxcws L_atX6P9YB_3ucqMU_jGbsWp_QZf3X5RP70DdH4n7xoXHbmjeoxv2jLbwSnVC_s3Jie0m62PgFU6 wgOSWJ08SeBiSoJIYGH_iwIFrUOzHHJQ4EVAIynXul9s4N6CFhlSqN0u26KCdqCkLthZJP79MXHL 4vMCNEvBmpUTJ6U4sqXjquoa_n.NLH_6D6jFzZB4HFfSLtFJDYnqFabH9UrTs3oFO2wdzz5GEOEw gKdM4av4ts7HP3J4GZJctwgD6WIJMMW1YRFSt9b89MtOgVvW4rR6NBlsElZlPeMdvYEvpeHQENgE wi7dMBxiGU1k5oWMDzcJE5qkKgsKmJN910ViQTW_RoMcX6EWcB44UfzqIWo5_JVG51_YcmfT_BU4 TEfoj6wUcQKDrTodm78ObLRR5Z2R0ifK2b3yu2osj.u5RITGM3pwslqV842LtObWXUdJaR2o40sS 96sSTKHdhZHXG9xZs1N2PuxUimRpuSoqzCRH6Txnzifcg20gJyamPzviMtrDGoqQ3xBHzlhgnRTp O_WZTBN08z5cIhkru2tbX2HBUUCMIaYPli1Z1lHP9np8UNeDd2WLbWe6vfTEOWSlgkdWCmJ1IY.x aPMJb9kfeov.5fH8dE07ZHLvQqy8OUHDqrOkVk5.wwqfExZHgAMJydquPQDQ8JexSHQI45AqiH7W wXUlVpiUi_SM9KIZpzOcFBPg3NZg8aV9uGYotBHTXu3oo4jc1yydPmVzycBRt87j4DnDrGE5RZTL CePEcD0xoGRzASpxq20Qt7XNju.ZP5Okzy3Rt74HsQPB0yU8gQuR8kNiWFqFbsdHjSnqFYo1ZAtq cIvWILOulw2pybomE8AGFIgKlAR81kbH7ACrGv9s213._ogip5GD8kN6Y.2dkIUWYyajheDqARqS PJs5Nh6IEyPTvtKXnUsdYw4P9_vAAqO_oGjkvk_KRr_TuWtZ.vXFCeLrs7Y_gfCm_KMgCS6Wikt5 86iI0qIaBTCCYICK1v3.xh4B15ju8BQa8yN2jB.2YQzBgq1Vx83SD38LrTsFRZ3PdM8csX9tgcXX 1UzZagFvVlGKm7nYKflIJPcYBGHUnkyG5GliCiYqMEQcuQ.v5UOt89x215Ev1mnCEyzJIznUw6NA GZ6kfE7zdcG4ggJsuO5nQbHVClQBEsccUdpitSmwKqcJATT6LbFRgywxJl8gXh9E3NXhacT5W_9p NRymhREw4XweUuSFzt5U96tqlR9TYMltnvliepmy7CKFl7bulnIItUoC.mOPAx4v1Ahb5LJQvU4W cnPclgeU52iUbHEyc2i_IrVqLBTX0ZkokAPs4QkTDGYHopJ0W2tGw9gs7yxfdB8kHuXRJ2TVvySl y7F4AnPd8A_RXuK6a.KtqM1pQLO1_Z0.GLLodcA6g6dv58vfUE8Ppmv7PH5U6MdqvV3DqlqFmaFF 6rBQiiaijYo2_fe_AUoZb_VhqLM1Ys0nzVJoTIgc6arztY52QIyo9fqjNbCMoITOTB3gWe7bbh4b H8fX5cmwOOkXCF6XbTNfRNfE4V06OgsOR.ESBbaG5cJkKa1vsbAS85zIW3oqsU_yRCIsEwI.rm86 A.OXhC55dQeSWN0nmuBHigu0XJpiHCCit.UmVgvHK42nj1be5d_eugFV_8OjdD.8fnfnqa6mexMc EcPuPQAqO0Zqux8VYQLr66oaOorldNC_5i_0W2mRE2MxqVSU94BKh6DP0PgCmbMp_EinfB9iOKCz UlBkitHRDi.I90OAwGNpk9sPJIXZjgQL98R748delGjYG_Ol4ciIXeAjN4or3w9VyvVFIu5D01jQ w._cHeMQE X-Sonic-MF: X-Sonic-ID: e84200eb-531b-4c8f-a9de-aa0a36d2dd20 Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Thu, 7 Sep 2023 18:17:33 +0000 Received: by hermes--production-bf1-865889d799-x5klk (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 54dbc57beef42a41a0b046b389ee1fcc; Thu, 07 Sep 2023 18:17:27 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> Date: Thu, 7 Sep 2023 11:17:22 -0700 Cc: Glen Barber , Pawel Jakub Dawidek , Alexander Motin Content-Transfer-Encoding: quoted-printable Message-Id: <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> To: Martin Matuska , Current FreeBSD , FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.205:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.205:from]; TO_DN_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RhSC260V0z3CKb [Drat, the request to rerun my tests did not not mention the more recent change: vfs: copy_file_range() between multiple mountpoints of the same fs type and I'd not noticed on my own and ran the test without updating.] On Sep 7, 2023, at 11:02, Mark Millard wrote: > I was requested to do a test with vfs.zfs.bclone_enabled=3D1 and > the bulk -a build paniced (having stored 128 *.pkg files in > .building/ first): Unfortunately, rerunning my tests with this set was testing a context predating: Wed, 06 Sep 2023 . . . =E2=80=A2 git: 969071be938c - main - vfs: copy_file_range() between = multiple mountpoints of the same fs type Martin Matuska So the information might be out of date for main and for stable/14 : I've no clue how good of a test it was. May be some of those I've cc'd would know. When I next have time, should I retry based on a more recent vintage of main that includes 969071be938c ? > # more /var/crash/core.txt.3 > . . . > Unread portion of the kernel message buffer: > panic: Solaris(panic): zfs: accessing past end of object 422/1108c16 = (size=3D2560 access=3D2560+2560) > cpuid =3D 15 > time =3D 1694103674 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe0352758590 > vpanic() at vpanic+0x132/frame 0xfffffe03527586c0 > panic() at panic+0x43/frame 0xfffffe0352758720 > vcmn_err() at vcmn_err+0xeb/frame 0xfffffe0352758850 > zfs_panic_recover() at zfs_panic_recover+0x59/frame 0xfffffe03527588b0 > dmu_buf_hold_array_by_dnode() at = dmu_buf_hold_array_by_dnode+0x97/frame 0xfffffe0352758960 > dmu_brt_clone() at dmu_brt_clone+0x61/frame 0xfffffe03527589f0 > zfs_clone_range() at zfs_clone_range+0xa6a/frame 0xfffffe0352758bc0 > zfs_freebsd_copy_file_range() at = zfs_freebsd_copy_file_range+0x1ae/frame 0xfffffe0352758c40 > vn_copy_file_range() at vn_copy_file_range+0x11e/frame = 0xfffffe0352758ce0 > kern_copy_file_range() at kern_copy_file_range+0x338/frame = 0xfffffe0352758db0 > sys_copy_file_range() at sys_copy_file_range+0x78/frame = 0xfffffe0352758e00 > amd64_syscall() at amd64_syscall+0x109/frame 0xfffffe0352758f30 > fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe0352758f30 > --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D = 0x1ce4506d155a, rsp =3D 0x1ce44ec71e88, rbp =3D 0x1ce44ec72320 --- > KDB: enter: panic >=20 > __curthread () at /usr/main-src/sys/amd64/include/pcpu_aux.h:57 > 57 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" = (offsetof(struct pcpu, > (kgdb) #0 __curthread () at = /usr/main-src/sys/amd64/include/pcpu_aux.h:57 > #1 doadump (textdump=3Dtextdump@entry=3D0) > at /usr/main-src/sys/kern/kern_shutdown.c:405 > #2 0xffffffff804a442a in db_dump (dummy=3D, = dummy2=3D, dummy3=3D, dummy4=3D) > at /usr/main-src/sys/ddb/db_command.c:591 > #3 0xffffffff804a422d in db_command (last_cmdp=3D, = cmd_table=3D, dopager=3Dtrue) > at /usr/main-src/sys/ddb/db_command.c:504 > #4 0xffffffff804a3eed in db_command_loop () > at /usr/main-src/sys/ddb/db_command.c:551 > #5 0xffffffff804a7876 in db_trap (type=3D, = code=3D) > at /usr/main-src/sys/ddb/db_main.c:268 > #6 0xffffffff80bb9e57 in kdb_trap (type=3Dtype@entry=3D3, = code=3Dcode@entry=3D0, tf=3Dtf@entry=3D0xfffffe03527584d0) at = /usr/main-src/sys/kern/subr_kdb.c:790 > #7 0xffffffff8104ad3d in trap (frame=3D0xfffffe03527584d0) > at /usr/main-src/sys/amd64/amd64/trap.c:608 > #8 > #9 kdb_enter (why=3D, msg=3D) > at /usr/main-src/sys/kern/subr_kdb.c:556 > #10 0xffffffff80b6aab3 in vpanic (fmt=3D0xffffffff82be52d6 "%s%s", = ap=3Dap@entry=3D0xfffffe0352758700) > at /usr/main-src/sys/kern/kern_shutdown.c:958 > #11 0xffffffff80b6a943 in panic ( > fmt=3D0xffffffff820aa2e8 = "\312C$\201\377\377\377\377") > at /usr/main-src/sys/kern/kern_shutdown.c:894 > #12 0xffffffff82993c5b in vcmn_err (ce=3D, = fmt=3D0xffffffff82bfdd1f "zfs: accessing past end of object %llx/%llx = (size=3D%u access=3D%llu+%llu)", adx=3D0xfffffe0352758890) > at = /usr/main-src/sys/contrib/openzfs/module/os/freebsd/spl/spl_cmn_err.c:60 > #13 0xffffffff82a84d69 in zfs_panic_recover ( > fmt=3D0x12 ) > at /usr/main-src/sys/contrib/openzfs/module/zfs/spa_misc.c:1594 > #14 0xffffffff829f8e27 in dmu_buf_hold_array_by_dnode = (dn=3D0xfffff813dfc48978, offset=3Doffset@entry=3D2560, = length=3Dlength@entry=3D2560, read=3Dread@entry=3D0, = tag=3D0xffffffff82bd8175, numbufsp=3Dnumbufsp@entry=3D0xfffffe03527589bc, = dbpp=3D0xfffffe03527589c0, flags=3D0) > at /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:543 > #15 0xffffffff829fc6a1 in dmu_buf_hold_array (os=3D, = object=3D, read=3D0, numbufsp=3D0xfffffe03527589bc, = dbpp=3D0xfffffe03527589c0, offset=3D, length=3D, tag=3D) > at /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:654 > #16 dmu_brt_clone (os=3Dos@entry=3D0xfffff8010ae0e000, = object=3D, offset=3Doffset@entry=3D2560, = length=3Dlength@entry=3D2560, tx=3Dtx@entry=3D0xfffff81aaeb6e100, = bps=3Dbps@entry=3D0xfffffe0595931000, nbps=3D1, replay=3D0) at = /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:2301 > #17 0xffffffff82b4440a in zfs_clone_range (inzp=3D0xfffff8100054c910, = inoffp=3D0xfffff81910c3c7c8, outzp=3D0xfffff80fb3233000, = outoffp=3D0xfffff819860a2c78, lenp=3Dlenp@entry=3D0xfffffe0352758c00, = cr=3D0xfffff80e32335200) > at /usr/main-src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1302 > #18 0xffffffff829b4ece in zfs_freebsd_copy_file_range = (ap=3D0xfffffe0352758c58) > at = /usr/main-src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:629= 4 > #19 0xffffffff80c7160e in VOP_COPY_FILE_RANGE (invp=3D, = inoffp=3D0x40, outvp=3D0xfffffe03527581d0, = outoffp=3D0xffffffff811e6eb7, lenp=3D0x0, flags=3D0, = incred=3D0xfffff80e32335200, outcred=3D0x0, = fsizetd=3D0xfffffe03586c0720) at ./vnode_if.h:2381 > #20 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff8095e1e8000, = inoffp=3D0x40, inoffp@entry=3D0xfffff81910c3c7c8, = outvp=3D0xfffffe03527581d0, outvp@entry=3D0xfffff805d6107380, = outoffp=3D0xffffffff811e6eb7, outoffp@entry=3D0xfffff819860a2c78, = lenp=3D0x0, lenp@entry=3D0xfffffe0352758d50, flags=3Dflags@entry=3D0,= incred=3D0xfffff80e32335200, outcred=3D0xfffff80e32335200, = fsize_td=3D0xfffffe03586c0720) at = /usr/main-src/sys/kern/vfs_vnops.c:3085 > #21 0xffffffff80c6b998 in kern_copy_file_range ( > td=3Dtd@entry=3D0xfffffe03586c0720, infd=3D, = inoffp=3D0xfffff81910c3c7c8, inoffp@entry=3D0x0, outfd=3D, outoffp=3D0xfffff819860a2c78, outoffp@entry=3D0x0, = len=3D9223372036854775807, flags=3D0) at = /usr/main-src/sys/kern/vfs_syscalls.c:4971 > #22 0xffffffff80c6bab8 in sys_copy_file_range (td=3D0xfffffe03586c0720, = uap=3D0xfffffe03586c0b20) at = /usr/main-src/sys/kern/vfs_syscalls.c:5009 > #23 0xffffffff8104bab9 in syscallenter (td=3D0xfffffe03586c0720) > at /usr/main-src/sys/amd64/amd64/../../kern/subr_syscall.c:187 > #24 amd64_syscall (td=3D0xfffffe03586c0720, traced=3D0) > at /usr/main-src/sys/amd64/amd64/trap.c:1197 > #25 > #26 0x00001ce4506d155a in ?? () > Backtrace stopped: Cannot access memory at address 0x1ce44ec71e88 > (kgdb)=20 >=20 >=20 > Context details follow. >=20 > Absent a openzfs-2.2 in: >=20 > ls -C1 /usr/share/zfs/compatibility.d/openzfs-2.* > /usr/share/zfs/compatibility.d/openzfs-2.0-freebsd > /usr/share/zfs/compatibility.d/openzfs-2.0-linux > /usr/share/zfs/compatibility.d/openzfs-2.1-freebsd > /usr/share/zfs/compatibility.d/openzfs-2.1-linux >=20 > I have copied: >=20 > = /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2 >=20 > over to: >=20 > # ls -C1 /etc/zfs/compatibility.d/* > /etc/zfs/compatibility.d/openzfs-2.2 >=20 > and used it: >=20 > # zpool get compatibility zamd64 > NAME PROPERTY VALUE SOURCE > zamd64 compatibility openzfs-2.2 local >=20 > For reference: >=20 > # zpool upgrade > This system supports ZFS pool feature flags. >=20 > All pools are formatted using feature flags. >=20 >=20 > Some supported features are not enabled on the following pools. Once a > feature is enabled the pool may become incompatible with software > that does not support the feature. See zpool-features(7) for details. >=20 > Note that the pool 'compatibility' feature can be used to inhibit > feature upgrades. >=20 > POOL FEATURE > --------------- > zamd64 > redaction_list_spill >=20 > which agrees with openzfs-2.2 . >=20 > I did: >=20 > # sysctl vfs.zfs.bclone_enabled=3D1 > vfs.zfs.bclone_enabled: 0 -> 1 >=20 > I also made a snapshot: zamd64@before-bclone-test and > I then made a checkpoint. These were establshed just > after the above enable. >=20 > I then did a: zpool trim -w zamd64 >=20 > The poudriere bulk command was: poudriere bulk -jmain-amd64-bulk_a -a > where main-amd64-bulk_a has nothing prebuilt. USE_TMPFS=3Dno > is in use. No form of ALLOW_MAKE_JOBS is in use. It is a > 32 builder context (32 hardware threads). >=20 > For reference: >=20 > # uname -apKU > FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #118 = main-n265152-f49d6f583e9d-dirty: Mon Sep 4 14:26:56 PDT 2023 = root@amd64_ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.a= md64/sys/GENERIC-NODBG amd64 amd64 1500000 1500000 >=20 > I'll note that with openzfs-2.1-freebsd compatibility I'd > previously let such a bulk -a run for about 10 hr and it > had reached 6366 port->package builds. >=20 > Prior to that I'd done shorter experiments with default > zpool features (no explicit compatibility constraint) > but vfs.zfs.bclone_enabled=3D0 and I'd had no problems. >=20 > (I have a separate M.2 boot media just for such experiments > and can reconstruct its content at will.) >=20 > All these have been based on the same personal > main-n265152-f49d6f583e9d-dirty system build. Unfortunately, > no appropriate snapshot of main was available to avoid my > personal context being involved for the system build used. > Similarly, the snapshot(s) of stable/14 predate: >=20 > Sun, 03 Sep 2023 > . . . > git: f789381671a3 - stable/14 - zfs: merge openzfs/zfs@32949f256 = (zfs-2.2-release) into stable/14 >=20 > that has required fixes for other issues. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Sep 7 18:48:23 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RhStf4gkfz4rrZG; Thu, 7 Sep 2023 18:48:26 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RhStf4DYhz3Nm7; Thu, 7 Sep 2023 18:48:26 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694112506; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8xGzZ6mOBVk45BMLBVTdI7zQeZeSv8bpekpKYxnAwgQ=; b=JpRPwCZOlPJnN6sDn+VBxmm0mrhczr8Povxhy40vrGpc6cSFClwZzsYEyMzWwTQ5p4syVH DIWAu+ea6URIslIgA6AIntPIJmmfPVCfX1FOmTliTAj5/D2g0RsyaFar9Cx9w3QFXIcNhZ Yegsv2mSkXApcFqTwPy7sMhTgiziCH9DwQrgwp8Zh4CYGPHUpaoPubSh2LuC4fY2zelw+m +x5BncHl+WK+OMx7t8m/V2varNEvZ4G9rT79a8n/CYQ7j483vPa4FF8wpN+ZskY8esE31p JcH1uWb3zeI5D21G4Ay7Js8V/Cq2tXp5FVmOTETimVhPyl6nfEiGF+LAFmfOpA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1694112506; a=rsa-sha256; cv=none; b=DVdAOLjdcYrn6yEmeLVCsGeJQy8rZGUFlraMfvDgQgCm0d0EMrN/KzuuLXCfFGWkA6CqkO bHeYSBqkd1Idjt/kKtshkdzlFA0J0AMmOaEyKyLqSaij19/S1kt9g4gVt1NhTYNvvNw2s5 Q1GjN6P30t53295shkUu8zBOhD90+tBx+cLBcAyIif04h74L1h/XWz3xY1DdJ631v4Fh1e fAJPllXxm57VID5lvvcvPNN8MHd9hqrdiT+7iPbVGfm/unjH+pRv1CmCn8/bUBPpu8GinZ 8FAETMSO3FHX+ZHGkUR6JLLPU6SGsiAq9vfpRZdqR/uwGmyIewlQajc1dqpfOg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694112506; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8xGzZ6mOBVk45BMLBVTdI7zQeZeSv8bpekpKYxnAwgQ=; b=rluSIV+WIVihpbchR+11R+q9CLqY7m7xHjLmu8d5FyO8LbIuEDUUZIm/rou5WZDpE4askE +vgJx9hjTV/0IcG9MgFo/X/glOzkrERWuny1g3r5B6XtxWR3a/v1/3DhLkp28pGmsEhwjf lTnfaZ97JgnA7/QOjLxiZeAX58Xv6s2eCWXz4Pjw/NOI/E91ntlbbpgJOC4XtdTQiMmqRS NkwUCq0JK4hBMKGZ9l30MnToPjER0rYQZFppL/QuqCAp2PlnpZX8J0iyRJ9ybcmPXwkOS2 mAYuRBTuC4iYZjtoNdwvsBzBZGzjk0av2j4IfpR6SZPLzCQdWNcBd9kjvYFnUw== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 24355158A9; Thu, 7 Sep 2023 18:48:26 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Thu, 7 Sep 2023 18:48:23 +0000 From: Glen Barber To: Mark Millard Cc: Martin Matuska , Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek , Alexander Motin Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics Message-ID: <20230907184823.GC4090@FreeBSD.org> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="AXo2lOxbfudqq8ta" Content-Disposition: inline In-Reply-To: <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> --AXo2lOxbfudqq8ta Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 07, 2023 at 11:17:22AM -0700, Mark Millard wrote: > When I next have time, should I retry based on a more recent > vintage of main that includes 969071be938c ? >=20 Yes, please, if you can. Glen --AXo2lOxbfudqq8ta Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmT6GvcACgkQAxRYpUeP 4pPW0BAAlnTUPTfLJYS6O+vHnV6YYBLf3stXqdf17hsdwNQtwrbeRuhJRnmoyN5T QUowdKDSEYPjp49ibMBnHgSTYtjbEux+xmO6ctaSvqJCSLqPtkKsCvCprgdU4o/P a2r+iwQk/sPXaNMbl3DIPhBGNkM4teL9h5EYXzIdI2pf9O9ouyZKLbsgtYa8d/TR WFOnckiGn0uzGY6JGZ6MafmxrJFm3CisD6w2zPZYPdZCrxpXSIFD5GwDGMcUkvmn yrihpQspXDYQlKpE9yqI+DiA19+/NNAHsGGlcEkSeL04+0MRRf5fmKrxYwrhzp7n n5rZd/Du7gL7cdMAcdbOfIEBvlwt9b+5tdZ5XQHT1FfUinolLlUyA0ofzCOhPdDr OqPnoau3g4BCw28VRjWdUIbIKhyqvdRumiUvY1MPJQBkSNhO45mbDuJ8dlyeNLVm Bmnir5AJCLax6+tb4zENEUgzsqDGZzrsGDFanlyfkEGeRtDSJ4wPyk96XO/zHOFo KU53upxNIYV08dth9R5pb+/iCVszcVijyPHR8QCAqdPvcEhJnDMD3iGqeGe+g0J8 UuNHTLeOLiQ6zHsFmwGmS1bjYg6CmtY2zSOTTNMospAcXoDS6vax9a6w2rA3RBG6 JkbmjeilURuQJqhO03kOG9U41N1bfLNL0tosRYhhAv4JL4SteoA= =CbtC -----END PGP SIGNATURE----- --AXo2lOxbfudqq8ta-- From nobody Thu Sep 7 19:40:36 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RhV3B3NWjz4sPdR for ; Thu, 7 Sep 2023 19:40:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RhV3B2D3qz4GW5 for ; Thu, 7 Sep 2023 19:40:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694115651; bh=M+MqA0csUHMUaWxa+zSanbioXgAJKG3pyTcBMhnMQXE=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=ExvcDhzXWBaMgD1BTMceAMM1v4sBmIiZsOgiuygpyy/Q7NcOyD0v/fHqeZLn5DiwcHCGZjqOEpp2PjteP6HUnkFs5fNwpHD26cW/EoPhyA69f2uthPWVLaNTQzcU4kp4dIG94Wgc7nBeF7o9apiz+WuEv6Fp4WZe7Y166ZG2Gs1bPN03zk7ZjxBeouBivpBXLgY2L85mObIFwgyL0bFtYK5s0u+xfQomAeLR0TIHXa7uSOesGinb9SBUGgpPqrfMdJ1jYM/V1bi7soovekgN4iSChdPShejYW0ZrO7C1Ou7hWEMwjTo7qlekZ9vP6rHCcjpMj4QZjfZ63dtfXIZBdw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694115651; bh=5uzItdXMPZXlqsvxzllCppFn3A7G0dLXEP8QXshmaEp=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=L0zLWaGAGoH19IMAqcMDWN7XD+/RMH0xYERyyIW7G7pJduSWghw8zi+1DS/r2IdUiRtAtBc11jonl4bsgY4jrZRfsvHs09S3s6dwxTnZyZAmtZj72uK6JqfqSWiLRD8Mw1AAOQbLLqHdgpLqa2wROfT2mNESKS1xbeVd0/dtr1AJxLB24pz50MA4cwGpv0BcZzYAAgyojvl6GCV3lCmyu7gq2s/cnIx/Fr+HZWK8s9yOv5Kdv4URVXXNKTniFNd4e0iDK247VKUvMWsM4R2mVRbKcioUwlrnkMBwQ+sBVhvEwaIUEjwJN0HPRwvglNDX9nBEv9ZBEvEgWiy8KifKsQ== X-YMail-OSG: XfCKd1UVM1mudsCeWaOX7hcoInv9rDMsEctY7fRMQvjpPpkFMfGLotROk51fY_t 95BdEfhaadg9fqfa77JsPiRs2LUL8vOjbcqT6JeVPoEmYiS7PrYy.tMFM4CKXn5CashVKqhL49tC yWrWyAyDCKZYrRFa6F3IimisOlEgu03Xq245g.l.sQV4rNAuWuu2AzGF5PXFO4TJUKcWaEhQYy72 3qqSfckpjt4EHBftR.DLN79w1L3O4UiXnIa1muZfXwQmDdC7wMjkTFmajWT5SziTISYG16q3lE0b o4U1x6qYjUX4bqwuU1_oGvqQkr0k83qhJ5bsazyEZpJqxvR2EOK04BFhnTwh3MrK3i4fyu9XKcLD .6z4tgUJc3xp1atCPPwgk27itEqoeyTXGnJQAbDDWb5XU8CNuYF15CDDNKrU3E2sQO2YbmHLIdrn 9G2cnyycFMlmGl4Api2AHKHjLnaZ4YNsshoWYnSNCq2jm.uyV.I4ZpD_j49OGftKHS5waevCi1EX 0dDicyM.UN3GBnHi_zWibgAD3UQ1OjQWVg1f50VzHque17xp76gGxeEwsD9foKyE4bod0.KWfnrU vsnk48BKdlEIhq.OGrQCkwrk92qsAIezjnfEpxLxuh9kPL8A2u2327ygF.ncOOc_J8oHFGZMo.ZQ gzSMwGvPDZmbX0TZMhBTHIMNvSAidG7Q16PCEMt3eyqhGV28xe3kfInlgI.n56kTXi6Tci0jL55u gomFAAR0o0ArF0oVOG7QA8I0BVc9zS2BKr3E1mBiphsrzqVDXwk8sy25ASSzsHLsX07FcyiUXBer p2ucCg.MGWxSBmfhUXd.ykeclhnqI9gpTCjWeQQKtQCjLfW1bHFSoxmqnswyn.f5nMpfYZkKBi83 3QtWuD1H9MiAsCrZPwaBk5T.ToXu5lt8Wpa74q8GqFQXCHRzhTMQkwLL.MkNFHlCOd1AVgaTXTzD RWqVEjzumWrvhQHfmD_bd_h1fqPiP4iw1R.PMlt.6RO1W2XJznfheMPyl158D_nptz0JYzRs2I1F mwWL3iLODzIenL1GSVMYUqE1oydAQQ_GNXNT_F7kQ3kTGlFWXMeJ9_k4ViXoaq2mQuqdLMex9jcl VVoGmxCEzooHvUsQOihlHMgvfD7A2kBnsUi94liIjNNfqM4z6UnPGJ2f0wRXS0y2zDEHJRPPqWSy TVZiyjas2R36tDQn1tSXRupNydbztFtTnNoNgZY9IPqptZtWmnAMzgRn4YQe4Sz92iV6g4on2Ndh iwXI0MIxqEqVQeigqoMZzxfPM5Fyt2JT66aak1l1LTaSnvMarXV4EP30aIX7sWXvKmd2HeMimuUI MoSg_H5xuDJVT6gPuNYge2DnD_P8VnN924P7o20FfDYM6dj6x..biD52HvfBs_Rtg48nbHMH.lIe rDSkMzMPMAvOf2v6sCI4tf5OFI5Lu23uHx0PQI08VggWVLvkYCwmC_5YBCKyvOO2qBUPcdrRJS8s yaTjfBi0e3hSEaTb3GFHap90Y.gppH8L6OIIiuj_UynyftrYP4jEAlC2g2XyNfJQH28TDCeQIWQx Iq0xuNmM.t.7CA4aaPyKrDvj_a2wGqLZPLrcQrx9Ki5vnXGGuWL9twZSKnjC5ACo0TBWGuU3_ZyM e1OH40L4371OZgni8rxJbcb2g00AL9nxW3jba.jg5UjggDN3OwAJj0hrQ.gnyK9VqnuSIAhD.QqO J8NzlMsfvAJpE7bZUp884aDX5GioytjHJYUbNLGgL._Bo8uAmytBRhNP6cKov.N7NEC4IaQP6.mA JEcWGo5w1ssQW_NOvC.xLbdkkbPsny8bOyze6vrJ1sW6KUhqjqFt0kHdaaMFmvQAmKKTsZWCZwEh 7XNaECEwtjRH3A3p85dUTOF9T9g4pwbtYjURftG3qVXxNnJ_xQ7kLoUCyuR0ck_EmI9_c_nNalbY VvhlSR7siflFU5744xfy1ndHGSeCqskPMjF4IVZLrrlfQCRzFNTYP3xwBBxuqoUTXQu9aRCERvab WWx11u0TMOyBT7TUz2geTC5X69SOq0WpdapAoDdOSJPaCbAOkw15guI2fvq92AajFRZE1eCjEjl5 C6SHQYZWB2R4dTmU_vC1tmlyXE8_ZmDNcin_CmHo9tXqgY.x.a5rxkPtquUDmYAnUQxVh9bAiBQZ uftSTs7vfBnDsPs1hUWoOUf8ZTBV2OZQ.IMrSwHCSwjdI6uKg0hBgvDZt0cQkvapf.rE1OXDf7ic yxmP9ZkoM62x4dcGtJwxR0JSVro07P0aloVeC6VAgCSXG53XNO4T_PTHnvgd57.P7vMPGxuxux7b sr9CQYPAUTfQtnOg- X-Sonic-MF: X-Sonic-ID: 18bdaa88-2da8-4fe7-af1d-7060ca596b3b Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Thu, 7 Sep 2023 19:40:51 +0000 Received: by hermes--production-gq1-6b7c87dcf5-qfzfj (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID f3670204f7cff43689bc2a6e6d8c3804; Thu, 07 Sep 2023 19:40:47 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: <20230907184823.GC4090@FreeBSD.org> Date: Thu, 7 Sep 2023 12:40:36 -0700 Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek , Alexander Motin Content-Transfer-Encoding: quoted-printable Message-Id: References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> To: Glen Barber , Martin Matuska X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4RhV3B2D3qz4GW5 On Sep 7, 2023, at 11:48, Glen Barber wrote: > On Thu, Sep 07, 2023 at 11:17:22AM -0700, Mark Millard wrote: >> When I next have time, should I retry based on a more recent >> vintage of main that includes 969071be938c ? >>=20 >=20 > Yes, please, if you can. As stands, I rebooted that machine into my normal enviroment, so the after-crash-with-dump-info context is preserved. I'll presume lack of a need to preserve that context unless I hear otherwise. (But I'll work on this until later today.) Even my normal environment predates the commit in question by a few commits. So I'll end up doing a more general round of updates overall. Someone can let me know if there is a preference for debug over non-debug for the next test run. Looking at "git: 969071be938c - main", the relevant part seems to be just (white space possibly not preserved accurately): diff --git a/sys/kern/vfs_vnops.c b/sys/kern/vfs_vnops.c index 9fb5aee6a023..4e4161ef1a7f 100644 --- a/sys/kern/vfs_vnops.c +++ b/sys/kern/vfs_vnops.c @@ -3076,12 +3076,14 @@ vn_copy_file_range(struct vnode *invp, off_t = *inoffp, struct vnode *outvp, goto out; =20 /* - * If the two vnode are for the same file system, call + * If the two vnodes are for the same file system type, call * VOP_COPY_FILE_RANGE(), otherwise call = vn_generic_copy_file_range() - * which can handle copies across multiple file systems. + * which can handle copies across multiple file system types. */ *lenp =3D len; - if (invp->v_mount =3D=3D outvp->v_mount) + if (invp->v_mount =3D=3D outvp->v_mount || + strcmp(invp->v_mount->mnt_vfc->vfc_name, + outvp->v_mount->mnt_vfc->vfc_name) =3D=3D 0) error =3D VOP_COPY_FILE_RANGE(invp, inoffp, outvp, = outoffp, lenp, flags, incred, outcred, fsize_td); else That looks to call VOP_COPY_FILE_RANGE in more contexts and vn_generic_copy_file_range in fewer. The backtrace I reported involves: VOP_COPY_FILE_RANGE So it appears this change is unlikely to invalidate my test result, although failure might happen sooner if more VOP_COPY_FILE_RANGE calls happen with the newer code. That in turns means that someone may come up with some other change for me to test by the time I get around to setting up another test. Let me know if so. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Sep 7 20:07:55 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RhVg86Z0Hz4sh32; Thu, 7 Sep 2023 20:08:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RhVg84j9Dz4R46; Thu, 7 Sep 2023 20:08:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oi1-x234.google.com with SMTP id 5614622812f47-3ab3aa9ae33so983599b6e.2; Thu, 07 Sep 2023 13:08:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694117316; x=1694722116; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id:sender :from:to:cc:subject:date:message-id:reply-to; bh=qYhcbEsafebrUYIdq8f68T9hpIi/Uoo+JVfBQEkiuQw=; b=MIHw3rKlwGexVTimGLCebSIIRsUKQJ+tYV68V2W1lhGu3+hqBQ9ePiWL5FrdRJabla 2YRCjwv5GWZWMpzZX2ILTXI8uqSqI76DkjgSie2/EPgiEE4wuXuhCO65zbqD5fLB4wXX 0Andd73hMsi5lHM+ioLaxJ9uydIUSW91m8cs+laMZkpy5rDCV7PX9DK38AYF3zMPJDqa yz3H9hmrsu+6n6mJCHZTixROcLhsROayW8/6EH30fi7uIl3kz/38yTIyO3aqpiNVCV5g dFJRu3Ia1fqH7ED4g+ZWMLk4x8PO4dW8mUhGR/Te3sP1i1ncQ+TThmhrF2+Qc9egRK7I d6eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694117316; x=1694722116; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qYhcbEsafebrUYIdq8f68T9hpIi/Uoo+JVfBQEkiuQw=; b=QHfqVALxlfqh9L07XOIb21AwlBo6pUcpHJGiAhMU606kFs02TKvdVM7+KftreH3hni tbuNXlZxnhv/x8EsrSlF9bp3PNkksoh7SDOQ3Cztrebc0YrviBLQI2cvJkc3X5CUjJ1b o6mtKjjxxPzafiWp3vwHxVYtimsxkZPyADv1i5z5/zewlOT9YO6vJa21BpNICnSgeeqF 8nR4+0hiZ8m46DI+4ciwR65gzGmUbMjJJv4ptE21YruFz9h1VOUKXCf3GpFBV7j+G8om QR2sJU4KAVo937/pvEUch+srswzpzwhS0hG3cHfNMxNBGgvGWNoLh3qeLSFDGNbcGW7B wtYA== X-Gm-Message-State: AOJu0YwFZBg7p71ERpGQv8kCaV10jtYUIbhba9rZFAnZrwZbbQtWgTZs aOla51v5HyzIBqA+irlgkMhDRBVl/fX46A== X-Google-Smtp-Source: AGHT+IG1FSLvSjXyMr8r+ifF0ilm5Q34xaA1h2cXf3b6mEp/AgqG5Qbd7ZptnYpluEz6A6yC2qB3BQ== X-Received: by 2002:a05:6808:21a3:b0:3a1:dd99:8158 with SMTP id be35-20020a05680821a300b003a1dd998158mr826362oib.6.1694117315617; Thu, 07 Sep 2023 13:08:35 -0700 (PDT) Received: from [10.230.45.5] ([38.32.73.2]) by smtp.gmail.com with ESMTPSA id x188-20020a0deec5000000b00592236855cesm51245ywe.61.2023.09.07.13.08.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 07 Sep 2023 13:08:35 -0700 (PDT) Message-ID: <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> Date: Thu, 7 Sep 2023 16:07:55 -0400 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Content-Language: en-US To: Mark Millard , Glen Barber , Martin Matuska Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> From: Alexander Motin Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4RhVg84j9Dz4R46 Thanks, Mark. On 07.09.2023 15:40, Mark Millard wrote: > On Sep 7, 2023, at 11:48, Glen Barber wrote: > >> On Thu, Sep 07, 2023 at 11:17:22AM -0700, Mark Millard wrote: >>> When I next have time, should I retry based on a more recent >>> vintage of main that includes 969071be938c ? >> >> Yes, please, if you can. > > As stands, I rebooted that machine into my normal > enviroment, so the after-crash-with-dump-info > context is preserved. I'll presume lack of a need > to preserve that context unless I hear otherwise. > (But I'll work on this until later today.) > > Even my normal environment predates the commit in > question by a few commits. So I'll end up doing a > more general round of updates overall. > > Someone can let me know if there is a preference > for debug over non-debug for the next test run. It is not unknown when some bugs disappear once debugging is enabled due to different execution timings, but generally debug may to detect the problem closer to its origin instead of looking on random consequences. I am only starting to look on this report (unless Pawel or somebody beat me on it), and don't have additional requests yet, but if you can repeat the same with debug kernel (in-base ZFS's ZFS_DEBUG setting follows kernel's INVARIANTS), it may give us some additional information. > Looking at "git: 969071be938c - main", the relevant > part seems to be just (white space possibly not > preserved accurately): > > diff --git a/sys/kern/vfs_vnops.c b/sys/kern/vfs_vnops.c > index 9fb5aee6a023..4e4161ef1a7f 100644 > --- a/sys/kern/vfs_vnops.c > +++ b/sys/kern/vfs_vnops.c > @@ -3076,12 +3076,14 @@ vn_copy_file_range(struct vnode *invp, off_t *inoffp, struct vnode *outvp, > goto out; > > /* > - * If the two vnode are for the same file system, call > + * If the two vnodes are for the same file system type, call > * VOP_COPY_FILE_RANGE(), otherwise call vn_generic_copy_file_range() > - * which can handle copies across multiple file systems. > + * which can handle copies across multiple file system types. > */ > *lenp = len; > - if (invp->v_mount == outvp->v_mount) > + if (invp->v_mount == outvp->v_mount || > + strcmp(invp->v_mount->mnt_vfc->vfc_name, > + outvp->v_mount->mnt_vfc->vfc_name) == 0) > error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, > lenp, flags, incred, outcred, fsize_td); > else > > That looks to call VOP_COPY_FILE_RANGE in more contexts and > vn_generic_copy_file_range in fewer. > > The backtrace I reported involves: VOP_COPY_FILE_RANGE > So it appears this change is unlikely to invalidate my > test result, although failure might happen sooner if > more VOP_COPY_FILE_RANGE calls happen with the newer code. Your logic is likely right, but if you have block cloning requests both within and between datasets, this patch may change the pattern. Though it is obviously not a fix for the issue. I responded to the commit email only because it makes no difference while vfs.zfs.bclone_enabled is 0. > That in turns means that someone may come up with some > other change for me to test by the time I get around to > setting up another test. Let me know if so. -- Alexander Motin From nobody Thu Sep 7 23:32:34 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RhbBr6wLvz4sfRv for ; Thu, 7 Sep 2023 23:32:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RhbBr48pTz4TJf for ; Thu, 7 Sep 2023 23:32:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694129570; bh=h1aQmrPwATVZowD7blxy+TP63SaEbhwfe/6+Klbn7sY=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=hY5OHGq6Z197abcd5rwNaRlnjh2DuXW8Mrn7O1r12QoKkHrS/EFcgk4qcQij1I+Z9v9C3fDnDpqvmEA/1ksaHEg0HY8iGYxra+SM3Vc5h9VCHbG0tea/GBda/vo1ARbsKc7t7oA3WR0TQrp5TyIMld5S51swhGLsJO9oKqKpPML4cKhahucerUapLZf7mK1/A3LrFzAwV6a8TJ+aeUSr5KALV86AZto41z4oDBW+uV34o3YQyp8kDlj6XPx5+lTM8nFnuJVhwCxujVqU2iQIVzW9Idx7+aj+SxfU1cGbmGmnRsqL23lnAOn0N55rITS8mC9DCS1bd4Tz9PAr/5HUTg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694129570; bh=Au1pKf3QM0gKdWxsuhGp96p2UJ7zPBBvqXFmYcWajBG=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=IUZEvTdXQE1lj8Gxgl+IVG51RoNGr83GFiBrbf66Jjg2T3V5Niwi7MZOC+/d9dT0J4TcVb1L7gdLAM+JcXugevUBILwzFQa1pLU9Gi0rX91Tn2PYvMJ3/tQaN/4C7dev2wg4Ser4FwRDlQePE2a2Dh9zT8sGQjHSnLnbZ/v+p8rOgzqGEVzUqYGAlgJmb21yWbUld1B4qwiSEn2dhB22ajheTKZIS4YXuS7i5m1vUiFT3JRLh0GN9moQ+QJFOCJJUUrdF6wY9MdmjA3eutSNvjLHR0RNjotuXbiHnc4HGdQwlCCy5P/BgD22UpznRgbg9K4J83A/1eHAJfoO6Jmnog== X-YMail-OSG: xT_GXLEVM1lTy3ILszW9tsmN5Fm7x9FquhYBqyIqgZA4ko1rR4.vWjuiAF8ABj_ CEVR7FSz82uleVcVGCOIix3ei5Hvobblg0m2BOYnSGHeQ_0ovT.ooFgSACtXUAcWSzUVBldFAlDk Y.UsD_x0Zq6ClyGPsj6HU6azgNNxbz7bLnQEb0z5n.c0UOZNlQKIcYfppa0_wLRrSwZRm4.glVEN nXwevOwdIlJkKW_jX4HhkK1N2g8EQOFDrEUYu3.sFwPhNHeIu1LJoCvt0brr7tzItHG6DdWLyTR6 EWH9QODAamsTt968D5OS7ldWVsqLmJ4KdXQx1XVWKdEkKCKE.gbWutAjUW_Hlw_LnkySPyx96xHj LKnrfG6XMDK1XreiTbeLfwK6KT2FlSnvuhZQkWhg_VHLHFikx3uLHibmDM0Ack3TIAdYkbUxxl69 jgceb8Iwf8u_GT_mAEkvv6NC1hBVMsxGkUOAjYDo_elBKFuWbqgm7d4FGNAPyLzzWLDDhJqP2ZH7 CNnAOj8PhKNWq6Y0.WfH46kIkTE8jBv.PkjDAoLBR8.3Qn2zPKz.UOqHGfvAwVUWCnByAZ4ppz1U vsHxtteYea_aPTDxqEsI5NSsTE8.6deIzbHv2Z9fDopBW31lMhhn.0pyz5DZnX5wwgk8oo1ls9mI 6AqW5q06cJvxNH1Pizot2zh9DUzfgqhmPVtn7iQWFS.H5W_tQcY.ncdlpBovCxajwylU5moq3GKq NefQBRsiSSxFhJ4e6zH.WVUBdS8hEi5lzQrdprrL2raugxrMP.ttpTh0zq6jSQOluxq9t54Z8kSz _70uGUL1Hqp2oeJdRJL3L85Gvg1a7SmQaZz3geI0XGymwQb0hMDHGErU.m9saLN67lo4GMgDwQbI HfWn9oi9EcN1V8dqVuAwiiwP7KuxgOZO4NK5LOu7waHkPTvrkaVlAviJGd4aSB5A8jy6yI_ye0mw g6lBfVWqLxw74O6A58Cw0hQasOONF0gTdGQFVPNVw7ZV5YfYEi9Lef_VR_7NzQic4oJgkqW1j4.Z O80mb6zt2cXamRJbUqZaUU7XbrdhkKAEHY6XPWnWZD7hGSS.RGKrqO5ahdXzarEJTS.XIJadlm58 Hqv5v9qd3k6jw24snMSKNL7SgwU8O8kPbM7kCYqGwVfkEJl_AbktmosMw9ypv_3HYVsVr_KgxSHj XHYpQdifaAKDyfS5p2g.fLPYdjRpALVSeBxC2Ttbf17zgZxq94XKkij7GLBZpHF6k_NG0R7TSigH iVmcXL2FvnYPiURpT6VshCvXmiMR41SXYn3wsNcEyhwPBXXCEuPPmoYBYuKw14nd7lqT.dbJiUJ4 1UWdBd44v0ftO9LY4okU4H25PtiGd8sUaYoByilf_GSgvw.EWUNPz0pDFrdDpJkZmVI7SchwttPg CGPJX6b8qYLjcsU4OayJoGe_nSrKEg4zyOPo1jXL3I6ipjNxFFV4jfhImMM_THH2ZewtfU91kOKo YzQ6JXSm07HSylHj_.R9PSkjmjsCOxBu6sRe7SnyFvi.WKLAD5zquMUTEGN3kw.t4.QuZmJQlUzt zk1JDfc3TB00LUxEoDn6TeBHpegEmBuNsMA8QVThvZTswK7vRXL4ncel_mULAqNrFUz9Vq71xTuj Xb4vXG4p3xnnzVB32kuUjuNts3gl3BPxlyAFxTvLvDl3wf0KLuoHUG.BMG9B6XlaSHB3sW9toaFA HrjnfFzAgZLJcogtbnhqsOSpZhnye6zIz6dbynAaMrpw3RHc11nUFPBgIb81ReGK5Q_UVy.k0kX9 DxWmWsdL2pRMiJK8IqjmR.UhO_Mqgojdi7v3mYhfnf3ApKGV2rC1qZQ7NJBHNoWQ_FbRQBOrxqDa r0AHhdAAveRTRpqhqoBHa916MyPvp6trN2xRujyoJxGosVAa9Mf0S9_aRwbHJZX6w4PqGNZmvlHG dPtfSX2Ril2p.TQlWroU4qjZXUK6W7p.3OHSsYYi1WTK6Kc.rO0KL723Vd6RD.gMNsK4MQEie1Rl s_J2CQdqw00PmRtwskfFV3lQADfzyH_QGxCOjngfxAJVrBh5Ba1.Fhp49K_L7dLGa33d3aQ9TMSz H0QskMlp0rJ2WOzG8LwsXCvbbvLncNedvo6.RuAywAkP3Dphe0BBzrjv3JAHd2zJyWpBg7.esUDe FtuPVcx3kKUqI2afubRBz9OQpwjLJPEtqbYzjwfB_YtqUGTQu9SOWZL05Yzi.XID.Xsj47dCvz0h aHlzUx6SedPfFjNleqvslr1_HvIM8iJR9u70Dq1PoDY6tf5QFYCxJIHfzbj7d9Hoj.uRdGGJcy4V HwhDpZUtJ X-Sonic-MF: X-Sonic-ID: 47d7f830-3b57-42e9-9641-67c4340520e4 Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Thu, 7 Sep 2023 23:32:50 +0000 Received: by hermes--production-ne1-7b767b77cc-q899j (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ad8c6981922bf36ed77b9b2519e01bd1; Thu, 07 Sep 2023 23:32:45 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> Date: Thu, 7 Sep 2023 16:32:34 -0700 Cc: Glen Barber , Martin Matuska , Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Content-Transfer-Encoding: quoted-printable Message-Id: <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> To: Alexander Motin X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4RhbBr48pTz4TJf On Sep 7, 2023, at 13:07, Alexander Motin wrote: > Thanks, Mark. >=20 > On 07.09.2023 15:40, Mark Millard wrote: >> On Sep 7, 2023, at 11:48, Glen Barber wrote: >>> On Thu, Sep 07, 2023 at 11:17:22AM -0700, Mark Millard wrote: >>>> When I next have time, should I retry based on a more recent >>>> vintage of main that includes 969071be938c ? >>>=20 >>> Yes, please, if you can. >> As stands, I rebooted that machine into my normal >> enviroment, so the after-crash-with-dump-info >> context is preserved. I'll presume lack of a need >> to preserve that context unless I hear otherwise. >> (But I'll work on this until later today.) >> Even my normal environment predates the commit in >> question by a few commits. So I'll end up doing a >> more general round of updates overall. >> Someone can let me know if there is a preference >> for debug over non-debug for the next test run. >=20 > It is not unknown when some bugs disappear once debugging is enabled = due to different execution timings, but generally debug may to detect = the problem closer to its origin instead of looking on random = consequences. I am only starting to look on this report (unless Pawel or = somebody beat me on it), and don't have additional requests yet, but if = you can repeat the same with debug kernel (in-base ZFS's ZFS_DEBUG = setting follows kernel's INVARIANTS), it may give us some additional = information. So I did a zpool import, rewinding to the checkpoint. (This depends on the questionable zfs doing fully as desired for this. Notably the normal environment has vfs.zfs.bclone_enabled=3D0 , including when it was doing this activity.) My normal environment reported no problems. Note: the earlier snapshot from my first setup was still in place since it was made just before the original checkpoint used above. However, the rewind did remove the /var/crash/ material that had been added. I did the appropriate zfs mount. I installed a debug kernel and world to the import. Again, no problems reported. I did the appropriate zfs umount. I did the appropriate zpool export. I rebooted with the test media. # sysctl vfs.zfs.bclone_enabled vfs.zfs.bclone_enabled: 1 # zpool trim -w zamd64 # zpool checkpoint zamd64 # uname -apKU FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #74 = main-n265188-117c54a78ccd-dirty: Tue Sep 5 21:29:53 PDT 2023 = root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd= 64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 (So, before the 969071be938c vintage, same sources as for my last run but a debug build.) # poudriere bulk -jmain-amd64-bulk_a -a . . . [00:03:23] Building 34214 packages using up to 32 builders [00:03:23] Hit CTRL+t at any time to see build progress and stats [00:03:23] [01] [00:00:00] Builder starting [00:04:19] [01] [00:00:56] Builder started [00:04:20] [01] [00:00:01] Building ports-mgmt/pkg | pkg-1.20.6 [00:05:33] [01] [00:01:14] Finished ports-mgmt/pkg | pkg-1.20.6: Success [00:05:53] [01] [00:00:00] Building print/indexinfo | indexinfo-0.3.1 [00:05:53] [02] [00:00:00] Builder starting . . . [00:05:54] [32] [00:00:00] Builder starting [00:06:11] [01] [00:00:18] Finished print/indexinfo | indexinfo-0.3.1: = Success [00:06:12] [01] [00:00:00] Building devel/gettext-runtime | = gettext-runtime-0.22_1 [00:08:24] [01] [00:02:12] Finished devel/gettext-runtime | = gettext-runtime-0.22_1: Success [00:08:31] [01] [00:00:00] Building devel/libtextstyle | = libtextstyle-0.22 [00:10:06] [05] [00:04:13] Builder started [00:10:06] [05] [00:00:00] Building devel/autoconf-switch | = autoconf-switch-20220527 [00:10:06] [31] [00:04:12] Builder started [00:10:06] [31] [00:00:00] Building devel/libatomic_ops | = libatomic_ops-7.8.0 . . . Crashed again, with 158 *.pkg files in .building/All/ after rebooting. The crash is similar to the non-debug one. No extra output from the debug build. For reference: Unread portion of the kernel message buffer: panic: Solaris(panic): zfs: accessing past end of object 422/10b1c02 = (size=3D2560 access=3D2560+2560) cpuid =3D 15 time =3D 1694127988 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe02e783b5a0 vpanic() at vpanic+0x132/frame 0xfffffe02e783b6d0 panic() at panic+0x43/frame 0xfffffe02e783b730 vcmn_err() at vcmn_err+0xeb/frame 0xfffffe02e783b860 zfs_panic_recover() at zfs_panic_recover+0x59/frame 0xfffffe02e783b8c0 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0xb8/frame = 0xfffffe02e783b970 dmu_brt_clone() at dmu_brt_clone+0x61/frame 0xfffffe02e783b9f0 zfs_clone_range() at zfs_clone_range+0xaa3/frame 0xfffffe02e783bbc0 zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x18a/frame = 0xfffffe02e783bc40 vn_copy_file_range() at vn_copy_file_range+0x114/frame = 0xfffffe02e783bce0 kern_copy_file_range() at kern_copy_file_range+0x36c/frame = 0xfffffe02e783bdb0 sys_copy_file_range() at sys_copy_file_range+0x78/frame = 0xfffffe02e783be00 amd64_syscall() at amd64_syscall+0x14f/frame 0xfffffe02e783bf30 fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe02e783bf30 --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D = 0xa8a8d32b55a, rsp =3D 0xa8a8c8fa158, rbp =3D 0xa8a8c8fa5f0 --- KDB: enter: panic __curthread () at /usr/main-src/sys/amd64/include/pcpu_aux.h:57 57 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" = (offsetof(struct pcpu, (kgdb) #0 __curthread () at = /usr/main-src/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=3Dtextdump@entry=3D0) at /usr/main-src/sys/kern/kern_shutdown.c:405 #2 0xffffffff804a505a in db_dump (dummy=3D,=20 dummy2=3D, dummy3=3D, = dummy4=3D) at /usr/main-src/sys/ddb/db_command.c:591 #3 0xffffffff804a4e5d in db_command (last_cmdp=3D,=20 cmd_table=3D, dopager=3Dtrue) at /usr/main-src/sys/ddb/db_command.c:504 #4 0xffffffff804a4b1d in db_command_loop () at /usr/main-src/sys/ddb/db_command.c:551 #5 0xffffffff804a81f6 in db_trap (type=3D, = code=3D) at /usr/main-src/sys/ddb/db_main.c:268 #6 0xffffffff80bb0793 in kdb_trap (type=3Dtype@entry=3D3, = code=3Dcode@entry=3D0,=20 tf=3Dtf@entry=3D0xfffffe02e783b4e0) at = /usr/main-src/sys/kern/subr_kdb.c:790 #7 0xffffffff810623c9 in trap (frame=3D0xfffffe02e783b4e0) at /usr/main-src/sys/amd64/amd64/trap.c:608 #8 #9 kdb_enter (why=3D, msg=3D) at /usr/main-src/sys/kern/subr_kdb.c:556 #10 0xffffffff80b61b33 in vpanic (fmt=3D0xffffffff82c3914f "%s%s",=20 ap=3Dap@entry=3D0xfffffe02e783b710) at /usr/main-src/sys/kern/kern_shutdown.c:958 #11 0xffffffff80b61913 in panic ( fmt=3D0xffffffff82369800 = "W\346\025\201\377\377\377\377") at /usr/main-src/sys/kern/kern_shutdown.c:894 #12 0xffffffff8299464b in vcmn_err (ce=3D,=20 fmt=3D0xffffffff82c6f791 "zfs: accessing past end of object = %llx/%llx (size=3D%u access=3D%llu+%llu)", adx=3D0xfffffe02e783b8a0) at = /usr/main-src/sys/contrib/openzfs/module/os/freebsd/spl/spl_cmn_err.c:60 #13 0xffffffff82ab1ab9 in zfs_panic_recover ( fmt=3D0x12 ) at /usr/main-src/sys/contrib/openzfs/module/zfs/spa_misc.c:1594 #14 0xffffffff82a10138 in dmu_buf_hold_array_by_dnode = (dn=3D0xfffff80b9edd67f0,=20 offset=3Doffset@entry=3D2560, length=3Dlength@entry=3D2560, = read=3Dread@entry=3D0,=20 tag=3D0xffffffff82c1c507, = numbufsp=3Dnumbufsp@entry=3D0xfffffe02e783b9bc,=20 dbpp=3D0xfffffe02e783b9c0, flags=3D0) at /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:543 #15 0xffffffff82a13c91 in dmu_buf_hold_array (os=3D,=20 object=3D, read=3D0, numbufsp=3D0xfffffe02e783b9bc,=20= dbpp=3D0xfffffe02e783b9c0, offset=3D, = length=3D,=20 tag=3D) at /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:654 #16 dmu_brt_clone (os=3Dos@entry=3D0xfffff8011e989800, object=3D,=20 offset=3Doffset@entry=3D2560, length=3Dlength@entry=3D2560,=20 tx=3Dtx@entry=3D0xfffff81b68d0bb00, = bps=3Dbps@entry=3D0xfffffe04bcc60000, nbps=3D1,=20 replay=3D0) at = /usr/main-src/sys/contrib/openzfs/module/zfs/dmu.c:2301 #17 0xffffffff82b7e393 in zfs_clone_range (inzp=3D0xfffff805a65f7000,=20 inoffp=3D0xfffff805c5e13548, outzp=3D0xfffff809d42c2cb0,=20 outoffp=3D0xfffff80134c184f8, lenp=3Dlenp@entry=3D0xfffffe02e783bc00,=20= cr=3D0xfffff8041841f700) at /usr/main-src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1302 #18 0xffffffff829b77ca in zfs_freebsd_copy_file_range = (ap=3D0xfffffe02e783bc58) at = /usr/main-src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:629= 4 #19 0xffffffff80c74694 in VOP_COPY_FILE_RANGE (invp=3D,=20= inoffp=3D, outvp=3D0xffffffff81183b4f,=20 outvp@entry=3D0xfffffe02e783bce0, outoffp=3D0xffffffff812265a2, = lenp=3D0x0,=20 lenp@entry=3D0xfffffe02e783bc70, flags=3D0, = incred=3D0xfffff8041841f700,=20 outcred=3D0x0, fsizetd=3D0xfffffe021b5f4740) at ./vnode_if.h:2385 #20 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff80435eef8c0,=20 inoffp=3Dinoffp@entry=3D0xfffff805c5e13548, = outvp=3D0xffffffff81183b4f,=20 outvp@entry=3D0xfffff80ce8c8ee00, outoffp=3D0xffffffff812265a2, = lenp=3D0x0,=20 lenp@entry=3D0xfffffe02e783bd40, flags=3Dflags@entry=3D0,=20 incred=3D0xfffff8041841f700, outcred=3D0xfffff8041841f700,=20 fsize_td=3D0xfffffe021b5f4740) at = /usr/main-src/sys/kern/vfs_vnops.c:3085 #21 0xffffffff80c6f22c in kern_copy_file_range ( td=3Dtd@entry=3D0xfffffe021b5f4740, infd=3D,=20 inoffp=3D0xfffff805c5e13548, inoffp@entry=3D0x0, outfd=3D,=20 outoffp=3D0xffffffff812265a2, outoffp@entry=3D0x0, = len=3D9223372036854775807,=20 flags=3D0) at /usr/main-src/sys/kern/vfs_syscalls.c:4971 #22 0xffffffff80c6f338 in sys_copy_file_range (td=3D0xfffffe021b5f4740,=20= uap=3D0xfffffe021b5f4b40) at = /usr/main-src/sys/kern/vfs_syscalls.c:5009 #23 0xffffffff8106323f in syscallenter (td=3D0xfffffe021b5f4740) at /usr/main-src/sys/amd64/amd64/../../kern/subr_syscall.c:187 #24 amd64_syscall (td=3D0xfffffe021b5f4740, traced=3D0) at /usr/main-src/sys/amd64/amd64/trap.c:1197 #25 #26 0x00000a8a8d32b55a in ?? () Backtrace stopped: Cannot access memory at address 0xa8a8c8fa158 (kgdb)=20 >> Looking at "git: 969071be938c - main", the relevant >> part seems to be just (white space possibly not >> preserved accurately): >> diff --git a/sys/kern/vfs_vnops.c b/sys/kern/vfs_vnops.c >> index 9fb5aee6a023..4e4161ef1a7f 100644 >> --- a/sys/kern/vfs_vnops.c >> +++ b/sys/kern/vfs_vnops.c >> @@ -3076,12 +3076,14 @@ vn_copy_file_range(struct vnode *invp, off_t = *inoffp, struct vnode *outvp, >> goto out; >> /* >> - * If the two vnode are for the same file system, call >> + * If the two vnodes are for the same file system type, call >> * VOP_COPY_FILE_RANGE(), otherwise call = vn_generic_copy_file_range() >> - * which can handle copies across multiple file systems. >> + * which can handle copies across multiple file system types. >> */ >> *lenp =3D len; >> - if (invp->v_mount =3D=3D outvp->v_mount) >> + if (invp->v_mount =3D=3D outvp->v_mount || >> + strcmp(invp->v_mount->mnt_vfc->vfc_name, >> + outvp->v_mount->mnt_vfc->vfc_name) =3D=3D 0) >> error =3D VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, >> lenp, flags, incred, outcred, fsize_td); >> else >> That looks to call VOP_COPY_FILE_RANGE in more contexts and >> vn_generic_copy_file_range in fewer. >> The backtrace I reported involves: VOP_COPY_FILE_RANGE >> So it appears this change is unlikely to invalidate my >> test result, although failure might happen sooner if >> more VOP_COPY_FILE_RANGE calls happen with the newer code. >=20 > Your logic is likely right, but if you have block cloning requests = both within and between datasets, this patch may change the pattern. = Though it is obviously not a fix for the issue. I responded to the = commit email only because it makes no difference while = vfs.zfs.bclone_enabled is 0. >=20 >> That in turns means that someone may come up with some >> other change for me to test by the time I get around to >> setting up another test. Let me know if so. >=20 > --=20 > Alexander Motin =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Sep 8 04:32:09 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RhjrX5vKbz4sNGd for ; Fri, 8 Sep 2023 04:32:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-21.consmr.mail.gq1.yahoo.com (sonic314-21.consmr.mail.gq1.yahoo.com [98.137.69.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RhjrW5prlz3Rfd for ; Fri, 8 Sep 2023 04:32:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=Rly7Vloo; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694147545; bh=T/859hwkDHR2VGagWWbzN+fjv13WjUFSPw4ddwNJIIY=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=Rly7VloohIcKt8EesI4ZSWhplqv+bXmnKAoOiGVGtSqX/fETBtihJlBnEb3rL/AVC5jqRmM/78/Rz3ss6g7j6jHC+QH+ltrz8JCnoHs9hEVh/sk4j7ldmN2VzRDswtpaPBFxBmmLLiTtxvoV+juy8a7cHtRzZuQLl+2kWE3KCrUbQYjRg/lHogHuWG6YBHjRv2wdkjpxh96jb+OW6Jx6YS+fpznQl7os9G+SeElHOLhyP5s924HHxLnZMVDk/y20Ca9zRimEQbYPvpyJe66CPl9cvm6BOSseUefPkama/ziOiZwggQXcMghHLSQKYXzhhsHI/s/xK/W/kgOPq/naCQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694147545; bh=thI0DAxiZxrCFQSMHaBOatWwW7+D5yBRTnfQPiBHHeP=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=qMm2kEZfQTLCGrYKcW5sDYg2KpU8sOJVwUCOy7HJUJM+cgSU6Qq5B68bIiQBbfNEQgwB2/a1DeVe6LpCJTh5Pz7Tq/Z6na+1I4J6yaEKwm2YBgA9VcM/yA6m5DEZK3fQZO9D1PjiFdNlaE53JktCDjqEGNmHN/sYPMtITYHIgpplvD8d6RH97tJ1a4J21rLN+RH9SXNdJSu388u3zSnQKITg18sKUr/uXPZAw9ktb0sb5BNx8dFlj9OojHTMxXfdEnHfSc0oJcHfwULcfctGACzOxzl9UpXFbhkGMDGMolq2AS1ZTD8ai8pi8OPig1gPWvPvoeUtygel2KAekhvhnA== X-YMail-OSG: PWQ3i68VM1k8XJfD7_RzRicVaJrYYmjnPK7T6jqv3xvza.vEOhtw1fDT73b8_qp rUq29CLnzOok013gqBGpMq4EyctE4LcPVPjisLvDLSAfj0Hl1.yZdS4Q16ZmO9RRd5Cr1VwZ7MLP OGaECnqaQF1vH5fvMhy5EEMg5f1FJjluDi8KHRc6se3lYMHWw.1Bih2CXRphj30bxE5aQNTLkEaE xMHprZ9rulMstDQSXMXFMHPE3scr2Wqd3_XAKEAy5AW9Z1MF8R02lqbJ.DyESyjiyyThQStYE0an J_sdnUinjPjLyupxxMkpUPqHvYex0CY6oslxl6DpADI2C1uc7GV6GlN4It260jP3E03YUeuKY8.o HRXQfiYDl5miSf9h7l6TheFtYVMvCWy1PVSlfnJZatCXFNsM3HQMuSsTv3Frh_g5XJYQvbuTxmFc Jaml7seJSw_pXxfAUoTPmRIElvtYb3wZ7lSeuBKwWW6FCoCHkqs1SlStn_pgcGF85WqLHxColy_t qVvko9H9TA1uNAEQXVAhxJI37ZyIjWW4n92vR1rIubuTKp72nKrqlkBhFiX2aYjVnElmbQRuoJzW cJbrVY15LH02.BMbOHTvo9fZhYIwIaRHdhgJb3vlXuH6TE2lSIew8nyFKRObLgy_MtTXaZSv4KP9 gpSK6fLlSHYWxqRA14Q4nSgDmJCSTwwOSqyahUUsIf78x8AE36JKWtbOKpbJD.y8RcizC7Vb9tN4 rbpNxX3a3ZKdvjv_4QOFHKa5dH3dCVwJOYBoAGpi6gcs2JP6FuC9XkXcV2u6g8E2brmWizcCAH7p Pz39rjINGfR9z6sTAHuUxu4OGaHc0HNvg66R20zlMosLirN51CMYvEUr8oe_d.rx_20DAh7_nff7 ypNCd5EaSDBpdDBDnJwPaMx19HuiPAiSN9BrZ6Js6gqXu2Ps815JZk58Y8Y_V1qyJx59uBGPM30S VwBcf7PdikQr4KYwERKDrnSs..H.NAVtD0wDu3TUscgjsZR0IudHMjFsjosws3jH8CD2QhZE_Ush bpOgkjZkp3RScKT7E3fPa8KNRWd7SpXqQEvujwYxKRBb_wtbC3jWuWADh0yYmaNBBG__PPszpJa8 XE1rRZmwknoTn8FjWuNED0JqSiGKRmbeQdTdcZd5kTdOcKadZ4Y.s4u2f_gqz2fyHknYlxAilQk5 EK76SOW7LWuE5CQ8QyPzkmVWTMJOW46av0NK0iS4mkD6O9x9WHnU8hkC9pJQXH6bob1LuXYjz5Ol g3tXc4ivCJ4E5rOIMCKyKpLJdAP77a9yqHeA51PxxFkrujPR4jqxPCak2MQongoV8FYcXiTKetwP lTbq42kJOoMQywwE.sm74g77dZ4caGwPCoGU3hZgZ_4j57RSGBfsDTsezAWUj_gLGyGSYVYYBA6y ZunszwCPZcFmMrIzDoGLeH.bUt0a4xHc33bTwT0NbXZCiK_qBJK6uj1efQxPwRRk4l4cGTjACdHK 2h2FOeIAqdcJ_P0KbA5864eYIskpMNjhM6KrRyGrkqiPUASSpM8EP0xeGADRtLQdDpUFD3Cw6O7t TNTpitNRlQ3EoKS33LFBc5AM8OJCLn5i_wxwYmdL.r5OowM05ErxjfzTOkiumjUDP6ytFRdp2Q3V 5cwDkAPRfjpwIp66R3BI_KMYVF7tr72089Fn9uAe9yPEkPcGefWFo2Ysq0XYLX0c4oClUyPLW7Uf 8stDNxq13X2uNDSaVmP2dyAu2.agXWS1ynOD8q9OXC8ODI43y1lZ3.qrkz630VTfJ1rXXCj7hThV PaMBwLHFEGMvIsKLKV3HHpdEzQ3RPVt9xEHFuuZanBaRMqCzJBWH0Ph00zksrG9Kw2cx8A7EDpfr 2Yy9O9xwJoIvYNTGNV1uta.AyPQHD.RuhFBO3Qze2vdN9SZMBlcilpJZ2FHdulAvED24qDFk7Mrx ztwijNAY8zQXTrAlzdV3tR7SUxZ5cogoLgTd7yqpqa8163M1Ft1daXkGmIIZgrfGrrFBaWBxsjaI 6WtrxqcsiowCKAvvpityMF0PwL4NTIisJqyBa_u8Yx.Lixv1YV2deB75xpeYnPoy.2C7wsk_DkLZ iWuWZajR29lvlpbuUi5drCYTAYZ9O3yko84KiTh.rmp5HvrSXoe.LVvtXvrzslUWfUBB6.gHvgO. dJ3DAHQ94qlCqgjO3tgZMw1TnEB4N3STKGVX6m.EY_WeRggZWTXJaMRobT7AFVcpGxhBcotJBb2i WYXwalPXtKgJYydEfqTHkb1CmbWpsUAIYCFokrg1VXh.POvdmMSpOIeZtqkueYhiXOD5LMG.gwqy Q6DLjB3A6Eef71rXy X-Sonic-MF: X-Sonic-ID: e4541169-7798-47f3-b7a8-60622728fefc Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Fri, 8 Sep 2023 04:32:25 +0000 Received: by hermes--production-ne1-7b767b77cc-27nt8 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 4475f12bd510bf710db871c929ca1675; Fri, 08 Sep 2023 04:32:22 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> Date: Thu, 7 Sep 2023 21:32:09 -0700 Cc: Martin Matuska , Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Content-Transfer-Encoding: quoted-printable Message-Id: <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> To: Alexander Motin , Glen Barber X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.84:from]; BLOCKLISTDE_FAIL(0.00)[98.137.69.84:server fail]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.84:from]; TO_DN_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RhjrW5prlz3Rfd [Today's main-snapshot kernel panics as well.] On Sep 7, 2023, at 16:32, Mark Millard wrote: > On Sep 7, 2023, at 13:07, Alexander Motin wrote: >=20 >> Thanks, Mark. >>=20 >> On 07.09.2023 15:40, Mark Millard wrote: >>> On Sep 7, 2023, at 11:48, Glen Barber wrote: >>>> On Thu, Sep 07, 2023 at 11:17:22AM -0700, Mark Millard wrote: >>>>> When I next have time, should I retry based on a more recent >>>>> vintage of main that includes 969071be938c ? >>>>=20 >>>> Yes, please, if you can. >>> As stands, I rebooted that machine into my normal >>> enviroment, so the after-crash-with-dump-info >>> context is preserved. I'll presume lack of a need >>> to preserve that context unless I hear otherwise. >>> (But I'll work on this until later today.) >>> Even my normal environment predates the commit in >>> question by a few commits. So I'll end up doing a >>> more general round of updates overall. >>> Someone can let me know if there is a preference >>> for debug over non-debug for the next test run. >>=20 >> It is not unknown when some bugs disappear once debugging is enabled = due to different execution timings, but generally debug may to detect = the problem closer to its origin instead of looking on random = consequences. I am only starting to look on this report (unless Pawel or = somebody beat me on it), and don't have additional requests yet, but if = you can repeat the same with debug kernel (in-base ZFS's ZFS_DEBUG = setting follows kernel's INVARIANTS), it may give us some additional = information. >=20 > So I did a zpool import, rewinding to the checkpoint. > (This depends on the questionable zfs doing fully as > desired for this. Notably the normal environment has > vfs.zfs.bclone_enabled=3D0 , including when it was > doing this activity.) My normal environment reported > no problems. >=20 > Note: the earlier snapshot from my first setup was > still in place since it was made just before the > original checkpoint used above. >=20 > However, the rewind did remove the /var/crash/ > material that had been added. >=20 > I did the appropriate zfs mount. >=20 > I installed a debug kernel and world to the import. Again, > no problems reported. >=20 > I did the appropriate zfs umount. >=20 > I did the appropriate zpool export. >=20 > I rebooted with the test media. >=20 > # sysctl vfs.zfs.bclone_enabled > vfs.zfs.bclone_enabled: 1 >=20 > # zpool trim -w zamd64 >=20 > # zpool checkpoint zamd64 >=20 > # uname -apKU > FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #74 = main-n265188-117c54a78ccd-dirty: Tue Sep 5 21:29:53 PDT 2023 = root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd= 64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 >=20 > (So, before the 969071be938c vintage, same sources as for > my last run but a debug build.) >=20 > # poudriere bulk -jmain-amd64-bulk_a -a > . . . > [00:03:23] Building 34214 packages using up to 32 builders > [00:03:23] Hit CTRL+t at any time to see build progress and stats > [00:03:23] [01] [00:00:00] Builder starting > [00:04:19] [01] [00:00:56] Builder started > [00:04:20] [01] [00:00:01] Building ports-mgmt/pkg | pkg-1.20.6 > [00:05:33] [01] [00:01:14] Finished ports-mgmt/pkg | pkg-1.20.6: = Success > [00:05:53] [01] [00:00:00] Building print/indexinfo | indexinfo-0.3.1 > [00:05:53] [02] [00:00:00] Builder starting > . . . > [00:05:54] [32] [00:00:00] Builder starting > [00:06:11] [01] [00:00:18] Finished print/indexinfo | indexinfo-0.3.1: = Success > [00:06:12] [01] [00:00:00] Building devel/gettext-runtime | = gettext-runtime-0.22_1 > [00:08:24] [01] [00:02:12] Finished devel/gettext-runtime | = gettext-runtime-0.22_1: Success > [00:08:31] [01] [00:00:00] Building devel/libtextstyle | = libtextstyle-0.22 > [00:10:06] [05] [00:04:13] Builder started > [00:10:06] [05] [00:00:00] Building devel/autoconf-switch | = autoconf-switch-20220527 > [00:10:06] [31] [00:04:12] Builder started > [00:10:06] [31] [00:00:00] Building devel/libatomic_ops | = libatomic_ops-7.8.0 > . . . >=20 > Crashed again, with 158 *.pkg files in .building/All/ after > rebooting. >=20 > The crash is similar to the non-debug one. No extra output > from the debug build. >=20 > For reference: >=20 > Unread portion of the kernel message buffer: > panic: Solaris(panic): zfs: accessing past end of object 422/10b1c02 = (size=3D2560 access=3D2560+2560) > . . . Same world with newer snapshot main kernel that should be compatible with the world: # uname -apKU FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #0 = main-n265205-03a7c36ddbc0: Thu Sep 7 03:10:34 UTC 2023 = root@releng3.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC = amd64 amd64 1500000 1500000 Steps: #NOTE: (re)boot to normal environment #NOTE: login cd ~/artifacts/ #NOTE: as needed . . . fetch = http://ftp3.freebsd.org/pub/FreeBSD/snapshots/amd64/15.0-CURRENT/kernel.tx= z fetch = http://ftp3.freebsd.org/pub/FreeBSD/snapshots/amd64/15.0-CURRENT/kernel-db= g.txz fetch = http://ftp3.freebsd.org/pub/FreeBSD/snapshots/amd64/15.0-CURRENT/src.txz #NOTE: /zamd64-mnt is a pre-existing empty directory for such use: zpool import --rewind-to-checkpoint -f -N -R /zamd64-mnt -t zamd64 = zpamd64 zpool checkpoint zpamd64 #NOTE: My BE's for bectl use have mountpoint=3Dnone normally. zfs set mountpoint=3D/ zpamd64/ROOT/main-amd64 zfs mount zpamd64/ROOT/main-amd64 mv /zamd64-mnt/boot/kernel /zamd64-mnt/boot/kernel.mm tar -xpf kernel.txz -C /zamd64-mnt/ mv /zamd64-mnt/usr/lib/debug/boot/kernel = /zamd64-mnt/usr/lib/debug/boot/kernel.mm tar -xpf kernel-dbg.txz -C /zamd64-mnt/ zfs umount zpamd64/ROOT/main-amd64 zfs set mountpoint=3Dnone zpamd64/ROOT/main-amd64 zfs mount zpamd64/usr/src #NOTE: /zamd64-mnt/usr/src/ being empty initially #NOTE: My normal environment's source is in a worktree /usr/main-src/ tar -xpf src.txz -C /zamd64-mnt/ zfs umount zpamd64/usr/src zpool export zpamd64 #NOTE: reboot to zamd64 #(my test environment) #NOTE: login uname -apKU sysctl vfs.zfs.bclone_enabled zpool trim -w zamd64 poudriere bulk -jmain-amd64-bulk_a -a Paniced. #NOTE: dump #NOTE: reboot to zamd64 #(my test environment) #NOTE: login ls -Tlod = /usr/local/poudriere/data/packages/main-amd64-bulk_a-default/.building/All= /*.pkg | wc -l 122 less /var/crash/core.txt.3 . . . Unread portion of the kernel message buffer: panic: Solaris(panic): zfs: accessing past end of object 422/10d670b = (size=3D2560 access=3D2560+2560) cpuid =3D 18 time =3D 1694146387 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe033f721590 vpanic() at vpanic+0x132/frame 0xfffffe033f7216c0 panic() at panic+0x43/frame 0xfffffe033f721720 vcmn_err() at vcmn_err+0xeb/frame 0xfffffe033f721850 zfs_panic_recover() at zfs_panic_recover+0x59/frame 0xfffffe033f7218b0 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0xb8/frame = 0xfffffe033f721960 dmu_brt_clone() at dmu_brt_clone+0x61/frame 0xfffffe033f7219e0 zfs_clone_range() at zfs_clone_range+0xaa3/frame 0xfffffe033f721bb0 zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x18a/frame = 0xfffffe033f721c30 vn_copy_file_range() at vn_copy_file_range+0x163/frame = 0xfffffe033f721ce0 kern_copy_file_range() at kern_copy_file_range+0x36c/frame = 0xfffffe033f721db0 sys_copy_file_range() at sys_copy_file_range+0x78/frame = 0xfffffe033f721e00 amd64_syscall() at amd64_syscall+0x138/frame 0xfffffe033f721f30 fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe033f721f30 --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D = 0x2c05a58a355a, rsp =3D 0x2c05a4312138, rbp =3D 0x2c05a43125d0 --- KDB: enter: panic __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 57 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" = (offsetof(struct pcpu, (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=3Dtextdump@entry=3D0) at /usr/src/sys/kern/kern_shutdown.c:405 #2 0xffffffff804a401a in db_dump (dummy=3D, = dummy2=3D, dummy3=3D, dummy4=3D) at /usr/src/sys/ddb/db_command.c:591 #3 0xffffffff804a3e1d in db_command (last_cmdp=3D, = cmd_table=3D, dopager=3Dtrue) at /usr/src/sys/ddb/db_command.c:504 #4 0xffffffff804a3add in db_command_loop () at /usr/src/sys/ddb/db_command.c:551 #5 0xffffffff804a71b6 in db_trap (type=3D, = code=3D) at /usr/src/sys/ddb/db_main.c:268 #6 0xffffffff80b9dfb3 in kdb_trap (type=3Dtype@entry=3D3, = code=3Dcode@entry=3D0, tf=3Dtf@entry=3D0xfffffe033f7214d0) at = /usr/src/sys/kern/subr_kdb.c:790 #7 0xffffffff8104d579 in trap (frame=3D0xfffffe033f7214d0) at /usr/src/sys/amd64/amd64/trap.c:608 #8 #9 kdb_enter (why=3D, msg=3D) at /usr/src/sys/kern/subr_kdb.c:556 #10 0xffffffff80b4f683 in vpanic (fmt=3D0xffffffff8223d54e "%s%s", = ap=3Dap@entry=3D0xfffffe033f721700) at = /usr/src/sys/kern/kern_shutdown.c:958 #11 0xffffffff80b4f463 in panic ( fmt=3D0xffffffff8196c800 = "$\222\024\201\377\377\377\377") at /usr/src/sys/kern/kern_shutdown.c:894 #12 0xffffffff81f9964b in vcmn_err (ce=3D, = fmt=3D0xffffffff82274462 "zfs: accessing past end of object %llx/%llx = (size=3D%u access=3D%llu+%llu)", adx=3D0xfffffe033f721890) at = /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_cmn_err.c:60 #13 0xffffffff820b6a59 in zfs_panic_recover ( fmt=3D0x12 ) at /usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c:1594 #14 0xffffffff820150d8 in dmu_buf_hold_array_by_dnode = (dn=3D0xfffff802c3c41be8, offset=3Doffset@entry=3D2560, = length=3Dlength@entry=3D2560, read=3Dread@entry=3D0, = tag=3D0xffffffff8222139b, numbufsp=3Dnumbufsp@entry=3D0xfffffe033f7219ac, = dbpp=3D0xfffffe033f7219b0, flags=3D0) at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:543 #15 0xffffffff82018c31 in dmu_buf_hold_array (os=3D, = object=3D, read=3D0, numbufsp=3D0xfffffe033f7219ac, = dbpp=3D0xfffffe033f7219b0, offset=3D, length=3D, tag=3D) at = /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:654 #16 dmu_brt_clone (os=3Dos@entry=3D0xfffff8019696b800, object=3D, offset=3Doffset@entry=3D2560, length=3Dlength@entry=3D2560, = tx=3Dtx@entry=3D0xfffff81433e03d00, = bps=3Dbps@entry=3D0xfffffe0442ce5000, nbps=3D1, replay=3D0) at = /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2301 #17 0xffffffff82183333 in zfs_clone_range (inzp=3D0xfffff80e37f64000, = inoffp=3D0xfffff8019ab32b38, outzp=3D0xfffff80d931a5910, = outoffp=3D0xfffff80662d0a548, lenp=3Dlenp@entry=3D0xfffffe033f721bf0, = cr=3D0xfffff8051d1c4800) at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1302 #18 0xffffffff81fbc76a in zfs_freebsd_copy_file_range = (ap=3D0xfffffe033f721c48) at = /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294 #19 0xffffffff80c612a3 in VOP_COPY_FILE_RANGE (invp=3D0xfffff80eb09c08c0, = inoffp=3D0xfffff8019ab32b38, outvp=3D0xfffff80e7f998000, = outoffp=3D0xfffff80662d0a548, lenp=3D0xfffffe033f721d40, = incred=3D0xfffff8051d1c4800, flags=3D, = outcred=3D, fsizetd=3D) at = ./vnode_if.h:2385 #20 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff80eb09c08c0, = inoffp=3Dinoffp@entry=3D0xfffff8019ab32b38, = outvp=3Doutvp@entry=3D0xfffff80e7f998000, outoffp=3D0xfffff80662d0a548, = lenp=3Dlenp@entry=3D0xfffffe033f721d40, flags=3Dflags@entry=3D0, = incred=3D0xfffff8051d1c4800, outcred=3D0xfffff8051d1c4800, = fsize_td=3D0xfffffe033fdf1c80) at /usr/src/sys/kern/vfs_vnops.c:3087 #21 0xffffffff80c5be9c in kern_copy_file_range ( td=3Dtd@entry=3D0xfffffe033fdf1c80, infd=3D, = inoffp=3D0xfffff8019ab32b38, inoffp@entry=3D0x0, outfd=3D, outoffp=3D0xffffffff8120f47f, outoffp@entry=3D0x0, = len=3D9223372036854775807, flags=3D0) at = /usr/src/sys/kern/vfs_syscalls.c:4971 #22 0xffffffff80c5bfa8 in sys_copy_file_range (td=3D0xfffffe033fdf1c80, = uap=3D0xfffffe033fdf2080) at /usr/src/sys/kern/vfs_syscalls.c:5009 #23 0xffffffff8104e3d8 in syscallenter (td=3D) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:187 #24 amd64_syscall (td=3D0xfffffe033fdf1c80, traced=3D0) at /usr/src/sys/amd64/amd64/trap.c:1197 #25 #26 0x00002c05a58a355a in ?? () Backtrace stopped: Cannot access memory at address 0x2c05a4312138 (kgdb)=20 . . . So the problem is not special to my personal kernel builds. >>> Looking at "git: 969071be938c - main", the relevant >>> part seems to be just (white space possibly not >>> preserved accurately): >>> diff --git a/sys/kern/vfs_vnops.c b/sys/kern/vfs_vnops.c >>> index 9fb5aee6a023..4e4161ef1a7f 100644 >>> --- a/sys/kern/vfs_vnops.c >>> +++ b/sys/kern/vfs_vnops.c >>> @@ -3076,12 +3076,14 @@ vn_copy_file_range(struct vnode *invp, off_t = *inoffp, struct vnode *outvp, >>> goto out; >>> /* >>> - * If the two vnode are for the same file system, call >>> + * If the two vnodes are for the same file system type, call >>> * VOP_COPY_FILE_RANGE(), otherwise call = vn_generic_copy_file_range() >>> - * which can handle copies across multiple file systems. >>> + * which can handle copies across multiple file system types. >>> */ >>> *lenp =3D len; >>> - if (invp->v_mount =3D=3D outvp->v_mount) >>> + if (invp->v_mount =3D=3D outvp->v_mount || >>> + strcmp(invp->v_mount->mnt_vfc->vfc_name, >>> + outvp->v_mount->mnt_vfc->vfc_name) =3D=3D 0) >>> error =3D VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, >>> lenp, flags, incred, outcred, fsize_td); >>> else >>> That looks to call VOP_COPY_FILE_RANGE in more contexts and >>> vn_generic_copy_file_range in fewer. >>> The backtrace I reported involves: VOP_COPY_FILE_RANGE >>> So it appears this change is unlikely to invalidate my >>> test result, although failure might happen sooner if >>> more VOP_COPY_FILE_RANGE calls happen with the newer code. >>=20 >> Your logic is likely right, but if you have block cloning requests = both within and between datasets, this patch may change the pattern. = Though it is obviously not a fix for the issue. I responded to the = commit email only because it makes no difference while = vfs.zfs.bclone_enabled is 0. >>=20 >>> That in turns means that someone may come up with some >>> other change for me to test by the time I get around to >>> setting up another test. Let me know if so. >> . . . =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Sep 8 13:52:02 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rhyq83pfpz4rn8S; Fri, 8 Sep 2023 14:17:08 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rhyq82SkCz4gG4; Fri, 8 Sep 2023 14:17:08 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from sslproxy06.your-server.de ([78.46.172.3]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qebty-000JMG-Vh; Fri, 08 Sep 2023 15:52:03 +0200 Received: from [188.167.171.2] (helo=[10.0.9.122]) by sslproxy06.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qebty-000IE2-M2; Fri, 08 Sep 2023 15:52:02 +0200 Message-ID: <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> Date: Fri, 8 Sep 2023 15:52:02 +0200 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics To: Mark Millard , Alexander Motin , Glen Barber Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> Content-Language: en-US From: Martin Matuska In-Reply-To: <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Authenticated-Sender: martin@matuska.de X-Virus-Scanned: Clear (ClamAV 0.103.8/27025/Fri Sep 8 09:37:45 2023) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE] X-Rspamd-Queue-Id: 4Rhyq82SkCz4gG4 I digged a little and was able to reproduce the panic without poudriere with a shell script. You may want to increase "repeats". The script causes the panic in dmu_buf_hold_array_by_dnode() on my VirtualBox with the cat command on 9th iteration. Here is the script: #!/bin/sh nl=' ' sed_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/ for ac_i in 1 2 3 4 5 6 7; do     sed_script="$sed_script$nl$sed_script" done echo "$sed_script" 2>/dev/null | sed 99q >conftest.sed repeats=8 count=0 echo -n 0123456789 >"conftest.in" while : do     cat "conftest.in" "conftest.in" >"conftest.tmp"     mv "conftest.tmp" "conftest.in"     cp "conftest.in" "conftest.nl"     echo '' >> "conftest.nl"     sed -f conftest.sed < "conftest.nl" >"conftest.out" 2>/dev/null || break     diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break     count=$(($count + 1))     echo "count: $count"     # 10*(2^10) chars as input seems more than enough     test $count -gt $repeats && break done rm -f conftest.in conftest.tmp conftest.nl conftest.out On 8. 9. 2023 6:32, Mark Millard wrote: > [Today's main-snapshot kernel panics as well.] > > On Sep 7, 2023, at 16:32, Mark Millard wrote: > >> On Sep 7, 2023, at 13:07, Alexander Motin wrote: >> >>> Thanks, Mark. >>> >>> On 07.09.2023 15:40, Mark Millard wrote: >>>> On Sep 7, 2023, at 11:48, Glen Barber wrote: >>>>> On Thu, Sep 07, 2023 at 11:17:22AM -0700, Mark Millard wrote: >>>>>> When I next have time, should I retry based on a more recent >>>>>> vintage of main that includes 969071be938c ? >>>>> Yes, please, if you can. >>>> As stands, I rebooted that machine into my normal >>>> enviroment, so the after-crash-with-dump-info >>>> context is preserved. I'll presume lack of a need >>>> to preserve that context unless I hear otherwise. >>>> (But I'll work on this until later today.) >>>> Even my normal environment predates the commit in >>>> question by a few commits. So I'll end up doing a >>>> more general round of updates overall. >>>> Someone can let me know if there is a preference >>>> for debug over non-debug for the next test run. >>> It is not unknown when some bugs disappear once debugging is enabled due to different execution timings, but generally debug may to detect the problem closer to its origin instead of looking on random consequences. I am only starting to look on this report (unless Pawel or somebody beat me on it), and don't have additional requests yet, but if you can repeat the same with debug kernel (in-base ZFS's ZFS_DEBUG setting follows kernel's INVARIANTS), it may give us some additional information. >> So I did a zpool import, rewinding to the checkpoint. >> (This depends on the questionable zfs doing fully as >> desired for this. Notably the normal environment has >> vfs.zfs.bclone_enabled=0 , including when it was >> doing this activity.) My normal environment reported >> no problems. >> >> Note: the earlier snapshot from my first setup was >> still in place since it was made just before the >> original checkpoint used above. >> >> However, the rewind did remove the /var/crash/ >> material that had been added. >> >> I did the appropriate zfs mount. >> >> I installed a debug kernel and world to the import. Again, >> no problems reported. >> >> I did the appropriate zfs umount. >> >> I did the appropriate zpool export. >> >> I rebooted with the test media. >> >> # sysctl vfs.zfs.bclone_enabled >> vfs.zfs.bclone_enabled: 1 >> >> # zpool trim -w zamd64 >> >> # zpool checkpoint zamd64 >> >> # uname -apKU >> FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #74 main-n265188-117c54a78ccd-dirty: Tue Sep 5 21:29:53 PDT 2023 root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 >> >> (So, before the 969071be938c vintage, same sources as for >> my last run but a debug build.) >> >> # poudriere bulk -jmain-amd64-bulk_a -a >> . . . >> [00:03:23] Building 34214 packages using up to 32 builders >> [00:03:23] Hit CTRL+t at any time to see build progress and stats >> [00:03:23] [01] [00:00:00] Builder starting >> [00:04:19] [01] [00:00:56] Builder started >> [00:04:20] [01] [00:00:01] Building ports-mgmt/pkg | pkg-1.20.6 >> [00:05:33] [01] [00:01:14] Finished ports-mgmt/pkg | pkg-1.20.6: Success >> [00:05:53] [01] [00:00:00] Building print/indexinfo | indexinfo-0.3.1 >> [00:05:53] [02] [00:00:00] Builder starting >> . . . >> [00:05:54] [32] [00:00:00] Builder starting >> [00:06:11] [01] [00:00:18] Finished print/indexinfo | indexinfo-0.3.1: Success >> [00:06:12] [01] [00:00:00] Building devel/gettext-runtime | gettext-runtime-0.22_1 >> [00:08:24] [01] [00:02:12] Finished devel/gettext-runtime | gettext-runtime-0.22_1: Success >> [00:08:31] [01] [00:00:00] Building devel/libtextstyle | libtextstyle-0.22 >> [00:10:06] [05] [00:04:13] Builder started >> [00:10:06] [05] [00:00:00] Building devel/autoconf-switch | autoconf-switch-20220527 >> [00:10:06] [31] [00:04:12] Builder started >> [00:10:06] [31] [00:00:00] Building devel/libatomic_ops | libatomic_ops-7.8.0 >> . . . >> >> Crashed again, with 158 *.pkg files in .building/All/ after >> rebooting. >> >> The crash is similar to the non-debug one. No extra output >> from the debug build. >> >> For reference: >> >> Unread portion of the kernel message buffer: >> panic: Solaris(panic): zfs: accessing past end of object 422/10b1c02 (size=2560 access=2560+2560) >> . . . > Same world with newer snapshot main kernel that should > be compatible with the world: > > # uname -apKU > FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #0 main-n265205-03a7c36ddbc0: Thu Sep 7 03:10:34 UTC 2023 root@releng3.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 amd64 1500000 1500000 > > > Steps: > > #NOTE: (re)boot to normal environment > #NOTE: login > > cd ~/artifacts/ > > #NOTE: as needed . . . > fetch http://ftp3.freebsd.org/pub/FreeBSD/snapshots/amd64/15.0-CURRENT/kernel.txz > fetch http://ftp3.freebsd.org/pub/FreeBSD/snapshots/amd64/15.0-CURRENT/kernel-dbg.txz > fetch http://ftp3.freebsd.org/pub/FreeBSD/snapshots/amd64/15.0-CURRENT/src.txz > > #NOTE: /zamd64-mnt is a pre-existing empty directory for such use: > zpool import --rewind-to-checkpoint -f -N -R /zamd64-mnt -t zamd64 zpamd64 > zpool checkpoint zpamd64 > > #NOTE: My BE's for bectl use have mountpoint=none normally. > zfs set mountpoint=/ zpamd64/ROOT/main-amd64 > zfs mount zpamd64/ROOT/main-amd64 > > mv /zamd64-mnt/boot/kernel /zamd64-mnt/boot/kernel.mm > tar -xpf kernel.txz -C /zamd64-mnt/ > > mv /zamd64-mnt/usr/lib/debug/boot/kernel /zamd64-mnt/usr/lib/debug/boot/kernel.mm > tar -xpf kernel-dbg.txz -C /zamd64-mnt/ > > zfs umount zpamd64/ROOT/main-amd64 > zfs set mountpoint=none zpamd64/ROOT/main-amd64 > > zfs mount zpamd64/usr/src > #NOTE: /zamd64-mnt/usr/src/ being empty initially > #NOTE: My normal environment's source is in a worktree /usr/main-src/ > tar -xpf src.txz -C /zamd64-mnt/ > zfs umount zpamd64/usr/src > > zpool export zpamd64 > > #NOTE: reboot to zamd64 #(my test environment) > #NOTE: login > > uname -apKU > > sysctl vfs.zfs.bclone_enabled > > zpool trim -w zamd64 > poudriere bulk -jmain-amd64-bulk_a -a > > Paniced. > > #NOTE: dump > #NOTE: reboot to zamd64 #(my test environment) > #NOTE: login > > ls -Tlod /usr/local/poudriere/data/packages/main-amd64-bulk_a-default/.building/All/*.pkg | wc -l > 122 > > less /var/crash/core.txt.3 > . . . > Unread portion of the kernel message buffer: > panic: Solaris(panic): zfs: accessing past end of object 422/10d670b (size=2560 access=2560+2560) > cpuid = 18 > time = 1694146387 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe033f721590 > vpanic() at vpanic+0x132/frame 0xfffffe033f7216c0 > panic() at panic+0x43/frame 0xfffffe033f721720 > vcmn_err() at vcmn_err+0xeb/frame 0xfffffe033f721850 > zfs_panic_recover() at zfs_panic_recover+0x59/frame 0xfffffe033f7218b0 > dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0xb8/frame 0xfffffe033f721960 > dmu_brt_clone() at dmu_brt_clone+0x61/frame 0xfffffe033f7219e0 > zfs_clone_range() at zfs_clone_range+0xaa3/frame 0xfffffe033f721bb0 > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe033f721c30 > vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe033f721ce0 > kern_copy_file_range() at kern_copy_file_range+0x36c/frame 0xfffffe033f721db0 > sys_copy_file_range() at sys_copy_file_range+0x78/frame 0xfffffe033f721e00 > amd64_syscall() at amd64_syscall+0x138/frame 0xfffffe033f721f30 > fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe033f721f30 > --- syscall (569, FreeBSD ELF64, copy_file_range), rip = 0x2c05a58a355a, rsp = 0x2c05a4312138, rbp = 0x2c05a43125d0 --- > KDB: enter: panic > > __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > 57 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu, > (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > #1 doadump (textdump=textdump@entry=0) > at /usr/src/sys/kern/kern_shutdown.c:405 > #2 0xffffffff804a401a in db_dump (dummy=, dummy2=, dummy3=, dummy4=) > at /usr/src/sys/ddb/db_command.c:591 > #3 0xffffffff804a3e1d in db_command (last_cmdp=, cmd_table=, dopager=true) > at /usr/src/sys/ddb/db_command.c:504 > #4 0xffffffff804a3add in db_command_loop () > at /usr/src/sys/ddb/db_command.c:551 > #5 0xffffffff804a71b6 in db_trap (type=, code=) > at /usr/src/sys/ddb/db_main.c:268 > #6 0xffffffff80b9dfb3 in kdb_trap (type=type@entry=3, code=code@entry=0, tf=tf@entry=0xfffffe033f7214d0) at /usr/src/sys/kern/subr_kdb.c:790 > #7 0xffffffff8104d579 in trap (frame=0xfffffe033f7214d0) > at /usr/src/sys/amd64/amd64/trap.c:608 > #8 > #9 kdb_enter (why=, msg=) > at /usr/src/sys/kern/subr_kdb.c:556 > #10 0xffffffff80b4f683 in vpanic (fmt=0xffffffff8223d54e "%s%s", ap=ap@entry=0xfffffe033f721700) at /usr/src/sys/kern/kern_shutdown.c:958 > #11 0xffffffff80b4f463 in panic ( > fmt=0xffffffff8196c800 "$\222\024\201\377\377\377\377") > at /usr/src/sys/kern/kern_shutdown.c:894 > #12 0xffffffff81f9964b in vcmn_err (ce=, fmt=0xffffffff82274462 "zfs: accessing past end of object %llx/%llx (size=%u access=%llu+%llu)", adx=0xfffffe033f721890) > at /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_cmn_err.c:60 > #13 0xffffffff820b6a59 in zfs_panic_recover ( > fmt=0x12 ) > at /usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c:1594 > #14 0xffffffff820150d8 in dmu_buf_hold_array_by_dnode (dn=0xfffff802c3c41be8, offset=offset@entry=2560, length=length@entry=2560, read=read@entry=0, tag=0xffffffff8222139b, numbufsp=numbufsp@entry=0xfffffe033f7219ac, dbpp=0xfffffe033f7219b0, flags=0) > at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:543 > #15 0xffffffff82018c31 in dmu_buf_hold_array (os=, object=, read=0, numbufsp=0xfffffe033f7219ac, dbpp=0xfffffe033f7219b0, offset=, length=, tag=) at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:654 > #16 dmu_brt_clone (os=os@entry=0xfffff8019696b800, object=, offset=offset@entry=2560, length=length@entry=2560, tx=tx@entry=0xfffff81433e03d00, bps=bps@entry=0xfffffe0442ce5000, nbps=1, replay=0) at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2301 > #17 0xffffffff82183333 in zfs_clone_range (inzp=0xfffff80e37f64000, inoffp=0xfffff8019ab32b38, outzp=0xfffff80d931a5910, outoffp=0xfffff80662d0a548, lenp=lenp@entry=0xfffffe033f721bf0, cr=0xfffff8051d1c4800) > at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1302 > #18 0xffffffff81fbc76a in zfs_freebsd_copy_file_range (ap=0xfffffe033f721c48) > at /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294 > #19 0xffffffff80c612a3 in VOP_COPY_FILE_RANGE (invp=0xfffff80eb09c08c0, inoffp=0xfffff8019ab32b38, outvp=0xfffff80e7f998000, outoffp=0xfffff80662d0a548, lenp=0xfffffe033f721d40, incred=0xfffff8051d1c4800, flags=, outcred=, fsizetd=) at ./vnode_if.h:2385 > #20 vn_copy_file_range (invp=invp@entry=0xfffff80eb09c08c0, inoffp=inoffp@entry=0xfffff8019ab32b38, outvp=outvp@entry=0xfffff80e7f998000, outoffp=0xfffff80662d0a548, lenp=lenp@entry=0xfffffe033f721d40, flags=flags@entry=0, incred=0xfffff8051d1c4800, outcred=0xfffff8051d1c4800, fsize_td=0xfffffe033fdf1c80) at /usr/src/sys/kern/vfs_vnops.c:3087 > #21 0xffffffff80c5be9c in kern_copy_file_range ( > td=td@entry=0xfffffe033fdf1c80, infd=, inoffp=0xfffff8019ab32b38, inoffp@entry=0x0, outfd=, outoffp=0xffffffff8120f47f, outoffp@entry=0x0, len=9223372036854775807, flags=0) at /usr/src/sys/kern/vfs_syscalls.c:4971 > #22 0xffffffff80c5bfa8 in sys_copy_file_range (td=0xfffffe033fdf1c80, uap=0xfffffe033fdf2080) at /usr/src/sys/kern/vfs_syscalls.c:5009 > #23 0xffffffff8104e3d8 in syscallenter (td=) > at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:187 > #24 amd64_syscall (td=0xfffffe033fdf1c80, traced=0) > at /usr/src/sys/amd64/amd64/trap.c:1197 > #25 > #26 0x00002c05a58a355a in ?? () > Backtrace stopped: Cannot access memory at address 0x2c05a4312138 > (kgdb) > . . . > > > So the problem is not special to my personal kernel builds. > > >>>> Looking at "git: 969071be938c - main", the relevant >>>> part seems to be just (white space possibly not >>>> preserved accurately): >>>> diff --git a/sys/kern/vfs_vnops.c b/sys/kern/vfs_vnops.c >>>> index 9fb5aee6a023..4e4161ef1a7f 100644 >>>> --- a/sys/kern/vfs_vnops.c >>>> +++ b/sys/kern/vfs_vnops.c >>>> @@ -3076,12 +3076,14 @@ vn_copy_file_range(struct vnode *invp, off_t *inoffp, struct vnode *outvp, >>>> goto out; >>>> /* >>>> - * If the two vnode are for the same file system, call >>>> + * If the two vnodes are for the same file system type, call >>>> * VOP_COPY_FILE_RANGE(), otherwise call vn_generic_copy_file_range() >>>> - * which can handle copies across multiple file systems. >>>> + * which can handle copies across multiple file system types. >>>> */ >>>> *lenp = len; >>>> - if (invp->v_mount == outvp->v_mount) >>>> + if (invp->v_mount == outvp->v_mount || >>>> + strcmp(invp->v_mount->mnt_vfc->vfc_name, >>>> + outvp->v_mount->mnt_vfc->vfc_name) == 0) >>>> error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, >>>> lenp, flags, incred, outcred, fsize_td); >>>> else >>>> That looks to call VOP_COPY_FILE_RANGE in more contexts and >>>> vn_generic_copy_file_range in fewer. >>>> The backtrace I reported involves: VOP_COPY_FILE_RANGE >>>> So it appears this change is unlikely to invalidate my >>>> test result, although failure might happen sooner if >>>> more VOP_COPY_FILE_RANGE calls happen with the newer code. >>> Your logic is likely right, but if you have block cloning requests both within and between datasets, this patch may change the pattern. Though it is obviously not a fix for the issue. I responded to the commit email only because it makes no difference while vfs.zfs.bclone_enabled is 0. >>> >>>> That in turns means that someone may come up with some >>>> other change for me to test by the time I get around to >>>> setting up another test. Let me know if so. >>> . . . > > === > Mark Millard > marklmi at yahoo.com > > From nobody Fri Sep 8 14:49:25 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RhzXm6G6yz4sB0Z for ; Fri, 8 Sep 2023 14:49:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-20.consmr.mail.gq1.yahoo.com (sonic305-20.consmr.mail.gq1.yahoo.com [98.137.64.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RhzXm38qNz3P3M for ; Fri, 8 Sep 2023 14:49:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694184581; bh=2Z4xapkPfoaIRV02NEk0K2X6SgdAMsAIQZ6OJIiM1WA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=IErHXnYm6XEnxviScSqdDRkm9XU5fGQacR6HQD+i7ushtz3x2c367v7HTthEboOq0BywLZhgzjeMRtdy+fq63ddbGlDG3ug5TidE/ckhZ16VnDJ6G9d96zfFC1cN93NXRVLKYIEkINGAQO60BFFsSUHn1sfmOGZDStd7QBw84izabmAgnadM/qacB2Scx29OFgDmFwnUti9RZ5Ra5oW/x6KDWtXHSsfQe5x66sXpJx77vlqfewDWVaeL1Y7lhDsDmP8uL5UGy/hTgUELFsh/Sgpol6HUj9soUF9iCGDV063xnFAT91/xPowl1fyRMKcFZzuTojyU7Xm0CD/SrY5j+A== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694184581; bh=9mtiyNMeT39N1n22jvLUUl8KGnGKHr+AxXpis9329qC=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=UQi92dPruxwXi2H2uWSI/moJLzetsZN1GfiLDMNFt9vDJLqlzMABQJ3+w4ymISMMkUXGxKD2p0MMM0YJEu2XFGqJ00c0A/ZsbqBD0dUI2SnPGp+Mv5xiulUq0qJiMk27FpOCVGF6RbiS/rxgYl7Lz7vmI602EOuETExbPnbkmoQ9K4PhTmYuHe0QUR2HRJVIw/2vxorSUT/lAUyIMcLpixerry6To7P979ZXWkM4HlxCB5wlMlMPXHoeONqNzstCMwFZF6+KBOkzetxuJHdKWv4nSthxb3FniRA/D6zB9DTU20D/hkyAq9tyf398kjgv1v1yrJQhZGRrTqkjFcB/vA== X-YMail-OSG: 5pgSvooVM1kH1PtzXznp8vwCOd7W8vLCFg7pgqtbbX_Z8l77HAP4Q759nQElY31 mS9q6DLy_4EqetglymwNdYDQmtwYmhoibvzShj24yUGbggkrCu5Xe9AXwBS01anU5U65pgETiBVO ApNL7RSwwi8vsYPZZCG2iQsJTqSo7CuTEblEnZiqT5rerdkIoCMIwq7MDUElqV9v55JG.T3wSKQT 1..4XYn9Zl6gpIXNYEh5BEgqk7H1pJTHvTu5gCVT.iA8BR9fZT47ItBoAOlb4cwFZzKsUPOPaIid ngzc5Zj7xfBPi3NVTzKoRLXdlBI9l3N48wzoRpzGFz1gH003XXjMxcD4.YgozZDJVVcg9AxR4tqC ld5372wrs4iv5fuAxlrMpVz8FVj8T3rvEVYbVXHr_.i9PyUFSGsm5NaoLm2QXn2KZiuAY_R2IYUh oN5m58cs53kAHrFwmF51NjICV7UyM4IyjjtVM3vi2_IBj1xTiPOShXlr4LmQOAwRp7HMMTUVegCX qNXmtrc9MLIsEsGyrapcRh8hVzCyGKo2iND4UMAJjlPHBBxLy2xJqMGcMd1D75m.YT4XB4fRwEfT Bjoxy1ag5ZtC17g1hkhrrMPweYgHhrBv3UgUNOZFJwdEmWLS0D4s8xs8fpTR1JeFb20cL_DGiD.H 1fH4BbgYIQSbSYVXHo3qKwdvnw6UatFAf.Q22AO61esUlh7PGedZ90Mg2gJ1VOa9nYO2vrrRHb_G BJxaWiI8bp3pVyjHAMUO09n1dMb.cpzK.vqthosPnYhPxQ0PsjMMpvabPqlnxIZXgyyYiizN.jXg 8BA7cnO.M47gzEO.yYwyAvFUDufL9O8ODAz9sWLYwWQ1ecxMoFJpB_Cm0suGzaIpLgTdjhiQc52o cgkjWvEXgL3YbJDFlNeKkCWl07UvJL6YILsMjo79m0pZeHBTo_JX7NCTjEveAFGW1qHfw2r3XDLY T76_z7bTX3ym895pWKSFDGq8eYyk51_3WLwrSCGi_t7YPbhe9S7sfauVANJnGraZPtlIWjqxgZ0E lW6z68s5f11PtWw_zQWuGYCcRAYahGzQU5XOCobcWpJjMujqmnOhLIGkJ87xPhjJnZCwEIQppBZC JJJhm7FhuA6O4oOf3wgfDVu.ugVHJvAvwQPKyNrjbRTivHLgrw03w75P6PuaIFOWsvhFn6X22xd1 a5sv1TxXyeSlDYKHPwqpPczCvkCWaWH2pisvB15QEL.ccgnQeAO1kAHOj4MulHOkeTPXO2Axz2ct .Ils4887B_g4EKhBY9aOBsVWI2z_epKPMHDank6H6TUF9m6ArLnTdBvlAHqGaXfusMZsCdkiCARh j6s_pnmQ98nl2ZUKyxjN7BRwFDUtG5fjOxSQTN4BR_dy6WkzavAPQnFv_cXHtQbYyREUBcIq95bn M8lEmA0bcJz3mVjjy.QeM1CevHGukYQiCTmtbJFHQPMKrZW9_2EvnpE1LhX1v4ecwlTqQTwdhJxM hT4He3ffF0fVin3w4gqJg0VRKa5f03nLteL.jCcn_Ekio5zCX1UyjVGP2qtliAMp2P3O_3YrbOM6 ZvjlUo34Fxv5Bu2sPaE_WiRqQa_RMLDIT4Rl3Jw2tv5BtT.Utbsun1RD58U5zc.Izj.AkrkmC8Te Dpzz6OoeKhCBxFRElSOlcFNFhXpq3iV5QkbAdipBYV79olv_tnyqqS1GxwedKBG6uy4H6_y9Eky6 yQYxVSBGRBy_tNu0JhRG9zZBm_aWA9FyJtvvVPMcXRbCPXMCnEvUyClIbnBScSPcRMjYX53mOLn2 QvuGDkZG.0XfPt0KPd8YENNdCMs0RLfHXqPJbmQtrbdygfKSlhzuftjnyFmjJ9RFJaxKfj3Jg0HX UqOf8IONNfVKb0jyT_t_7.2Gw2keBdzHLFRo9dQu5WYlQHhV9swZbZW_S4zIEkRPigsuCUQvgd_S qePwnWIE2VH3fASsWDunME3E0ezP34MAxOUEmCX3u55pBg1K2sc_zP0ZeW0fc1GZ.0JPvCMI3FF8 p1duezu74V.hX23ggItL9BdYZtdmsXlSHrxhs85.EmEglugsPEIE20o37gs9QLn3Zjd_MLcHkgvq 93Jv3z00oMZI9ZAroSGKAGf1xk7LSbIbRn2rqH7bRIDGYZ22W6hqnfwApr9TzWknGwGC5OfyVLNq PhpSW_XMK8_CdvDQyZfGrfOR8xgi_9Lkp1rc_LZh.5aUoPDG5nYAgCoZsUIQNaMIga7L2dx8fRUD gM6LfXem6W8SieiRo4l6Thx1DQiLcy5W.sVQe3BdZOfYhoJMPwa91EkLcyI.PQZY1MgofxUq45Fj Zk_HE X-Sonic-MF: X-Sonic-ID: 14158615-367d-4d7b-b367-50df115f91d5 Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Fri, 8 Sep 2023 14:49:41 +0000 Received: by hermes--production-bf1-865889d799-xc84r (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID e09beb3c2fbe6fee8a89eb4f2216e9d4; Fri, 08 Sep 2023 14:49:38 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> Date: Fri, 8 Sep 2023 07:49:25 -0700 Cc: Alexander Motin , Glen Barber , Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Content-Transfer-Encoding: quoted-printable Message-Id: <62BE6390-3893-4909-B5A0-A98015FF5083@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> To: Martin Matuska X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4RhzXm38qNz3P3M On Sep 8, 2023, at 06:52, Martin Matuska wrote: > I digged a little and was able to reproduce the panic without = poudriere with a shell script. >=20 > You may want to increase "repeats". > The script causes the panic in dmu_buf_hold_array_by_dnode() on my = VirtualBox with the cat command on 9th iteration. >=20 > Here is the script: >=20 > #!/bin/sh > nl=3D' > ' > = sed_script=3Ds/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbb= bbbbbbbbbb/ > for ac_i in 1 2 3 4 5 6 7; do > sed_script=3D"$sed_script$nl$sed_script" > done > echo "$sed_script" 2>/dev/null | sed 99q >conftest.sed >=20 > repeats=3D8 > count=3D0 > echo -n 0123456789 >"conftest.in" > while : > do > cat "conftest.in" "conftest.in" >"conftest.tmp" > mv "conftest.tmp" "conftest.in" > cp "conftest.in" "conftest.nl" > echo '' >> "conftest.nl" > sed -f conftest.sed < "conftest.nl" >"conftest.out" 2>/dev/null || = break > diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break > count=3D$(($count + 1)) > echo "count: $count" > # 10*(2^10) chars as input seems more than enough > test $count -gt $repeats && break > done > rm -f conftest.in conftest.tmp conftest.nl conftest.out . . . (history removed) . . . # uname -apKU FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #0 = main-n265205-03a7c36ddbc0: Thu Sep 7 03:10:34 UTC 2023 = root@releng3.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC = amd64 amd64 1500000 1500000 In my test environment with yesterday's snapshot kernel in use and with vfs.zfs.bclone_enabled=3D1 : # ~/bclone_panic.sh count: 1 count: 2 count: 3 count: 4 count: 5 count: 6 count: 7 count: 8 then panic: no 9. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Sep 8 22:09:20 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rj9Jq11vLz4sRdF; Fri, 8 Sep 2023 22:10:03 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rj9Jp6HrRz3WVl; Fri, 8 Sep 2023 22:10:02 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-5924093a9b2so25118047b3.2; Fri, 08 Sep 2023 15:10:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694211001; x=1694815801; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id:sender :from:to:cc:subject:date:message-id:reply-to; bh=rWBcnRs5Wav0FBJFlqfy4EcvCQStG37/cpDo/kq1GZc=; b=Bs4Csdb0qBE+PRhk91BZf052zY+P9xpBVhB2zAUrcezDaNoqcg0y1oTZnInG8BpmZO 4r8HI4I/dQo1jhTQYJy+bbhuSPMrgCeIrlmGf5rI4iA/XkX4khhM8vydgXuVlhyPDp6t teJESr595Y0KlOLGwLU088vkf6gqIwz592AxEuFWVNW3K8cfdOeyzW6g1dCdCLDR2RP3 6OlwtF2FZY6jOCKYX+e8EUscI33+hYPyNJiNnLpunQUgbW/hFj2vtFBNflyOJsARhBkf a681fxyuX7Lot/SVA4niRCZjbyGUBRZW8IZkL313vnHB7OzQXNJIANJ7pOT0M05C78NN 5HNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694211001; x=1694815801; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rWBcnRs5Wav0FBJFlqfy4EcvCQStG37/cpDo/kq1GZc=; b=ohxKn4WjdOYE0dGZYuRXbcA3OqYc6ikzV1oPVWNHy33qX5v1JormXDteycyUSBITeA X2CTTZkDW6Bpsmo/Ds991rlAWrL8Rto22EpRSuXTUIp0lNXwTVh2W/ZX+Y4BuRrYdGet 7TXHo0h4GL23ryHmLkV8a91Qvor0o0BY2k4HXjx95Mp5M2DrXVXiCJvniUGowvDN9ak5 UpNM1I8uRdVryk0grj4RVglVYqVTixwJy7pfnB96doJg72iJVrv21T8iFlYu8yJiCDLe +VTQXLP0FNG3gzRj15330tJR2GQmo/RoNIsIQg94n+I0PiXD9miCRO5VGeUFhx++3rnG GoIw== X-Gm-Message-State: AOJu0Yy50c1YIau5nDs7BWNoDfm8E9Ut42K/pEFhOB6bys3xJw9kYKSk 9nEHupKEWkF5aLhxnDRsVxaCbTLrjcQNjQ== X-Google-Smtp-Source: AGHT+IGok/kyWyNBPq7I2HyFBtDAgA5PcnpedWXw0CYb9a7rTLn+MWnnHjhT9Q025JNrmYsERzmHoA== X-Received: by 2002:a0d:ea0b:0:b0:57a:3dd8:1038 with SMTP id t11-20020a0dea0b000000b0057a3dd81038mr4089938ywe.12.1694211001386; Fri, 08 Sep 2023 15:10:01 -0700 (PDT) Received: from [10.230.45.5] ([38.32.73.2]) by smtp.gmail.com with ESMTPSA id h6-20020a0df706000000b0058c55d40765sm639106ywf.106.2023.09.08.15.10.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 08 Sep 2023 15:10:00 -0700 (PDT) Message-ID: <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> Date: Fri, 8 Sep 2023 18:09:20 -0400 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Content-Language: en-US To: Martin Matuska , Mark Millard , Glen Barber Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> From: Alexander Motin Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics In-Reply-To: <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Rj9Jp6HrRz3WVl On 08.09.2023 09:52, Martin Matuska wrote: > I digged a little and was able to reproduce the panic without poudriere > with a shell script. > > #!/bin/sh > nl=' > ' > sed_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/ > for ac_i in 1 2 3 4 5 6 7; do >     sed_script="$sed_script$nl$sed_script" > done > echo "$sed_script" 2>/dev/null | sed 99q >conftest.sed > > repeats=8 > count=0 > echo -n 0123456789 >"conftest.in" > while : > do >     cat "conftest.in" "conftest.in" >"conftest.tmp" >     mv "conftest.tmp" "conftest.in" >     cp "conftest.in" "conftest.nl" >     echo '' >> "conftest.nl" >     sed -f conftest.sed < "conftest.nl" >"conftest.out" 2>/dev/null || > break >     diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break >     count=$(($count + 1)) >     echo "count: $count" >     # 10*(2^10) chars as input seems more than enough >     test $count -gt $repeats && break > done > rm -f conftest.in conftest.tmp conftest.nl conftest.out Thank you, Martin. I was able to reproduce the issue with your script and found the cause. I first though the issue is triggered by the `cp`, but it appeared to be triggered by `cat`. It also got copy_file_range() support, but later than `cp`. That is probably why it slipped through testing. This patch fixes it for me: https://github.com/openzfs/zfs/pull/15251 . Mark, could you please try the patch? -- Alexander Motin From nobody Fri Sep 8 22:30:41 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjB7d0jK8z4sqYV; Fri, 8 Sep 2023 22:47:09 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjB7c53Ydz4FHZ; Fri, 8 Sep 2023 22:47:08 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from sslproxy06.your-server.de ([78.46.172.3]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qejzv-000LO3-HV; Sat, 09 Sep 2023 00:30:43 +0200 Received: from [188.167.171.2] (helo=[10.0.9.122]) by sslproxy06.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qejzu-000RYw-De; Sat, 09 Sep 2023 00:30:42 +0200 Message-ID: <63717d32-f340-1320-3335-85135d1b62bc@FreeBSD.org> Date: Sat, 9 Sep 2023 00:30:41 +0200 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics Content-Language: en-US To: Alexander Motin , Mark Millard , Glen Barber Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> From: Martin Matuska In-Reply-To: <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Authenticated-Sender: martin@matuska.de X-Virus-Scanned: Clear (ClamAV 0.103.8/27025/Fri Sep 8 09:37:45 2023) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE] X-Rspamd-Queue-Id: 4RjB7c53Ydz4FHZ Hi Alexander, I can confirm that the patch fixes the panic caused by the provided script on my test systems. Mark, would it be possible to try poudriere on your system with a patched kernel? Thanks mm On 9. 9. 2023 0:09, Alexander Motin wrote: > On 08.09.2023 09:52, Martin Matuska wrote: >> I digged a little and was able to reproduce the panic without >> poudriere with a shell script. >> >> #!/bin/sh >> nl=' >> ' >> sed_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/ >> >> for ac_i in 1 2 3 4 5 6 7; do >>      sed_script="$sed_script$nl$sed_script" >> done >> echo "$sed_script" 2>/dev/null | sed 99q >conftest.sed >> >> repeats=8 >> count=0 >> echo -n 0123456789 >"conftest.in" >> while : >> do >>      cat "conftest.in" "conftest.in" >"conftest.tmp" >>      mv "conftest.tmp" "conftest.in" >>      cp "conftest.in" "conftest.nl" >>      echo '' >> "conftest.nl" >>      sed -f conftest.sed < "conftest.nl" >"conftest.out" 2>/dev/null >> || break >>      diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break >>      count=$(($count + 1)) >>      echo "count: $count" >>      # 10*(2^10) chars as input seems more than enough >>      test $count -gt $repeats && break >> done >> rm -f conftest.in conftest.tmp conftest.nl conftest.out > > Thank you, Martin.  I was able to reproduce the issue with your script > and found the cause. > > I first though the issue is triggered by the `cp`, but it appeared to > be triggered by `cat`.  It also got copy_file_range() support, but > later than `cp`.  That is probably why it slipped through testing.  > This patch fixes it for me: https://github.com/openzfs/zfs/pull/15251 . > > Mark, could you please try the patch? > From nobody Fri Sep 8 23:33:08 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjC8k2MyZz4sZn8; Fri, 8 Sep 2023 23:33:10 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjC8k27nCz4Tdr; Fri, 8 Sep 2023 23:33:10 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694215990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=mIgwZ3JXv4YeEbbTcGzf13s7LQ4jeWyoXJp6rOVMabU=; b=uQEGgsOvd13JCwDkpXZ+6T/9OEcDAz8HfXy4nH+hTfpPpQxLHTlwNmnkLSQxgsAPhIG3SG +PJ0qlL/b3DyeqqNcI9SdCbxvY9yQha59KZp10qImCG9xANceP5xx2AoZZlrohpwLvXhLe DQ1+bA9B0bbBpg1k1F7ucR54alAnR/xz7h8Blcm+vCzzScoFqK3VR0SvOskPJUih3SZr0N x0MdHDJaBnkmFeezjn9ntYFl5MxvimzXkQoPKCUbSdJSMJNVcFX2uRZOwoAUmvEhrR3Oex hbqO7qOfjgn0Xxi+Vov/+GuWe4gBxZRcOD3VWrlXyB2nnaL48QrUhh2BLa+LHg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1694215990; a=rsa-sha256; cv=none; b=BXCVYCJpyrW6qRFJ7WZuOnN841SAHYW/SM6MpDm8JwWg4uL7GrMYmYbc2yyhbk9lt1PRkR TpxqK+D7DUR8ifkawK7wgDVEq2POlDYJeghQBeoa9KMHsHXkxXnVjWk4WdRlY+uc62XJDB fXF1wmImRWC8YFPiBuvkxlCscElUqb2PcLRaK6oNsYiwRCrQcFcxho8LRfM18BUB9XJKBY 7hF56H8Le81OZWAp2GwNaS7zFdE3JeZPKzWaZutTUcE4EFocRL0zKQKffOtIcE93AK9pau XF02GdhSz37XN3x+G2kCDN4iJW3uRxddNRY3JjTIzdwhFN8mRXjRQ5qlNgfkhw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694215990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=mIgwZ3JXv4YeEbbTcGzf13s7LQ4jeWyoXJp6rOVMabU=; b=Q4zfmnRMA6XDkXYaA4Bsc42XB+KZM3LG74UZC06HXi0+y6L7o0j5W6XXqA8csnA7wSTukC tBem3sNIomKNS75jN5tyNjSBM35Jt7rpXMFm0FtzW8UoiAN9wE57ERh6JDR/ZQmPVqhRgj 9MC+kKWTWYzseBlUyHNGUGNmeeq1wl8JwvdvQL0Kuli1RaP0a2isoHbt6M9M5s0k8aDfcQ cQxJ3GhaXa2z81uOfT5x0w1He2Rk36kJ9D/lOhZpZvoL9LwLfuMg6NW44mrUv1cb3GiGTt UOrWLpMfqEAUji+6oVLS5K8D+2kXN2W7yWXn8okyjZVU+EYn9q5m05fbArlouQ== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id E1B1118DF9; Fri, 8 Sep 2023 23:33:09 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Fri, 8 Sep 2023 23:33:08 +0000 From: Glen Barber To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Cc: FreeBSD Release Engineering Team Subject: FreeBSD 14.0-BETA1 Now Available Message-ID: <20230908233308.GS1219@FreeBSD.org> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 The first BETA build of the 14.0-RELEASE release cycle is now available. Installation images are available for: o 14.0-BETA1 amd64 GENERIC o 14.0-BETA1 i386 GENERIC o 14.0-BETA1 powerpc GENERIC o 14.0-BETA1 powerpc64 GENERIC64 o 14.0-BETA1 powerpc64le GENERIC64LE o 14.0-BETA1 powerpcspe MPC85XXSPE o 14.0-BETA1 armv7 GENERICSD o 14.0-BETA1 aarch64 GENERIC o 14.0-BETA1 aarch64 RPI o 14.0-BETA1 aarch64 PINE64 o 14.0-BETA1 aarch64 PINE64-LTS o 14.0-BETA1 aarch64 PINEBOOK o 14.0-BETA1 aarch64 ROCK64 o 14.0-BETA1 aarch64 ROCKPRO64 o 14.0-BETA1 riscv64 GENERIC o 14.0-BETA1 riscv64 GENERICSD Note regarding arm SD card images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root. It is strongly recommended to change the password for both users after gaining access to the system. Installer images and memory stick images are available here: https://download.freebsd.org/releases/ISO-IMAGES/14.0/ The image checksums follow at the end of this e-mail. If you notice problems you can report them through the Bugzilla PR system or on the -stable mailing list. If you would like to use Git to do a source based update of an existing system, use the "releng/14.0" branch. === Virtual Machine Disk Images === VM disk images are available for the amd64, i386, aarch64, and riscv64 architectures. Disk images may be downloaded from the following URL (or any of the FreeBSD download mirrors): https://download.freebsd.org/releases/VM-IMAGES/14.0-BETA1/ BASIC-CI images can be found at: https://download.freebsd.org/releases/CI-IMAGES/14.0-BETA1/ The partition layout is: ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label) ~ 1 GB - freebsd-swap GPT partition type (swapfs GPT label) ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label) The disk images are available in QCOW2, VHD, VMDK, and raw disk image formats. The image download size is approximately 135 MB and 165 MB respectively (amd64/i386), decompressing to a 21 GB sparse image. Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI loader file is needed for qemu-system-aarch64 to be able to boot the virtual machine images. See this page for more information: https://wiki.freebsd.org/arm64/QEMU To boot the VM image, run: % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt \ -bios QEMU_EFI.fd -serial telnet::4444,server -nographic \ -drive if=none,file=VMDISK,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -device virtio-net-device,netdev=net0 \ -netdev user,id=net0 Be sure to replace "VMDISK" with the path to the virtual machine image. === Amazon EC2 AMI Images === FreeBSD/amd64 EC2 AMI IDs can be retrieved from the Systems Manager Parameter Store in each region using the keys: /aws/service/freebsd/amd64/base/ufs/14.0/BETA1 FreeBSD/arm64 EC2 AMIs are not available for BETA1, and the cause is being investigated. === Vagrant Images === FreeBSD/amd64 images are not available for BETA1. The cause is being investigated. === Upgrading === The freebsd-update(8) utility supports binary upgrades of amd64 and i386 systems running earlier FreeBSD releases. Note, aarch64 binary updates are expected to be available starting with BETA2, due to a configuration error. Systems running earlier FreeBSD releases can upgrade as follows: # freebsd-update upgrade -r 14.0-BETA1 During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly. # freebsd-update install The system must be rebooted with the newly installed kernel before continuing. # shutdown -r now After rebooting, freebsd-update needs to be run again to install the new userland components: # freebsd-update install It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example, FreeBSD 12.x. Alternatively, the user can install misc/compat12x and other compatibility libraries, afterwards the system must be rebooted into the new userland: # shutdown -r now Finally, after rebooting, freebsd-update needs to be run again to remove stale files: # freebsd-update install == ISO CHECKSUMS == o 14.0-BETA1 amd64 GENERIC: SHA512 (FreeBSD-14.0-BETA1-amd64-bootonly.iso) = 655b733d5897e741bde7d0a03ddc7893237dc468263303e5898d0590e6b7223354ce5554b9c6d1c9b6dc11d446e1804a7238382e0f5bbf59f8c54b9aed4e6ec0 SHA512 (FreeBSD-14.0-BETA1-amd64-bootonly.iso.xz) = 2ecf86e2f198e0f01a305f572cad987e0589e587cf9d2bebd0740c18de35e7272fe06dcfd2456c641c2924def75e734e91de28bb965b929556a842ad70923a4f SHA512 (FreeBSD-14.0-BETA1-amd64-disc1.iso) = 2f49b058bd39f6491ae720af773dad7861509a5da2a5439e66b8c8ae36baff3be1ac266600de83b7dd073b49cb57114e404cf6339870fc66937d282ade249e05 SHA512 (FreeBSD-14.0-BETA1-amd64-disc1.iso.xz) = 4ceb48c7e5b3a11e9d7953c482ce3f51f397d88e919b80446c1883c1bb904cc2b33b6e058a33ba2e1affc605f1f1ba99b3a8d48235ac06deaca0370ab21aa0e3 SHA512 (FreeBSD-14.0-BETA1-amd64-dvd1.iso) = efef7c826a002a5cd180b8e3aaa4f50f46c752314ffb115078655c73464a7310e23a0a3cc8fab63c7ba103750314c9e042da9ec4710d8550db1f5497e33eff8e SHA512 (FreeBSD-14.0-BETA1-amd64-dvd1.iso.xz) = 03bc53d4643bee6f6c406a564c88d1430551edd3aa30d31089dafc5abde469888c881b018038c8176d169f94593e779eb20e035a5aa6f88b16439bd8317bf9f8 SHA512 (FreeBSD-14.0-BETA1-amd64-memstick.img) = 63f1554816df44a0dbfdbe77e845ca899d6f26bb7f4553ca43e92f2f964970e47f5f07fba7cf0f2d2767c2a518ffa970e4180e896d7d91fe547029a469004957 SHA512 (FreeBSD-14.0-BETA1-amd64-memstick.img.xz) = 858cefe490018be1925e04eb8f1f4ed320840d85caf82f071d8b8d8d9f00fb8ebbefef874662da71a0f4abf7c244c989be1c2f85c41781d53cdea444a7fa276a SHA512 (FreeBSD-14.0-BETA1-amd64-mini-memstick.img) = 2fbfb0aa9e437922fea5a4c00573d189859f88a9a01453b2a462a1a78c8bb02fb11033594ecafa56947dbb30a37616369ff8a15fe9c13a2e8d189091495233e7 SHA512 (FreeBSD-14.0-BETA1-amd64-mini-memstick.img.xz) = 312d59eae8614efdabc86ec7e79f857ae3662ecc0de69d7ea17a929bea6ae246566f2d6a36de000eabe4a4747e4c6abf1bde39904303d71f4e792a8f0bf73570 SHA256 (FreeBSD-14.0-BETA1-amd64-bootonly.iso) = 1aa1b288a3d3ea789af77cbb36c3a7ae7818630f4871df3851b0d8d8f6ed1152 SHA256 (FreeBSD-14.0-BETA1-amd64-bootonly.iso.xz) = b755ed0620c42fd191edf58d8c30ce0dd07354919db61866e707b20c13f98af2 SHA256 (FreeBSD-14.0-BETA1-amd64-disc1.iso) = 3833852cf8823d2fca82ab14cee3841459fa40fef179d030b96200686571bd36 SHA256 (FreeBSD-14.0-BETA1-amd64-disc1.iso.xz) = 16f0d548d371d800cbda29443d6bbefe8cf002e58723da94fdc31f785c3899eb SHA256 (FreeBSD-14.0-BETA1-amd64-dvd1.iso) = e07f48ae2cc0c4a5a60cab73d4a1ca3c00b5c4fa337497dea9ed7095c66c7f3d SHA256 (FreeBSD-14.0-BETA1-amd64-dvd1.iso.xz) = 90abc23bc95bc6e53a6373f1292b49093637d9e15eee2e8b3852d50d162bb178 SHA256 (FreeBSD-14.0-BETA1-amd64-memstick.img) = 40f54a892614430e8bc91ac6f2cda3f324ef26f5fd152d1f8fb547d74c48f497 SHA256 (FreeBSD-14.0-BETA1-amd64-memstick.img.xz) = 52e5ed0e46c729fa6a27734b512065daa4fe7fb1d8b50768d6f857efffe22d96 SHA256 (FreeBSD-14.0-BETA1-amd64-mini-memstick.img) = 71d5f0cea86d5810a7988664c1bf9f32ddb1c422b565c6514d934efe211e91f8 SHA256 (FreeBSD-14.0-BETA1-amd64-mini-memstick.img.xz) = 0b6aa0b44f52782407856371c980cbe3f7446d9eb779e9e9132851f377bf3b9c o 14.0-BETA1 i386 GENERIC: SHA512 (FreeBSD-14.0-BETA1-i386-bootonly.iso) = 04fd64c4ce87ab4c821c60dd049d2f671c519ba2816ef3f2fb76df79751eb86335a1dd5e0d037ec5a72d20ac68fd751638a41392fcb0803510a22d777d0a5624 SHA512 (FreeBSD-14.0-BETA1-i386-bootonly.iso.xz) = ce8c7e4e0e128a86f0bc869d4bbd57736d18ebcb49ad132c523a51bf5f3ad8f5d1258836f3a3f0200397d781ee36afc2493a70332504791c4d3e04384c5bb765 SHA512 (FreeBSD-14.0-BETA1-i386-disc1.iso) = c66d06ea252803f811508638e1ce267dbf9de27b7743a9747d3dd3ab64f7a84159ab27ed65319900eb92f43990ce81fbb7d471fc43b677d913034c72ac321647 SHA512 (FreeBSD-14.0-BETA1-i386-disc1.iso.xz) = 7350934e0f1ebd66cf676b77db4e985644a5acfb5a0c8a547090e78c8da156d05adeb83d6e68aea10bf9b9c92e22fb6f15ad461d9f0d80132db3c2215209250c SHA512 (FreeBSD-14.0-BETA1-i386-dvd1.iso) = 458f6ee3d5abbbcb419bbc0a388003f387d0b98bfd50d7a75c84f1e1af580665f5d16e6ed29b736867f9247fe8a9a5be84cd82edf3c5461f9965d19a2ac5f0a5 SHA512 (FreeBSD-14.0-BETA1-i386-dvd1.iso.xz) = 89de14a82abf6b67f4c89354f6713561da92bb89ca58d19f550bbd33ec11775bbe8ef67647b2316c78fd4e7d96ccd37599c9bc98bbb00a625d92ce146921e47c SHA512 (FreeBSD-14.0-BETA1-i386-memstick.img) = 7e4b922aea158f481b30f59a69e2ec542e2551335d4880848d4fe3d28d8afbb18bfab532530c934ff9371a180d4db7fc66ddcafdc9a0842b8179ae8140e4b5d9 SHA512 (FreeBSD-14.0-BETA1-i386-memstick.img.xz) = 67d4d86f7168514908f3606dce132dc442e4ca4ebd380b16c77268e7140e0433ff5a2204da541272dbed172093f130eef2db39e04c79089d1e02c67d58aef22d SHA512 (FreeBSD-14.0-BETA1-i386-mini-memstick.img) = a1934b51b8a54738d30212294b969911bd13b1391cc563ea8045d7e9b7ae51a9aaa32c7ceb0cab1cb8ba6f7fdd17e3bb579c791f9d8c80b39be773ff3d522815 SHA512 (FreeBSD-14.0-BETA1-i386-mini-memstick.img.xz) = 22270b48861ad88d32f75ae7f012ef6c615bfa451e862ce52a00641dc42c7633321bba4c9fafef60678868e4b7808aa79d2bdde2254b50def0a32cf69c1a028f SHA256 (FreeBSD-14.0-BETA1-i386-bootonly.iso) = a325495deb3e1a7d2a0119e2d9f693e5ab2256790853c6c836827fe70d8e20f8 SHA256 (FreeBSD-14.0-BETA1-i386-bootonly.iso.xz) = f31092bf005ff8765a1ca5ce18324e22c091920e72162da3ad61fb78b01cbb1b SHA256 (FreeBSD-14.0-BETA1-i386-disc1.iso) = b6eb1170716b2e31ac1b86ec465b44c6e67737f592479541749e38879614a23d SHA256 (FreeBSD-14.0-BETA1-i386-disc1.iso.xz) = a21d3a3dac2141d71469b59f159643f782413c4038e58d0a300c01f84d7584b0 SHA256 (FreeBSD-14.0-BETA1-i386-dvd1.iso) = ed9d9bba43d4cc8c832c2cfaf4e36868b7cf263568d8da69f819eb2999e068f0 SHA256 (FreeBSD-14.0-BETA1-i386-dvd1.iso.xz) = 71a9c3e7d877d459fe49deaf26229e0bec25efdb1117dbb8590258ea76bd6e94 SHA256 (FreeBSD-14.0-BETA1-i386-memstick.img) = d2b9136ef3c6f61bd2321a4f8b3cb19930deb9fc05671630e98b4dea19515e82 SHA256 (FreeBSD-14.0-BETA1-i386-memstick.img.xz) = be56668113af3d1126ac18c6a0f4e2c0244bd3fce39c60ab91f50a61eb8fafa2 SHA256 (FreeBSD-14.0-BETA1-i386-mini-memstick.img) = 31922474320f6474c6260b5a3a6ec499326a31bb9c93c8821a738c38172b4133 SHA256 (FreeBSD-14.0-BETA1-i386-mini-memstick.img.xz) = 1ccdb5d43962ac5c145864c0382ab83745afd6bee0912233e231fa3a755398fc o 14.0-BETA1 powerpc GENERIC: SHA512 (FreeBSD-14.0-BETA1-powerpc-bootonly.iso) = 3502711cc2738e153145e0ae5a265ec320da21d50ab214b953fa7c9d73a76e3ee081e23d98a34edc8a4339b6de0c9b2f637d579ad8bc00eaed01d2afe6578232 SHA512 (FreeBSD-14.0-BETA1-powerpc-bootonly.iso.xz) = b2f3ab3b722aa75f0734e3f49f90ee088c035ae8572ef9ab6f2067b220c4beb2ddc931b8847d45e5747b11916615faff29f556b6a8dc43c8d33521ab94471b90 SHA512 (FreeBSD-14.0-BETA1-powerpc-disc1.iso) = 934c615d6e2954d9cb995f45567390e4d78389252b2aae99833e5ddd693d5ff0f3aad44d5c6a9cc9775d5a3a12885ebccd1bcb955c4de6acd0c43e853aabcf77 SHA512 (FreeBSD-14.0-BETA1-powerpc-disc1.iso.xz) = 3ff3e33f5bdc9d8050d3bc42f6200930876c3ff455f4af6e8e4b8454c72b4f7f4a0065ebc1fbcceb1be2c922b07a6088474a55c7d85dee23fda08d1ecf149566 SHA256 (FreeBSD-14.0-BETA1-powerpc-bootonly.iso) = 31ad840e58f401943ae9d6f55231d771c098fb377ce4bc931541fc3aa22b37ae SHA256 (FreeBSD-14.0-BETA1-powerpc-bootonly.iso.xz) = f96d19fc7bdc0f4d50a0214fa483e51baab558c85a333e4e62cc46ea8a9b7591 SHA256 (FreeBSD-14.0-BETA1-powerpc-disc1.iso) = c7d3d1d889f92a169e431a77e213b26afbfbefe8ac2cecdaa9770f17fb0a76b5 SHA256 (FreeBSD-14.0-BETA1-powerpc-disc1.iso.xz) = e951b5015ee15b8f900a0df4baa9569fcf4d763ec095ef40d1a560461dcd7864 o 14.0-BETA1 powerpc64 GENERIC64: SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpc64-bootonly.iso) = e3102d3696afdae14650d6c4993001c57f10e91cb0b0bd57b04e5ac348b308918715756d95c098a4536447baf19248dd79f5871db7d8fdcd4e0778e51f038997 SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpc64-bootonly.iso.xz) = caa2e6dc0c115c4361a30ac7703e8952048e922ce04cac4940e9cf4c0e8e593e317b13aaec4c3eafc59560d2fbdcc2188b5a33366fbcaa3ddc75388a5b461d5f SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpc64-disc1.iso) = 735287f54959a28358d39bde078a3b0642f718b3d53f4b2d4afd66fa23bd29ce7757e5d00c46b8c7c8a5c70cd5cbf092e7a8220c6aafd746e8e38093de13a032 SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpc64-disc1.iso.xz) = 999db24d107d77f852587ee05cd481d0a916a50d094c28801cb488ce6a9986038aab4f757be004740b29f038efc6453696b9abb0abe41d90382d582ed638b493 SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpc64-bootonly.iso) = bc37e4aa9586d5e1c2c2559c320b39c01150f7d664f3453d81e4a9b40c30243b SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpc64-bootonly.iso.xz) = aa3c8acfbc6b4836c45bad46fd6677c56bff53dee8a303dc2e43ae84a5da9a08 SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpc64-disc1.iso) = 3c4b7eb3c6f143c8d6fcd6a5159f97f1dc79690751ca2921c8b7fa0bb6b5a4fe SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpc64-disc1.iso.xz) = 013f41b7efca47f0380f0036d44eae318c6cb1993d52a6756b60721f25508525 o 14.0-BETA1 powerpc64le GENERIC64LE: SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpc64le-bootonly.iso) = 61eab0e88566cf97d9ea0611ebcd99650ef221999e165c13a6aed4cc5d3049c33070d22c1cac8a1b34ae1cf835be8a25228ea730ac85819c78e3298dfb0917ac SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpc64le-bootonly.iso.xz) = 2676fd4db2005563924e9395445ed4f01b9e5a2ad099c1fa6055afcc4623ab4fa37dded0adb02429177155b4624f98b739e2aec96268693a8d43e9ebffaec68c SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpc64le-disc1.iso) = 45e543fbfd6e68a746de636593de39798d5c0a682a151594b37d8cb63d4f03ee1f9beeee09caffd271e82776ba8ee353ff4b92468eaf8358447a7bb466961c49 SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpc64le-disc1.iso.xz) = 6da8e96a7dffd296ed98618f5eb8683cbfa4caaf7e955be06b0acf7dc9e79f0e2cbbeb5fe3ecf28ad70acfca7df0fcaf9c3abb9a76741640b39c7b44997f1309 SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpc64le-bootonly.iso) = 05935b7803b52cbfd3473364285b86aff244188d7b4d5effba80b438270917b1 SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpc64le-bootonly.iso.xz) = 750ac8874cc57601b1349228a1510bdba41c732ef65235c7f66e352712dbb5ce SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpc64le-disc1.iso) = 1d530fc643a1cc4cf383293f6b039b688833b7235c26de6cd9b5509e4bfd4097 SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpc64le-disc1.iso.xz) = 02ed5b81115ce5f1dfd88e8c60a6c73a758abf5eae0caf3eebbeec58f057a37e o 14.0-BETA1 powerpcspe MPC85XXSPE: SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpcspe-bootonly.iso) = f50b60ded12c1124696114a64ad7c8320979b7154815ba42869d4f2504bbb2c17de04cce0fad821c85f60b9709af74d4e46b9106adc6f10de93bfb1d45413917 SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpcspe-bootonly.iso.xz) = 30a81688340e7b0f6538e6117ad9a34562a3ddf7d21f08d1de1f0937896d04b7135757de8c113833f1f9472ecb1a19754b937b9de99434b57046fc3ab829784d SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpcspe-disc1.iso) = 9bdc5ff370ef76b63f822cbf40a2725d430bf05573a62ac4fb9c40b50596d3a87f258e707fb50dffe0c572f8c2e4c61a47d2c3904da6393670d4c9333dcb5642 SHA512 (FreeBSD-14.0-BETA1-powerpc-powerpcspe-disc1.iso.xz) = dc13ecdddcfcd3d43ecb8e5265ebb8283ceec0c655818dfdc6c3e80d4317956d37f96c1d9804fbac1f6c6f76745ed2013cb230204aa3521280abc3084fc7582e SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpcspe-bootonly.iso) = d0b11a84e249a3c348c1b7549dbfb54ea92409a7756eb9990cdf7b4101139c3b SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpcspe-bootonly.iso.xz) = dd3c592f5ba69680f4bb74c83ec9a04df228e5c58740ce32abaa742e88c5569d SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpcspe-disc1.iso) = 52895e8c340247c1a5e70a9c8b2c93b0a875340dfa86d8b3d18a7c409e5bdd7b SHA256 (FreeBSD-14.0-BETA1-powerpc-powerpcspe-disc1.iso.xz) = d8db29545adccced5ac248c96fead88920b86b877e179fa879df07edd122918f o 14.0-BETA1 armv7 GENERICSD: SHA512 (FreeBSD-14.0-BETA1-arm-armv7-GENERICSD.img.xz) = 6d96bd66538a08d176afeaa34d876a37c1b45f0058d600392ffd13dfee3b1513a7a699d4f08632b5d6ebbecb969ae636553a56de1a1b8337686b74d05c6b1ea6 SHA256 (FreeBSD-14.0-BETA1-arm-armv7-GENERICSD.img.xz) = cbed08bc31f47a24ca8bfbeece0811c1a41ec3fab9d66aac4abf206d5f152c93 o 14.0-BETA1 aarch64 GENERIC: SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-bootonly.iso) = 2db57093f2eba676cb4028beb823f9511ba13afd6865a67d24d74160f30e536fe5c5796847453f132e3cbbe67bfa916c2c75cc3b5acf217a07bd9ce9949169d9 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-bootonly.iso.xz) = 572c3c295fdb804c98c3e7d9e656223dea4a81046ac57f906c2ca1002578ef662b7e82cdc6db6b3de5c373efb12b385d21dfa20b20a02f145087ae134f97a44e SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-disc1.iso) = 60a7e38d47541820b53be0f754f4f89ba02a8e341d9bcf2d75aa4d366d0dd98d8609bddb76f35446297da3c9da3536667a416f15c9f9f705b53df1ce4947c2d6 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-disc1.iso.xz) = 20849e69878d071f4d8da766fce40bda65e8b218a716213df8c351a2e6c544cb0364b19ab7735fd07e3fd2accb57b8a6fcf561cbcf7e8c19f3da98766fedecb3 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-memstick.img) = 17389f625de79d98d06dab7ebd01ee394a5123c9a10ef501254a5b5391d54552848179477ea269c264c46402defe8575ef98da3c05acd16cbb77421fee7b68a7 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-memstick.img.xz) = a1a430b75e2d80823b8a0d7ef00a6a92f50ec4b8d568d7e0f40eb359c3a4479b633f99b17e5a4016cd437a2df1359bfec0bab8db833e924388589c9ade58a56f SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-mini-memstick.img) = 7d4842772383b55e46e5ce5c217bf5526ef9e514d0f52564747582403392716169c36f22b60049e51deea26f5af8eb8f68267192288f5f3e194e90766cc57f7c SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-mini-memstick.img.xz) = 7594b98776b5b9760c2e45fb29bc5581761a1249171a6062e2f640b07f130c2300c7a42af71c6564dae513e80d6b4ba62dc786493d6a1296c1e57d61e3d50b7b SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-bootonly.iso) = a573d36991e781809695c97946b6645d2f8150419afe774eee83bc012321ab2f SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-bootonly.iso.xz) = 44da455aeeb7b4231d07843a49c99f393a49d27ab35f125a897e788dee538b52 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-disc1.iso) = 462f266b972629938ef0a23b905b42135fb7554a128a9f78c7cd34fe54246aa0 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-disc1.iso.xz) = 553155bb0b7bc02be09a7f82820f6b65f1b531f6d39cfb3f2de3ed3d767cd19d SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-memstick.img) = 5974fe276f31d14c68c841c12558a26b200c9c41aa92d5f30afcce0120c1943a SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-memstick.img.xz) = 475b028431e576c1c76e309056363f14bb48cb4be0f72194f033906ea80fec48 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-mini-memstick.img) = 53d3be2f526590fe1869e483a328c2ed423a89d45819d5a3e2b5e70809c3c762 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-mini-memstick.img.xz) = a725482606079dcfcaa2310540505ca77a5e3c2e38c0b3f251e40c45381a080c o 14.0-BETA1 aarch64 RPI: SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-RPI.img.xz) = c29f37ac3852ff0320bca1b05bbfbfa68aef81fb8267d4d4bc0bc418043a20819aa768abcbfa373184ee1c4f132d2ae8bdf99b107878afba228d81b730f71572 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-RPI.img.xz) = 820041ba13c813ce9240d7cadabd4b030907d0cfe19cbef1bbbd058608f54365 o 14.0-BETA1 aarch64 PINE64: SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-PINE64.img.xz) = 92ed0bee06815aed119874164a0891c1b93a440e31066a881cd438a4bb2974982bdaf9c4e2c2fdbf274407210d6ebea6768d49e5a595ced31b73fc1be559719d SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-PINE64.img.xz) = 74a653492f04c189c4ab0c07fe3cebd3b27b17b54e3038a97c76b67267a20de1 o 14.0-BETA1 aarch64 PINE64-LTS: SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-PINE64-LTS.img.xz) = 66adcd68b06207acd80d75118c1a49ce79bde3f6d7ad11548f68a0afcca1d99d6c858c3fb10a83bac490aebdb523e9fa86084cc857897ab641873c4f10a6b553 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-PINE64-LTS.img.xz) = 89d32d693687c016e904ecc9289c9f6ce078dc2b69ebc11c0b7c10fbe1149893 o 14.0-BETA1 aarch64 PINEBOOK: SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-PINEBOOK.img.xz) = 13739881805b7402d825df8c899195ad8b62c38df42849e63b8635e21c3c53483de97bd6b548adf79f1d88c1b345fb646825ac760649406f13e6a8d9a7edabc7 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-PINEBOOK.img.xz) = cb434360482298a6a4a613775d93614d4296c78bc9008ab95271b9e62a7dae8b o 14.0-BETA1 aarch64 ROCK64: SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-ROCK64.img.xz) = 05aeb5d9d00a1e537d619ca92173c34c8af6e89210499de34f2ca0337127b95c205c6eb3449fc06a61e3916cf533686265e2b0b1103daf5acd6de4604ab30ffd SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-ROCK64.img.xz) = 7b826c1721eb186e4510eb01379df1c59c014edd15302f048195cdef57503127 o 14.0-BETA1 aarch64 ROCKPRO64: SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-ROCKPRO64.img.xz) = 7f7c6c6f3561d16ce67a8b38850acd15ae51a2e0f70d70cb7217da0488107bc1d1e359420ee4cd8499bd7c2d6c68e9234566e270ddd6674404133b25aba3d7b8 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-ROCKPRO64.img.xz) = 012f67a7f05d458908126bdcff697ea36d0bbe7f933bc51770be795f60652de6 o 14.0-BETA1 riscv64 GENERIC: SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-bootonly.iso) = e38c3fd40a92b72b53e2a28367fb4f1c21399cf87ec76865a1fc20326d9fca3e4115088956cb872534352fc75a2004b419069f6fef22259757eec762e7c081ad SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-bootonly.iso.xz) = 72c72038a78806ed9ce91ff3409fc42264846f66ddcf7dd377bb7f4a3aa13fc0a7ae27cf7bb1f88440cb64a1a0dda5ec0696a46b3f29f62f0df2b75eaeed21e4 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-disc1.iso) = 2d28e5478bf1bd0bbfd17d664bc62db025df4952e6a1900f2bf06716ab189dee32ac2fb95755d17d3115047cdc3508ea87225bc8d4661e7f2811666c0b660a78 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-disc1.iso.xz) = eca5d47a08cf3714770df5d8ea1f9d51f5cdbad717f5d416866f64b499b5d03256b491dbe4ea0166f69070ea75bb786d14809f55bda73347c7c0bf3908ee3e19 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-memstick.img) = 11c9eabbca2b2b7ab0fa3462aa26181724cd9e3079c094660c8e1861b9829a4e11d6d3c681ef0a6200aaa8045179b05a25aaace99440538302b7acc616809d58 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-memstick.img.xz) = a93aa4a8f563d18b2a75ac67ff825413969badc04ea7680e7ead47ff2695acc6cd02929e7b24e9c8df03e0c92a98aba5d7683c70ad571c171fe3dbb589453b2f SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-mini-memstick.img) = e162df26f8ecc528ea872dc4a03047a0f85d21c47d7dc25661dd72f8e7941020c4c1d8f009fc74a94277e4788f3aefdd55d3a451aa5f87cd9b5ffea8f29dda42 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-mini-memstick.img.xz) = 66619e79600a1cb1ee4df110bf50b9b8d6150064b60195e9ddef1f8fa18fd70a3cef678830466ad532b6879437170687168baa1844e80d21ba8d94cbbb5c0672 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-bootonly.iso) = f492fc9d9a26f23442abefdf9f7f232cc6e8bd856e3572e6e4378cbefeef3833 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-bootonly.iso.xz) = d891c8044f6d7197901881f85672a17699702d347a9965153b910fbcd2c1b62d SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-disc1.iso) = 9fd0fa4d59d33aa42bf153bb029da620fd37d87fa413bb2615e1d52a5aba6ddc SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-disc1.iso.xz) = 6ec312a7d08d01ee4a1ee53bc377bc7c42acb2ab29c68104de9b2e267998725c SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-memstick.img) = 7e15ad1fbd3a5460e9175b67f4d019bdab6905735d9f585ab91660f5b3dfe4f9 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-memstick.img.xz) = 7117043efdb2c5abb1298412c064f646930c1c9e9d182c966ec051437a22c985 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-mini-memstick.img) = 0dc180ea2e26f667a6687a4061ed4e2c019935e95169ba1b08d205ec88afe417 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-mini-memstick.img.xz) = 291e268b7ae6d257de9148d4bfe6ded3c69f0bb5e6eea4b57f623c8088c2f5ce o 14.0-BETA1 riscv64 GENERICSD: SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-GENERICSD.img.xz) = 5de5af587e2a3e7702d8aa3800d8fbc6be1fc2484c710f6d6417d4e70b2279d209bbbb1a57ffd2a63efefb1155fda759c9123439bebcf484c868090ac69ae213 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-GENERICSD.img.xz) = 38d886814a1e0071f50b4685a2a92f8519aa2640da88a951b201140e51d7b571 == VM IMAGE CHECKSUMS == o 14.0-BETA1 amd64: SHA512 (FreeBSD-14.0-BETA1-amd64-ufs.qcow2.xz) = 88768e7cd7dcbfb158eeed5a0bbe06e51ee3c22095d14d355d0a6ef81e6ffd9d59d93aaa92b42a0affed0224c2b4119202fce7b92c52f6c9ad9413d1e57909f3 SHA512 (FreeBSD-14.0-BETA1-amd64-ufs.raw.xz) = e557273cadd6e4a0236168c24679309232e6e08eea32d63f1f3a06e59ecd0029e7d3a59760eca4dc316f604e9058d9d5f7a11959cb86ede821a091b5ed45b0ae SHA512 (FreeBSD-14.0-BETA1-amd64-ufs.vhd.xz) = 1c1b469ce6c9fe29c082a3261f8a5b9683bd679a01a3f5d8e8be8f65d2ac3243b673e0364f5377901c57fa7312220bf10da2eedb5f29f144bacce144dfec4517 SHA512 (FreeBSD-14.0-BETA1-amd64-ufs.vmdk.xz) = e8d4b57a4f6d7ed36b1dd9ac4542dce9e19c85b498089e308fe5d8a269228a1930141293d81dbd61edb2c36e3638c9eb4644f43169a02d4f8d0eb541484df90c SHA512 (FreeBSD-14.0-BETA1-amd64-zfs.qcow2.xz) = 6250f1b95b6c8bddd05ae9c7338014d4f427ba10a5318609a1c09bbdac328a226c8620d5a1a64bd5d366e45d91fccc266a9693a243e21aa180b75dc8553db9d3 SHA512 (FreeBSD-14.0-BETA1-amd64-zfs.raw.xz) = fb83cf9e958ffb100451f15757445c9789c3f76ef71db83e066db05bd3894f10715c607c164a47c2ccc58eb1332876978e7f19deca5b5e5a664f4d7647436227 SHA512 (FreeBSD-14.0-BETA1-amd64-zfs.vhd.xz) = 8ccde4e0f81a0ea713e96fe394c5df0c5d1db7e15f5beaf9c755b51f30d16f65f83edbab3c8a16cde59f07282529a112eb268c0b80154c0562a958f23220c08f SHA512 (FreeBSD-14.0-BETA1-amd64-zfs.vmdk.xz) = 81d84da3112999f6747f0d7dd68302f9779b3ccb4ec81835cb36569b64e201be5204ae565d9a7325781c14a9c2d55fc36329d7754116e89f4227b59e55821a74 SHA512 (FreeBSD-14.0-BETA1-amd64.qcow2.xz) = 88768e7cd7dcbfb158eeed5a0bbe06e51ee3c22095d14d355d0a6ef81e6ffd9d59d93aaa92b42a0affed0224c2b4119202fce7b92c52f6c9ad9413d1e57909f3 SHA512 (FreeBSD-14.0-BETA1-amd64.raw.xz) = e557273cadd6e4a0236168c24679309232e6e08eea32d63f1f3a06e59ecd0029e7d3a59760eca4dc316f604e9058d9d5f7a11959cb86ede821a091b5ed45b0ae SHA512 (FreeBSD-14.0-BETA1-amd64.vhd.xz) = 1c1b469ce6c9fe29c082a3261f8a5b9683bd679a01a3f5d8e8be8f65d2ac3243b673e0364f5377901c57fa7312220bf10da2eedb5f29f144bacce144dfec4517 SHA512 (FreeBSD-14.0-BETA1-amd64.vmdk.xz) = e8d4b57a4f6d7ed36b1dd9ac4542dce9e19c85b498089e308fe5d8a269228a1930141293d81dbd61edb2c36e3638c9eb4644f43169a02d4f8d0eb541484df90c SHA256 (FreeBSD-14.0-BETA1-amd64-ufs.qcow2.xz) = 9806c8b312e8924b9b410f8b8364aa27fe855efaec196477b5523b2108e67c43 SHA256 (FreeBSD-14.0-BETA1-amd64-ufs.raw.xz) = a5a75f11020a6faed8e72b148176c7629a391f560348d465f4e5a5bbe9fb844a SHA256 (FreeBSD-14.0-BETA1-amd64-ufs.vhd.xz) = 42abf5406b835996f0dc2dddb777df4360b78ec7ebcd7e63df0c4a02663bbe4a SHA256 (FreeBSD-14.0-BETA1-amd64-ufs.vmdk.xz) = e13c5caf25b4516bb95de414ed93f8e36bb851c217867ed89e708449d42ede2b SHA256 (FreeBSD-14.0-BETA1-amd64-zfs.qcow2.xz) = 615b06d1aaa88947b7a811eee9e27032041694dfebbf19cfa862554d932c5fb8 SHA256 (FreeBSD-14.0-BETA1-amd64-zfs.raw.xz) = bb92d4f7dfdb2008a9420399e5b7504f5db7c3d33b2fe2921f615f700c73913e SHA256 (FreeBSD-14.0-BETA1-amd64-zfs.vhd.xz) = 82c7f85cdfa72e45b6a559708ce4b38735f73cd777a29e3b137629046c6642db SHA256 (FreeBSD-14.0-BETA1-amd64-zfs.vmdk.xz) = 9774d8285019809fec7b3bdbec1d58093f422146cce70a3b4e26a49277d716cf SHA256 (FreeBSD-14.0-BETA1-amd64.qcow2.xz) = 9806c8b312e8924b9b410f8b8364aa27fe855efaec196477b5523b2108e67c43 SHA256 (FreeBSD-14.0-BETA1-amd64.raw.xz) = a5a75f11020a6faed8e72b148176c7629a391f560348d465f4e5a5bbe9fb844a SHA256 (FreeBSD-14.0-BETA1-amd64.vhd.xz) = 42abf5406b835996f0dc2dddb777df4360b78ec7ebcd7e63df0c4a02663bbe4a SHA256 (FreeBSD-14.0-BETA1-amd64.vmdk.xz) = e13c5caf25b4516bb95de414ed93f8e36bb851c217867ed89e708449d42ede2b o 14.0-BETA1 i386: SHA512 (FreeBSD-14.0-BETA1-i386-ufs.qcow2.xz) = 398ed465ed36e3b08a2b84c05686a22ebaf2bcdedd150cee4ca116ed232844814bf3a54d012350cd50103f75c15ac63c6e1dbd497c8b49719089e5f7e26331aa SHA512 (FreeBSD-14.0-BETA1-i386-ufs.raw.xz) = 33aaebe58651d33907dc80e877be744340048a7d909862bd2ca2ed1a3ae5b3e318a505f9067abdd0da0c18ffb88e187bb815484d85cd412433ea88486690cfd1 SHA512 (FreeBSD-14.0-BETA1-i386-ufs.vhd.xz) = 8316cd09e32a2d0168a6597453b7e030ace221876846d3b62a3bf04e451735da734780e9f57cbcf222dd14912409cee83dd71b2bf0fa05718989cfa159892937 SHA512 (FreeBSD-14.0-BETA1-i386-ufs.vmdk.xz) = 29a51d2dd9e3c02bdec68ed5386342bc094c8cbeda57515d0d61f40b17afa227aba484efd8fb8137c9853020cb3c9954822a1c185be8c0e0af8693ba51db702e SHA512 (FreeBSD-14.0-BETA1-i386-zfs.qcow2.xz) = d3217800085df6ac550226f8cb4ebc1bef0d138fbfece3b35573d3b684e4cf5b134e8efa0cbfbd981c36c21639dc622b9d74cba28c975f6c470f62d8214927ca SHA512 (FreeBSD-14.0-BETA1-i386-zfs.raw.xz) = e043234ac5b2a68844e5254a845f7e67f3080bab42d0213dd60b8ac3a6190961cfc88a4291b59d031bdc5a1a867df866e92965af6a86b845a273687e7553e0c7 SHA512 (FreeBSD-14.0-BETA1-i386-zfs.vhd.xz) = 1a761c97f23f2e8748dc821c2ab22d38889b4318f4f505153f01318e92644b25dd71ece546dfa2240bf6063356b8c776cac659af304dc5d64022b0106d1dcf86 SHA512 (FreeBSD-14.0-BETA1-i386-zfs.vmdk.xz) = 49b67ecda81b0353f113a5af15b7a1dee69158e91ab87e6230635d384941ef07c7a2713cf4465eafd8db3b346a96b03af6c9b508141ffcf7b1f5df72688e429d SHA512 (FreeBSD-14.0-BETA1-i386.qcow2.xz) = 398ed465ed36e3b08a2b84c05686a22ebaf2bcdedd150cee4ca116ed232844814bf3a54d012350cd50103f75c15ac63c6e1dbd497c8b49719089e5f7e26331aa SHA512 (FreeBSD-14.0-BETA1-i386.raw.xz) = 33aaebe58651d33907dc80e877be744340048a7d909862bd2ca2ed1a3ae5b3e318a505f9067abdd0da0c18ffb88e187bb815484d85cd412433ea88486690cfd1 SHA512 (FreeBSD-14.0-BETA1-i386.vhd.xz) = 8316cd09e32a2d0168a6597453b7e030ace221876846d3b62a3bf04e451735da734780e9f57cbcf222dd14912409cee83dd71b2bf0fa05718989cfa159892937 SHA512 (FreeBSD-14.0-BETA1-i386.vmdk.xz) = 29a51d2dd9e3c02bdec68ed5386342bc094c8cbeda57515d0d61f40b17afa227aba484efd8fb8137c9853020cb3c9954822a1c185be8c0e0af8693ba51db702e SHA256 (FreeBSD-14.0-BETA1-i386-ufs.qcow2.xz) = dbfc09bcdea24d8d2ef33896586db1e17a307fca0378995073bd84c78d40b403 SHA256 (FreeBSD-14.0-BETA1-i386-ufs.raw.xz) = 7115a9b00e1057db11c4c328cf8b26fea0c8f24c95d34090b5110cfffbc296df SHA256 (FreeBSD-14.0-BETA1-i386-ufs.vhd.xz) = 1718ea51019f8356af9096019822530661732de2e7ff561e00bf2fff6f8de561 SHA256 (FreeBSD-14.0-BETA1-i386-ufs.vmdk.xz) = ce45a6099f2a617ec6250c0856aad9d695ae81adb120b731a68231662147f837 SHA256 (FreeBSD-14.0-BETA1-i386-zfs.qcow2.xz) = cc778f4268de4b5bbe115f217b5d02c293fd5f588a1c6db4e991d3e468a845f0 SHA256 (FreeBSD-14.0-BETA1-i386-zfs.raw.xz) = 8bf661596986525569ba00993b2ee8498fb82955896686bdd7d082677a46a48d SHA256 (FreeBSD-14.0-BETA1-i386-zfs.vhd.xz) = 04ece57110af7301fb9c5bc68fcdfb42da428ce09c3dbff20530c0c516116ecc SHA256 (FreeBSD-14.0-BETA1-i386-zfs.vmdk.xz) = ccecaa6381acc6960afe23a8a4476b03109032f0315e9bc2f31bf114d4539db3 SHA256 (FreeBSD-14.0-BETA1-i386.qcow2.xz) = dbfc09bcdea24d8d2ef33896586db1e17a307fca0378995073bd84c78d40b403 SHA256 (FreeBSD-14.0-BETA1-i386.raw.xz) = 7115a9b00e1057db11c4c328cf8b26fea0c8f24c95d34090b5110cfffbc296df SHA256 (FreeBSD-14.0-BETA1-i386.vhd.xz) = 1718ea51019f8356af9096019822530661732de2e7ff561e00bf2fff6f8de561 SHA256 (FreeBSD-14.0-BETA1-i386.vmdk.xz) = ce45a6099f2a617ec6250c0856aad9d695ae81adb120b731a68231662147f837 o 14.0-BETA1 aarch64: SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-ufs.qcow2.xz) = a004a801504f01e8f228443f11decb0fb4a50a758e13f20e007094311780fa320635c0f1c14e92b0513f24a721f46daa90a26769139de4b3ca31f63326a7d368 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-ufs.raw.xz) = f76fd36bc4a95ccdd1b6745c0b75036605affb525c8e985bbc38232ffbca8cad3e7d52e3807e77b9d8f101ac948d47f071b6f6a125f4905c728256fd34d9eb27 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-ufs.vhd.xz) = dc6b9fd69079587f93c068c2516a2fa3ba2159017754292489b4a7c057c7399b2a36cb231dc6a1dca5f2f0ea49f8c3d1b0b8216ffb8118254637839195724917 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-ufs.vmdk.xz) = 87deac1af5ef3ebff01f18349f19fad2eb65c46f7f66a96dcbad77634245f85ad4065a6fb4920cd64a9139d4d9f84769157cb6c9568eabb72c4898e8e54e6463 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-zfs.qcow2.xz) = 2570d176320c4f1021ad27bc00260fc42299a2dba53c631ab976962b8c660c7ffc1dae469ed0bc465c1395461dcb5099ba6c94d0c5e2c087c5ac9eb793ba1b85 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-zfs.raw.xz) = e26f9c15bffb280e8d96c24297490dba673e2d41aa62ef35442958af2e2fa850b5dae9b06aebd0c32f71f52acb63bab598b0871690985abf911d50a5e3bfe9e1 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-zfs.vhd.xz) = d005b1b0cdd3a8fb8474fa5517c8c75e50d2e254af8fa42700b96e024489c1e9fee68c2cdda2b4f6fca102134bd47a7e4374e02b65fc55f6a13b800186371d4d SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64-zfs.vmdk.xz) = eece73d646984e4898d84e75a4b1442c31d5d157f6c393efa8ced85c73f6b9bc94923d1c327a86495f0b7fc7f388af4b8fbbb8c2bc5ec38de3c5d1bea549b5fa SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64.qcow2.xz) = a004a801504f01e8f228443f11decb0fb4a50a758e13f20e007094311780fa320635c0f1c14e92b0513f24a721f46daa90a26769139de4b3ca31f63326a7d368 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64.raw.xz) = f76fd36bc4a95ccdd1b6745c0b75036605affb525c8e985bbc38232ffbca8cad3e7d52e3807e77b9d8f101ac948d47f071b6f6a125f4905c728256fd34d9eb27 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64.vhd.xz) = dc6b9fd69079587f93c068c2516a2fa3ba2159017754292489b4a7c057c7399b2a36cb231dc6a1dca5f2f0ea49f8c3d1b0b8216ffb8118254637839195724917 SHA512 (FreeBSD-14.0-BETA1-arm64-aarch64.vmdk.xz) = 87deac1af5ef3ebff01f18349f19fad2eb65c46f7f66a96dcbad77634245f85ad4065a6fb4920cd64a9139d4d9f84769157cb6c9568eabb72c4898e8e54e6463 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-ufs.qcow2.xz) = 1971abc90261d0a356ce6abbe15350845986f60995ce923fc2c5bcd52f205df5 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-ufs.raw.xz) = 16137190d8c7195eab4ea5abf41675f87e8f3c74317cc9ab7cae35b02b668b0c SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-ufs.vhd.xz) = 87a648c8ead3b41d8ea085caa7c93eb6864f63fa9a47ff81483477cee5135cb5 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-ufs.vmdk.xz) = 02edb922169a770fec77960627737ec3b4d4a5d4dede8c0b68bb2af12d515453 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-zfs.qcow2.xz) = 6ac7ba709e07324aab6ac1938f5c21f949de281b955fc3c25e33014188726fd9 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-zfs.raw.xz) = 7c9f5f4f495126648fc76d1c8841bdb733c1002e69f875856a3c4c1118999a7b SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-zfs.vhd.xz) = 3ec7d38f1054e47a922d784a9aca0ecfa5e31337592a339c927465d0cc5e7385 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64-zfs.vmdk.xz) = b64d6734fc5462686ffbe837e0f062869ffdac91e51672d2742ba4286cb5ac8d SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64.qcow2.xz) = 1971abc90261d0a356ce6abbe15350845986f60995ce923fc2c5bcd52f205df5 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64.raw.xz) = 16137190d8c7195eab4ea5abf41675f87e8f3c74317cc9ab7cae35b02b668b0c SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64.vhd.xz) = 87a648c8ead3b41d8ea085caa7c93eb6864f63fa9a47ff81483477cee5135cb5 SHA256 (FreeBSD-14.0-BETA1-arm64-aarch64.vmdk.xz) = 02edb922169a770fec77960627737ec3b4d4a5d4dede8c0b68bb2af12d515453 o 14.0-BETA1 riscv64: SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-ufs.qcow2.xz) = 16f6ec04b5a60b65e0f4762730be806362b94d76427e6474356c866b1aa313ddf3d8b2eb9acd80c70a9e644349e80af94e63014560b9ca6ef59363fb2cf9d775 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-ufs.raw.xz) = 3c20f49cf239c90f50adb1a821cea5f6b674e2e658784642f1701a0d7334b5f44efe0228d4fa450216d79ce354478b38da20b9bebca8ec42532d1f7a8627e824 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-ufs.vhd.xz) = 0fbb1d46fc7cc52336f3d9be2ac9739b0851c258aa1bd473f5718f55b635acf592faf87c7b4d402d28e5fcfc6257afcc77f1554ca9c2a7da4913e210a543650f SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-ufs.vmdk.xz) = 3bdbc17b3d94e33324a4613629281d8fa3e9890edae26921c078d6c314bb463bc16dfbdbc75c3be0b5297ea406338db4e188cc0cf362761d4a098b0b7c8c45de SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-zfs.qcow2.xz) = b8797bac6878775a397e3efeaa22a4fa087931d44242de35cf2333b295926bdbc8396d732004822b11801321fdce19a14ad9f598bc46f00e396576d67fd94f2e SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-zfs.raw.xz) = d334777ec1f290a7b9ec48b7439b7a12bc2a2416959eff6886b05486ad9e5a1b789432a546409ef82b6fd1b3fef1d07a0553f8ac7225aca900d98aec7ca299af SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-zfs.vhd.xz) = 5f2acf783a41dd51f28fa48d25f460070249352728191f61f5ffca0644e55c1c7b15db4680c50ea3d8525c32eb4628656266b73fb2742ac72f10780e257cd09d SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64-zfs.vmdk.xz) = 0820b06227aab1f3528412d57723858b57d5d5144f622dc69040955d3d72c5c6fc6bddb394ad96525b84220071926d910ae01b3f04d87d24cbd20feb79543f82 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64.qcow2.xz) = 16f6ec04b5a60b65e0f4762730be806362b94d76427e6474356c866b1aa313ddf3d8b2eb9acd80c70a9e644349e80af94e63014560b9ca6ef59363fb2cf9d775 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64.raw.xz) = 3c20f49cf239c90f50adb1a821cea5f6b674e2e658784642f1701a0d7334b5f44efe0228d4fa450216d79ce354478b38da20b9bebca8ec42532d1f7a8627e824 SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64.vhd.xz) = 0fbb1d46fc7cc52336f3d9be2ac9739b0851c258aa1bd473f5718f55b635acf592faf87c7b4d402d28e5fcfc6257afcc77f1554ca9c2a7da4913e210a543650f SHA512 (FreeBSD-14.0-BETA1-riscv-riscv64.vmdk.xz) = 3bdbc17b3d94e33324a4613629281d8fa3e9890edae26921c078d6c314bb463bc16dfbdbc75c3be0b5297ea406338db4e188cc0cf362761d4a098b0b7c8c45de SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-ufs.qcow2.xz) = 76e835eca7f7d11a218ccceba865df3b8b3f8c9b9990e84161cc10223eb7aa74 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-ufs.raw.xz) = d62d0e74ea0799fb0f4a129bf63e3baa206945ee8f50181238ee203e05ebf66b SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-ufs.vhd.xz) = 657fca6466ffaccbbef099e9be0937ce9cb9fa3aee7e9ed96838f8a566b31254 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-ufs.vmdk.xz) = e9059d8b62cf646a9797d7e3480127f60085b7f4e1e7044bd5485f64b6f49250 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-zfs.qcow2.xz) = 50c85066d87cc5d763ab647729c846518354f7f9c8bb3b5ffff7f1d92633718a SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-zfs.raw.xz) = 0e68f2510f29806a2d04888dabd7a54ff165ae347aed413932dfbfc2c8aa2e14 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-zfs.vhd.xz) = e885095fa91ff0b814d4c16d3e7be89c4ad590b9d4a0f064b11e4ab7b963e7d1 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64-zfs.vmdk.xz) = 3390d44a9db209678e11e5771e12d882fe3eb2635235e91989cda71c4913f262 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64.qcow2.xz) = 76e835eca7f7d11a218ccceba865df3b8b3f8c9b9990e84161cc10223eb7aa74 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64.raw.xz) = d62d0e74ea0799fb0f4a129bf63e3baa206945ee8f50181238ee203e05ebf66b SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64.vhd.xz) = 657fca6466ffaccbbef099e9be0937ce9cb9fa3aee7e9ed96838f8a566b31254 SHA256 (FreeBSD-14.0-BETA1-riscv-riscv64.vmdk.xz) = e9059d8b62cf646a9797d7e3480127f60085b7f4e1e7044bd5485f64b6f49250 o 14.0-BETA1 amd64 BASIC-CI: SHA512 (FreeBSD-CHECKSUM.SHA256-amd64-BASIC-CI.raw.xz) = e0a5d92b960d1057eaeb029b1483a39cef853a79af3fbcd0081d78904b168bf5bd42619aab4b34e03e2e7f8b0a42568afb28b5d3197d13d1bab5cbf3e0ca52d7 SHA256 (FreeBSD-CHECKSUM.SHA256-amd64-BASIC-CI.raw.xz) = de92cc7e3fcb44e3370539adcf6ca6f5abe85371100b0a5d18b32694232a60fc Regards, Glen Please consider donating to help support my FreeBSD work: https://www.gofundme.com/f/gjbbsd https://paypal.me/gjbbsd Love FreeBSD? Support this and future releases with a donation to the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmT7rzMACgkQAxRYpUeP 4pPB8g/8DLSW5lcJBJLpmxBLrEG0f7gIkSNNT+nR4RdWYdiZaGqkkY6DXcWp33Lt Y4ILsBY4UPmkkC8DWVsdk8scIIpXdc5i6IyYbycZ6hdRBortYT7KcvE6I+u8641/ X7/akgRxLzuMtUMsG+y4+G3bH8uwlhquY0EA6czhM1G+b3jZI0WN59tRiClkh0yt M4/M7MgsTKSePBkYN9zK9eydkPwb6W1V7yGA4JTBkqtfcR2SPilLCq7WDGzZT0D1 Cd0HvXMAJ+zbZgRmNwt+cG09n2Nh99EQhdJqYNbxUEi34c7JcrQIZ86MqyRVHUFL WFHGVN87LP1R0vRIYvozCH0cEBghOxLWi5pStBGl3K5Des8lDBXnVJyxiwJuoply 6AZztkUXKKx7VAX97244ffEA0Y6oBhq72GJsuL8m2L9Lw4cdroANhyA2oEgT2Bu/ X4944o/KDVLwFsLlmmytziN6rMAFXzNZnoCF99wu5+BGrqjPYGS6+f7hvYGlPj5k RVQdC/78zEEeYUNc6K9Wxa1XrA/4AULkHv5UTLIzqHauPo+0RzvjgtLpT+QkmdGR vqoVR0lHyWFLpLYzPKz55uNQ2OCH1ffknzTzBjgKnECLGLJ0SsZqklYU/SH5W8FP 7LC+vi2kdJ6pjgKi6+ks2wg8tGi0MbL3Hz1I+/viYjCc7BOsgvU= =qncn -----END PGP SIGNATURE----- From nobody Sat Sep 9 00:03:07 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjCqh1LH4z4stHQ for ; Sat, 9 Sep 2023 00:03:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-55.consmr.mail.gq1.yahoo.com (sonic316-55.consmr.mail.gq1.yahoo.com [98.137.69.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjCqg5tLxz3CkJ for ; Sat, 9 Sep 2023 00:03:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694217805; bh=gk0+mRoepxUPcivNE0z95JLP4LsmUN7aHapQf5FZhJ4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=cQD57SOw6dAMEG5Bdp40dE5a5gquRKgmKYbMm8uhrdPKi/t87cM8ruAT5DgDHmqhRmZ4acElwWWYpt3gFNUFPC1QKqRxAQ7yqkxzgdaIR134val9ose8/4qls3ZzDtIl+8CyXF24xGUl2aJe1uWq9InQ/XhL1mQZaI9dbJGaZWb4LKU5vZXDpr+8YzAf9ACalvyI7WzPQQdaHgAMp7DIVMO8k5O9ISPH3MGQ9RHOU9LBeBs539NUFwKx5HK9YmIC3W8NsJR55RI6IK0meN//5LmG09iWMOHXWr+hCNSDCNddCh9qmnYeRPTOqVP3axF/dd7xfCj9pLKCDLbBHRN4lA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694217805; bh=dgitIsqkIIOfeXEpp1DL4p7WqHYy90NlIovfOwV8M3k=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=tJRJut6tnO0lROH4AOaQsh345jsvIZ799ra8SaiXFzPoYh3c3pWUFGs6D7ivkgz2YjpzkZfr1XxbCZZ+OD/0Fpuv3dUO8CqHYsH1bNfQr2+iZHtAdYsWN1iBmRwNU/kgXoyLJHSejAVy7Qi1TBSThN4ra+rdYgoZhFOKyP0TYeGDlDwVrzd4CwirIoGEukm/fPs3FJUo4WyA4YMg2dfr2JjQe2It6COjvrv5p7I8hU+d2KyjtBdeuzSaWvFI5vP0d7tutqQTW4g3xJmJetRAcTdG13Pl2I3Ih2qSkaG0Bp1LUEH2w4kpRzivj+WYayheeI+Z/P+uvNL0ie344/H2EQ== X-YMail-OSG: B47ojbEVM1nqPHtZtQohdkx.Jw0uTreT5y21sbz0_DUlHCM4.WJZZG16gvxnS__ JjTwKJkiU8Ufv1NElLTtKExsbsUiwzEGlowzb_k.8KW7b9XjkNUZlxBcXzT3rAHrfG1YIalV6NTu YBQBUpSTmgYvVPqr3whsYKb8w2k.GUDli5iFxjDJzW1cTEqxamFEY3PrUfzV8dtZaC21RK1fq6Mc 8Dz0EkQVSlzgoXwx3iJjTVXKPPqaYkNDLxof9Smz1Kru108OXS_cxzi2rMuzt8OSLZbcD0wHCYGI psOebP5SqW96QoqTXs_s20asYpCWWbcc3JOGZJWInycufDg8Jzhd2GDkjSTNsrpkAL9aZuddxjwO kWwITq7Cg6Z2dnz_fvpSjfQgj7YKgkdFeyDF0nXbU7KuzesxJ1w2ouSiMoYlwOkGHpLelhYbMFTo P29D1j6_WEfa_1AP5AD0i6VjxN_RMbYQJaFrxD5UxeH7DRm62u17u3yzqzP0Qz9tbUdmX8qc4evk UKzcfRmjaZoxoeinAatElU_xKLxggKOeweqcMgrYXO.efY.bFk4NQbX.pT4ZIQLIdS8n5YfnNcrO 0tIZrubvAhMfrLSAk0w1c92oZiAYvFQSymIPuZeAbdn2q0cD_XcuEnYANBpeuJ6dxWpNvWNsDV1u 7dP_FZqiVBkV1DyYo0QrGY5DyoTaVbPvzXemFI0c02NxyLf3A_TZ2sKVdBPRXnko6axF.IU16eUs Jtet96wbLvQDBXb1yAMxf7hYP.50tqMFjcqujNUH3YFS_6iJQnZA1uomPMmcrzp8olNXl6wQDLE0 cBtW_nyXNJRmLA2.RWITUm_edF8BEqikf5Ac1i9nRuRPj0hIPy7HvX1gEZHCzS1i2ggKXpDyV0s. S55G2YqQfLtQQfAkCX9hNYQB.kj.KDpqr4PDowNZqeSVlX9ZY9202uEc1AaI8u.4jl9GgLP9RK.8 HXraZB2w8bQ1DwSuGYb7GR5aOBEtFBVxjQgCwLqWYNN1A6DxakQ9Ak03fqeAYG1sOsgu.kFyeBd2 4TbIigjxpNIyaC2nhTzWrl_3YV54AkFvFGr8YWqG7G3dtVOUBhpKz568eYwDW2PHKqMyM7YsTdup tRK8oLHAcOAmpcUpZKIm6Bj1uPA2BfUYR2F0ScSLLQqgoMuEe0DOSxkguCP52xba2oPcvELauQ74 HG9teShSQbsg2DP.VZStmgQSf5jI0bbLxaCSHf7.oFhm.FKy.qBI4KJ8oWHDG6O.PJf8XogFjiLI FS6pJiHvkPEueBcMyNUTYXfVuNvCC2jvf6UsNUfetMx2jqnDE6P2mEpku9Wg8ouhyQsYMj0KAetL np8gGrDOBNlyIbs5NvdNNLP2lWdVRBfq3zcoXZH7QJJZstEmtEpyy7XUO7G27jPK37jvgbD4CYTi rYplJHfyGCsHcwULUdL1jrrr2Fidm4Sv_saRw9I5cxDdt5CInjQNB4Wt7HDoioOhdBSdbOJb_bqS rcVE3YD7QMhlimsX245qZpZtmOYdcNJiyxNa3BGpamp00auzo724.MNpkaYfqYT41LeQn7cgMgj5 88RUAt0GFPWlvnz9iHj6PrGnyssWE2wR7CIlbbo4VQBkwRXh5eVY3l98XoKarMaprIskvmAyqna_ cbH.Xykniw40QNoVwZcdKAAWUqUzTYd4K8xa1A1ZA1C0P0Kj9Ca8r3lDgA5eqbkVsSiwCLo4ZVG8 L2HDu7dJms0hlwgN344AzEaowYHoDslIZz7L6RYUrlWzEa5mh2y01pqta37XlLhUbOR5tID_64Vv eHLLpoLyvF9YQKFLVWb7hu42n3IZEwxrP36DOy4c9NByTG6lSzQB9NuSBx1deqQnIQZhBNaERMM7 hJ6zeOqxC1sJA1bz1gzgY4BHf0oHID9OpMtLx7mHOEDIOQezjJ_BoWb3hnuEg8m9L.a2f5JU4Z0K Kes_Q_lNliMUIMteSxGOVUs065OTIJhNdnUlUZZKouSQkaquRmcAI5_C8m8pgv8WrU8bF1GdQ.55 T7yHcwGnkVqjrupWZ6tmgo418pw3We3Tl93xaVAj1y5YxSd2KguRvApu3_bfxEXJcb7dPicD83yQ zzcldkXOAGndkUxvP7Hr2d.PwHUzIEva5w1.Lfe_TMa1DLWsdV26emLF4AvM0JhITZ_thRuuJZkO VXGLulIghNxjkT0iK3BtugYB.TC0cE2f.tGjXWs02a4EZLGYez3OVpGeNGs2Dt35MKNshEBnUDNt r4Ifqw3OmtZTgVBHUb49JjGx1uT4aLPLVU8ZRD7A3oUlw.bi9vNbwYYKzWzNhjwAiam7CKCBjBs6 _HwCOT3N7TQ-- X-Sonic-MF: X-Sonic-ID: 5acf8791-a29f-474f-b383-0802f0077ea0 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sat, 9 Sep 2023 00:03:25 +0000 Received: by hermes--production-ne1-7b767b77cc-6mpms (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 09a2eb0089434ee07565f42060571837; Sat, 09 Sep 2023 00:03:19 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: <63717d32-f340-1320-3335-85135d1b62bc@FreeBSD.org> Date: Fri, 8 Sep 2023 17:03:07 -0700 Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Content-Transfer-Encoding: quoted-printable Message-Id: <05C47E15-640D-41AD-9C4C-73A1D5041CF4@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> <63717d32-f340-1320-3335-85135d1b62bc@FreeBSD.org> To: Martin Matuska , Alexander Motin , Glen Barber X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4RjCqg5tLxz3CkJ On Sep 8, 2023, at 15:30, Martin Matuska wrote: > I can confirm that the patch fixes the panic caused by the provided = script on my test systems. > Mark, would it be possible to try poudriere on your system with a = patched kernel? . . . On 9. 9. 2023 0:09, Alexander Motin wrote: > On 08.09.2023 09:52, Martin Matuska wrote: >> . . . >=20 > Thank you, Martin. I was able to reproduce the issue with your script = and found the cause. >=20 > I first though the issue is triggered by the `cp`, but it appeared to = be triggered by `cat`. It also got copy_file_range() support, but later = than `cp`. That is probably why it slipped through testing. This patch = fixes it for me: https://github.com/openzfs/zfs/pull/15251 . >=20 > Mark, could you please try the patch? If all goes well, this will end up reporting that the poudriere bulk -a is still running but has gotten past, say, 320+ port->package builds finished (so: more than double observed so far for the panic context). Later would be a report with a larger figure. A normal run I might let go for 6000+ ports and 10 hr or so. Notes as I go . . . Patch applied, built, and installed to the test media. Also, booted: # uname -apKU FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #75 = main-n265228-c9315099f69e-dirty: Thu Sep 7 13:28:47 PDT 2023 = root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd= 64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 Note that this is with a debug kernel (-dbg- in path and -DBG in the GENERIC* name). Also, the vintage of what it is based on has: git: 969071be938c - main - vfs: copy_file_range() between multiple = mountpoints of the same fs type The usual sort of sequencing previously reported to get to this point. Media update starts with the rewind to the checkpoint in hopes of avoiding oddities from the later failure. . . . : [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 414 Failed: 0 Skipped: 39 Ignored: 335 = Fetched: 0 Tobuild: 33800 Time: 00:30:41 So 414 and and still building. More later. (It may be a while.) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Sep 9 00:27:52 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjDN470yNz4t76d; Sat, 9 Sep 2023 00:28:04 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Received: from mail.dawidek.net (mail.dawidek.net [94.130.64.56]) by mx1.freebsd.org (Postfix) with ESMTP id 4RjDN40Pplz3LjS; Sat, 9 Sep 2023 00:28:04 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from [192.168.250.133] (c-73-70-21-143.hsd1.ca.comcast.net [73.70.21.143]) by mail.dawidek.net (Postfix) with ESMTPSA id 6EEBD22F73; Sat, 9 Sep 2023 02:27:55 +0200 (CEST) Message-ID: Date: Fri, 8 Sep 2023 17:27:52 -0700 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics Content-Language: en-US To: Alexander Motin , Martin Matuska , Mark Millard , Glen Barber Cc: Current FreeBSD , FreeBSD-STABLE Mailing List References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> From: Pawel Jakub Dawidek In-Reply-To: <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE] X-Rspamd-Queue-Id: 4RjDN40Pplz3LjS On 9/8/23 15:09, Alexander Motin wrote: > Thank you, Martin.  I was able to reproduce the issue with your script > and found the cause. > > I first though the issue is triggered by the `cp`, but it appeared to be > triggered by `cat`.  It also got copy_file_range() support, but later > than `cp`.  That is probably why it slipped through testing.  This patch > fixes it for me: https://github.com/openzfs/zfs/pull/15251 . > > Mark, could you please try the patch? Thank you Alex for the fix! -- Pawel Jakub Dawidek From nobody Sat Sep 9 01:19:09 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjFWL2JKFz4rw1p for ; Sat, 9 Sep 2023 01:19:26 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-21.consmr.mail.gq1.yahoo.com (sonic313-21.consmr.mail.gq1.yahoo.com [98.137.65.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjFWK09NKz3c5n for ; Sat, 9 Sep 2023 01:19:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=tphWE8p3; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694222363; bh=wqX5AFH2jH+nSZtZh3giI4iv5Iq0vNudXur0k95KmeE=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=tphWE8p3+zDN4B2kRkBB7jA5NQnQZTwQVuNulalPc92bwd3jUKaLXhfG50WvjmUHp8ePGH7I8AHTWVXzScfoTX1skjpaqxP6AIrS05ItLKMf4HT6K3heOploJkwq1NuBHqPS26+LHr957bXhJyvaRx3+uJwEuj3/3su+tAm+632LtuoNl4G4AtbhhgaZTzUUBNFD6JOn5Ri9WOj2srVSI8whKTuNMaObKpZ0ABd8lkoZWvgETAbcKNuPRNOLOTjKPfpfEQRRQs1gLYGbh/LssdIWQUmrvsxSr4ZWRREc/Bq5Xw8A3qzb9O4bdA4CKSgKmvwqe7biSPNHk8unvqIbpw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694222363; bh=1TpCDeS5If32Qv/NV8n3/H/SDYbpxTsehHbgFlgrEDk=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=G1RgmNrQMarxfrOXFfizTfT1OEWgTG33EnVqiDEP4IvxWKASniVQZ6+pEGarHZv+ND7WAa/uZhqMvSqI94YNWk+HSV4XYifXBa6ZKPr+m9iQ+ptTKIE1pQJS6e7oBsAEgg/JxF34T6BTOK+ExIP2NMmStV//jsaYmr8EYKv/YK+nzXgpbySSuuzzlxsp5wVal0ul8Ao6nglqCFsJPuJqpfG6xyQS3k7PPjfNx3ty6U9fldxV1JjnPgE7U+TYfGjQoNckN4p0mf1dNUhndCRdngVPQjvKMlt58/TtCBAAyGxOOKo0sIbankYkA4IKB42lYz5REQp/4FHRrOxk6GEvAA== X-YMail-OSG: SHWlyX0VM1nwkdKGbY0mw7w.xpYhCnuu3ZsUjiKH3_osr4o22ERo2aeQPMMYgfM y9vEUex8UiL.eoGiF7WkkGeS2mAuKlM0d0qSYuFyz78ruteYkckPixJ2u3hFRMD0tS2KO3nh1zZ9 Tw0oDcwfB8s9KuYE8ONj3SgM.svvj_jo0NeCk0yN.V67YIOF3tx_xrDdnrb4QiYzdqfg9T3BcQj4 8K.Zcag4KVOgBV4oX9yYfLypftnY5Zw6eG4Dn_MDfRguBrxUbZnahlZq.SDmwR5I.EzEcVojDWaw EpLYuK7ud9T9uq3BusD.amxrWkTZwR_YGK_FcNuSM.CEw1DR1yIQlSK0Cfq8HMi_b_SUzg0d9ADd A0gK1mBJE0Zom84JSpkdmyYpKL2K59f.4FK01gydJsT1JsxmwUxE4gtg_LKkATlB3rQolFZicnSS VCkTw_HTrcFuZ5ZqlNWHqlNJafg4oitOfAV5lAc7RClPYThLsKugmSl.l7HyDBXRsLO5TN1Q9qnX 850YPQ6ul32JNFo_4vx1hS_hG96DCFguUP00tqYgOtg.4fjAh93eQUnAf_32xF7PRzbE.1Auc9CY .zew3mIxeD1ZYF4WcaHX.jEjFO.UDGmpvC0FJMW3urAK.OhtQXPExTUcAb9KwtSt390Yd7Kcsiw6 E0Stb4ims7SY6doUsAgZkx9fmu5ziEnY326pySqq0B5LNst9kXqcfS82hXZ1IHBIV07HICQ3G1YT 473rflZ.HRYS2O5jjfhAlwFhbS5jcT7DKJmkl.hL6iSylLpMCLYMfW1rOEsVsYl7QBAWd0QyFO8a RAa0WpAOCDV8eK7X9EV_biPL9KpzNFzXIu6uAvvKzkOri4pBSRuGNXFWwTlRvilcDwYxr86iGhM4 Txa6Q443.4EgzUVtG7ilqIGlBtqZ_zLV3sG487DWBl4cPP2_IIoX3UqfaGzBrYy81JljOPIKH131 lCug5T9qfmy1O0wcP5isQAopt3XxdZHp1.ttcU66CGPWPHhurd3G3wqTWE2wdvpj9hRAQICD5cDX kEgNPNDS38jmNjWenm7kpMYxEw49voMyd9lFiHNDuWTOHza6w8mzNT.YPgmImfMXA3Yxfo7S0USu IxkiMkTGZWhYms59Gv9_NCFMdIu.jlk69y2KaJ1ITzaHmTjaH_Ihyzo9DrQ9B1tJQEOJIBht_KZJ sgty6UX8kyYgSRrd_m4lqsaPK.YHTqLcioJj8iyjBTMoMm73Hv19dBIqiSF.05SKvV6Mgcq9qtsg pNb3NSPZ5rpkcqHLlD6rWI7SkqVJtyDOJyzzkDpmFRjeQ2s7RI.AaYjyf6s2ka2l9Xu.8mcgU_HL OzBmz2bOS3j9qtcfGAzHVTSLTXo.n4Wv0vTeZ9AGCbWE3X4wDrZ6bQWKFErASLrgd3QH6MiPrQ8d zLSogHl8qx0qqHrSddAATQ.Qw0G5yT1bTL9LoY_L6CXwT32yxcMItKUSuQn3tFboFbFzWlBSAIqT 9OGzqNdgty3XuQr._OIqbLNTRmCLRs00pBOfO002Aw1WvmJyDgHmbv6boTWV2fGQ0G_RSuSHgjr1 ephbGgbLopMMcWcHf41.i3ZoqCUJjyKDnWShGaxqvicp4RMBiCivSosrYnUCxb8ehZFdKfcvAy3S SCe7.NUNJqpYv3RV5OmdUseo4Ghg.3FMKHdvhdtSHpf6SN10M9utuBpRNxybeGhTV.Q4BLs7USv0 wVct2w.fqyep99NeA9RsTrf2Vl2D0rGbRMsLNwWWLEytURaPeIVocgeIJvVEOQUJ6tlcrAQZqWNK 7NAGjwPaPkwjNQNVkJETqojrZfsmo2k4NG1tLoBDfaTpDozQflJ24miLbgVtC7uhWSxu2nID9x7L beulFCMlFH11AnhqtuCRdkRosiduu8c4Lig2hX.KbByqyrIYqYsQ2FwlNhguHdzViVXtYc2QKmzE itvClJRonD9BYhcci47ngMwrav2XJkJ9oDJZt2qRSiQQgIDZaQjLmzm.PNwkB5Gz7FW6mFhX5nxf UbbfN6tMr8tFzx9DmZe9Qh3R7W8fNi12a.eJE4q1EtjOKJBJEAKqfPn65mq7Xaw5_KZv7IEByRDO Nhz60koXbB7aoQIhEFyn6e3QejzPH7E5BD1OQKoFMUF74Qk3XGRAcKo1LEKM.xKr6K8iFV9603rS lDnIBK56FyIofyEzFc1qxe1HBduG5i1B1lZRrAR7i7crP06cJyHYUzRgumYVRykTCWP7I6gq6jug rKpC0PfOed3sfmzE48pHxXYkl8ZbDpJhhlLqQq9vhwyf21OxRcKchFilx882I1.HUcCc97o19CVv sUMZnb.q0rQ-- X-Sonic-MF: X-Sonic-ID: 8943c28e-9c3a-4d3b-a271-a12530d7bef4 Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Sat, 9 Sep 2023 01:19:23 +0000 Received: by hermes--production-gq1-6b7c87dcf5-m4lb7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 25afddf1784b80f9a64ce82e4d6dbd5b; Sat, 09 Sep 2023 01:19:20 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: <05C47E15-640D-41AD-9C4C-73A1D5041CF4@yahoo.com> Date: Fri, 8 Sep 2023 18:19:09 -0700 Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Content-Transfer-Encoding: quoted-printable Message-Id: References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> <63717d32-f340-1320-3335-85135d1b62bc@FreeBSD.org> <05C47E15-640D-41AD-9C4C-73A1D5041CF4@yahoo.com> To: Martin Matuska , Alexander Motin , Glen Barber X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.47 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.970]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.84:from]; BLOCKLISTDE_FAIL(0.00)[98.137.65.84:server fail]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.84:from]; TO_DN_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RjFWK09NKz3c5n On Sep 8, 2023, at 17:03, Mark Millard wrote: > On Sep 8, 2023, at 15:30, Martin Matuska wrote: >=20 >> I can confirm that the patch fixes the panic caused by the provided = script on my test systems. >> Mark, would it be possible to try poudriere on your system with a = patched kernel? >=20 > . . . >=20 > On 9. 9. 2023 0:09, Alexander Motin wrote: >> On 08.09.2023 09:52, Martin Matuska wrote: >>> . . . >>=20 >> Thank you, Martin. I was able to reproduce the issue with your = script and found the cause. >>=20 >> I first though the issue is triggered by the `cp`, but it appeared to = be triggered by `cat`. It also got copy_file_range() support, but later = than `cp`. That is probably why it slipped through testing. This patch = fixes it for me: https://github.com/openzfs/zfs/pull/15251 . >>=20 >> Mark, could you please try the patch? >=20 > If all goes well, this will end up reporting that the > poudriere bulk -a is still running but has gotten past, > say, 320+ port->package builds finished (so: more than > double observed so far for the panic context). Later > would be a report with a larger figure. A normal run > I might let go for 6000+ ports and 10 hr or so. >=20 > Notes as I go . . . >=20 > Patch applied, built, and installed to the test media. > Also, booted: >=20 > # uname -apKU > FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #75 = main-n265228-c9315099f69e-dirty: Thu Sep 7 13:28:47 PDT 2023 = root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd= 64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 >=20 > Note that this is with a debug kernel (-dbg- in path and -DBG in > the GENERIC* name). Also, the vintage of what it is based on has: >=20 > git: 969071be938c - main - vfs: copy_file_range() between multiple = mountpoints of the same fs type >=20 > The usual sort of sequencing previously reported to get to this > point. Media update starts with the rewind to the checkpoint in > hopes of avoiding oddities from the later failure. >=20 > . . . : >=20 > [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 414 Failed: 0 Skipped: 39 Ignored: 335 = Fetched: 0 Tobuild: 33800 Time: 00:30:41 >=20 >=20 > So 414 and and still building. >=20 > More later. (It may be a while.) >=20 [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 2013 Failed: 2 Skipped: 179 Ignored: 335 = Fetched: 0 Tobuild: 32059 Time: 01:42:47 and still going. (FYI: The failures are expected.) After a while I might stop it and start over with a non-debug kernel installed instead. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Sep 9 02:30:06 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjH574XM4z4svd2; Sat, 9 Sep 2023 02:30:19 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjH5674dkz4Tq6; Sat, 9 Sep 2023 02:30:18 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 3892U7r0007987; Sat, 9 Sep 2023 11:30:07 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sat, 9 Sep 2023 11:30:06 +0900 From: Tomoaki AOKI To: Mark Millard Cc: Martin Matuska , Alexander Motin , Glen Barber , Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics Message-Id: <20230909113006.47f1f0d60a8c9820131e8020@dec.sakura.ne.jp> In-Reply-To: <05C47E15-640D-41AD-9C4C-73A1D5041CF4@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> <63717d32-f340-1320-3335-85135d1b62bc@FreeBSD.org> <05C47E15-640D-41AD-9C4C-73A1D5041CF4@yahoo.com> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] X-Rspamd-Queue-Id: 4RjH5674dkz4Tq6 On Fri, 8 Sep 2023 17:03:07 -0700 Mark Millard wrote: > On Sep 8, 2023, at 15:30, Martin Matuska wrote: > > > I can confirm that the patch fixes the panic caused by the provided script on my test systems. > > Mark, would it be possible to try poudriere on your system with a patched kernel? > > . . . > > On 9. 9. 2023 0:09, Alexander Motin wrote: > > On 08.09.2023 09:52, Martin Matuska wrote: > >> . . . > > > > Thank you, Martin. I was able to reproduce the issue with your script and found the cause. > > > > I first though the issue is triggered by the `cp`, but it appeared to be triggered by `cat`. It also got copy_file_range() support, but later than `cp`. That is probably why it slipped through testing. This patch fixes it for me: https://github.com/openzfs/zfs/pull/15251 . > > > > Mark, could you please try the patch? > > If all goes well, this will end up reporting that the > poudriere bulk -a is still running but has gotten past, > say, 320+ port->package builds finished (so: more than > double observed so far for the panic context). Later > would be a report with a larger figure. A normal run > I might let go for 6000+ ports and 10 hr or so. > > Notes as I go . . . > > Patch applied, built, and installed to the test media. > Also, booted: > > # uname -apKU > FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #75 main-n265228-c9315099f69e-dirty: Thu Sep 7 13:28:47 PDT 2023 root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 > > Note that this is with a debug kernel (-dbg- in path and -DBG in > the GENERIC* name). Also, the vintage of what it is based on has: > > git: 969071be938c - main - vfs: copy_file_range() between multiple mountpoints of the same fs type > > The usual sort of sequencing previously reported to get to this > point. Media update starts with the rewind to the checkpoint in > hopes of avoiding oddities from the later failure. > > . . . : > > [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] Queued: 34588 Built: 414 Failed: 0 Skipped: 39 Ignored: 335 Fetched: 0 Tobuild: 33800 Time: 00:30:41 > > > So 414 and and still building. > > More later. (It may be a while.) > > === > Mark Millard > marklmi at yahoo.com Would it planned to be MFC'ed to stable/14, and then releng/14.0 once MFV'ed to main? Regards. -- Tomoaki AOKI From nobody Sat Sep 9 04:54:14 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjLHW1hkbz4sBZ6 for ; Sat, 9 Sep 2023 04:54:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-21.consmr.mail.gq1.yahoo.com (sonic314-21.consmr.mail.gq1.yahoo.com [98.137.69.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjLHV0Tjdz4DRG for ; Sat, 9 Sep 2023 04:54:29 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=EKK6YCv3; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694235267; bh=v585CdV9aZ5BPDMDMynso/u18oN39eBJDpgvByZYHLs=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=EKK6YCv3/H8CY0qXxdj4WZb5KxrOVa7HCDz/uvU5cP8st+hR8TzF5IT98tKILQlXoYTMntjCDYRiWdTV3YthDsMJvrJ94tb3r7tp+JzA8n6NvLqlPCCLOb4E2uOCV6O136o9WNZvNnHiMrTraiL98it7iC1S/DyvcYW9CI3u53EXvMSwLa5LlWrD7dvjwcrVIXbe9vaUjBZcUYZ1MLoq4/E5aFllM1xX1yqUK4YoIHpxjRXHBnx1lCS26uSWiXpSzZrYVbm8YgEgXEQN1BjZmTbs1fd378ALpYX21QS+MLk8VMQGGV7p8AgJh9AwnPpEthyhIfPeFq2G7f1VYZ+/Ag== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694235267; bh=lfgbNcpldc8ZauWVBpY7oLccS0XNDjvqHdWxUCn87Kl=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=p+Enj3LxwwfX2jAnzT1D3X6S8Lhb7uNt9odk7/78uY9/MiOruM9RGg7frEpBlp+TWv95MgkrN9Gh4G7fGVXxSvASPxt3jKUNuVPHmpvkr7qEvKuqF1LW2jnC14moLcRqRcthHWbBKApgx3Z8c4K+dGZ5ECHMjGGPiWncQA1NYuhNRGW0aIE/ngVlv2p+YFbnXDnsbx6Q0pXhSfG+/4mIuKg9oZB0pY/bLcOuFV0iK0WhcoFJKBtLykj3RJveO05Zi8ZmYwiCOObBMufeUhPgccb4mUBuqNIVO1PRWdg760S4xu1/27GNsvNP0L/uHCkHAb2XFNalvB3zXvO+n9NsTQ== X-YMail-OSG: p81Q6soVM1m_XWNTO7l1.pvAmHNtvdCdr8S1KM7O05wCtfGqc4x_gXZcbA_F5Rz bwn4cUBBh9Jqx.1cCPmrbkXOqsvglreqSa3FyyZ7dX.gXwuVLP5N9PCle2G0ikQVLTOY38UWUK9M GmuRthKtlSvqoo3dV3nomivLTvYWXJFB33rk69CeJXgXs2_2jE661t82dQQyO3HeY3G98egBEhR7 7aQnKhu159YZcrFfoEKyqOg1d5LSxd3vwJDraNn8Y0pOF.8aYEvePawTodAl4SK_1ZDWrQoPM9Md 7NGsWyFMleoPQcKZ1wTSRl5GwKrCig.GtLD7SpeAHj9ZazXhNiIiL_9pweIHhZGZA.tEQVnwxcsW SmJQ2cmAOy9PHKr0vKJVgdZ4EnYL2y5xvVyNq3RX8DU5wg7_reak1yW.m8mBAY0lKfJaUVt3mF3c JDkY5IYoFpjddspxR9r7dA.XEW7BA4L4uPmWHUb_F3K3kYefQ4e80t1zDqQ2f8ge4qiaCCrZ74lG 5SF1_uG.6.iSqHJxIAooPkYe3saHj642QSYho0BHyGgSbYi6qyweNHPs0G2.grW7DBW3Zx4PoIt1 TdS9DTo6S9tng_DfCft_7ZULL0oOzRp2NYBJfFbBcxJvzg9U54Bpj8rc3KkBBwqlmAyCrdP4l1_s oaPlq9gun_Me11hhA1KK9qGh5oqxoXrBC2eiGBOzXjbMZU3ItIci1nfSlHFH6uqvyE7Ryb_MFDnb tXXO8kiR.Lz.gD4fjB_aE02hd_nHFiQXdfRozvg2hmWk6QnmkqkbgtOxb08.XfVylldj3rBisMuB BnGu3tqc0ScEk2U5ZXUoN8mQBi39Ll3R6l7cKqNOXBZmS78aQvln7GvirfUY3oZOBugJLJZVa9qP a9rZLR5ZK1N5OTZyK5n6MNoi5fM1itDC7VHu.v2e5iMaz_r2ePA3km2wNK7O50ezJxlNtEFTo0H9 faxptq9UmT0qTE5NT6BVGjwDtlPgYjgsy7RV5CGpSkO052pM6EJSaUNCtTDDSse7nqAQi51GfzQt e9oJtairXriHKpSLiH.POTyuBH7zidOdIRloZnbT6uJ4UkRGdSQ5UJTPFnpyzmK9j8aSR34dZErG 5B0mYA6i.18vQz3rgPtY8junN4N208vwl2UF837qFt5kgCnY5N_62QDTBF5y86EsMz8vE9WpSMna DXSSJ_.CLccnt0PzQnZcBja_O5Kc79jlZNAgTT7XxuEo.bWhpn3w.qlRzymbwxKAs7Dso5L_KAIl gUAvQZH2Mhzp3miC30GG.Oz9gg43.ue0zF2hEaXbe73P33hnKXLZElLjtX0kdJcFtxvaXWYW5c2E .nGhHM7PjEVPtqLxvTAEcWSi7g0f.N6m7vUUSkLbrHfZJeff7m_h6o3v1lBTc43LEFTZGyRmWiRO XINiAtFgnusjcCzzsTPVr5LMZMoNkppiKdFgUeELUD73GLhTSYn_wubLxnuUu1gMJUsdJ1G0r09L ph5D3SwmdMFtGK_flMh_sKlsC94R1ZZEXKHgMBYxphaTSZ6FTng5UdjACsMRxSKNvIJxJr.XyhRk ZxWElNdWlTcoU6Zi6pDukLlXpu8g2NvfHm0E6oiC6EBfyDlWr.rOfKBfZHVsPoPTFTrn5tnjktgj br0Inv4NK_R3hXENa6vlkytIC1BkL0Y8DtrmtvCjBghi2dKUutrlY34jWs_FiResnZ4YyTEi5il0 UvIvXABOoIG0OD6WZn9Tys47R.XLCCdHq4xg7ebwmmO4r6yc9.kmRNnpwx7xFOQx73BPfj6qKrEg 1L1egrxbBKZ84bgkE9e6t5D_FcdArYruI8ZgiW1WdP3kiZF4jP.slaH6cbCDyi9sJkBG3w1NH425 qL8xRj1y0dSuUTPZfTF8AgDXarHux9lQVmc5e6csFVf7MVDeoiu75Z2nV4mEDszKBaWOt_6k87rZ aXdVapfgOsIpkX8h.wJo4.tMFALsS9BZbz5FixP.ZE1RkfFqkkIffoiMK92zuIrm_cISV_Wi33iv vm3aiplPAiRszqXeexpRDw0A9fnO220snMwZvVUFCCeAI6jO7UaBYfBfItwntZH_TXXGJmn9ilCy 7QGfrxBZ_nlTjVJ5QsXji4WNxpItWV1oCA2fe8EtuIK5M10VLvyKdKkXPd0UJ1rxZWt2tElzxveA qRO5ZmGafLZeC349Gn8nGchfE9Ad5TIeDX1HEyMsj0On4y_S0vfaRphDPVdhrUqXPr2EOgaEyQ7q qdYNiYUmJc3P8yVijjFmMK0sBv70dtL4JtDY.tQECX32hcPiuVa3QK9MoHGqjyCJAPg28UuRNsGk vbfd9mZEKLOs- X-Sonic-MF: X-Sonic-ID: 264c2783-4b2e-48d8-b189-d16e630b9fee Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Sat, 9 Sep 2023 04:54:27 +0000 Received: by hermes--production-gq1-6b7c87dcf5-wlch2 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 407869484ee6bdc0f2b208629e18a038; Sat, 09 Sep 2023 04:54:24 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: Date: Fri, 8 Sep 2023 21:54:14 -0700 Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Content-Transfer-Encoding: quoted-printable Message-Id: <8746A218-F83A-40E7-95F8-5EC1E36411C1@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> <63717d32-f340-1320-3335-85135d1b62bc@FreeBSD.org> <05C47E15-640D-41AD-9C4C-73A1D5041CF4@yahoo.com> To: Martin Matuska , Alexander Motin , Glen Barber X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.988]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.84:from]; BLOCKLISTDE_FAIL(0.00)[98.137.69.84:server fail]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.84:from]; TO_DN_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RjLHV0Tjdz4DRG On Sep 8, 2023, at 18:19, Mark Millard wrote: > On Sep 8, 2023, at 17:03, Mark Millard wrote: >=20 >> On Sep 8, 2023, at 15:30, Martin Matuska wrote: >>=20 >>> I can confirm that the patch fixes the panic caused by the provided = script on my test systems. >>> Mark, would it be possible to try poudriere on your system with a = patched kernel? >>=20 >> . . . >>=20 >> On 9. 9. 2023 0:09, Alexander Motin wrote: >>> On 08.09.2023 09:52, Martin Matuska wrote: >>>> . . . >>>=20 >>> Thank you, Martin. I was able to reproduce the issue with your = script and found the cause. >>>=20 >>> I first though the issue is triggered by the `cp`, but it appeared = to be triggered by `cat`. It also got copy_file_range() support, but = later than `cp`. That is probably why it slipped through testing. This = patch fixes it for me: https://github.com/openzfs/zfs/pull/15251 . >>>=20 >>> Mark, could you please try the patch? >>=20 >> If all goes well, this will end up reporting that the >> poudriere bulk -a is still running but has gotten past, >> say, 320+ port->package builds finished (so: more than >> double observed so far for the panic context). Later >> would be a report with a larger figure. A normal run >> I might let go for 6000+ ports and 10 hr or so. >>=20 >> Notes as I go . . . >>=20 >> Patch applied, built, and installed to the test media. >> Also, booted: >>=20 >> # uname -apKU >> FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 #75 = main-n265228-c9315099f69e-dirty: Thu Sep 7 13:28:47 PDT 2023 = root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd= 64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 >>=20 >> Note that this is with a debug kernel (-dbg- in path and -DBG in >> the GENERIC* name). Also, the vintage of what it is based on has: >>=20 >> git: 969071be938c - main - vfs: copy_file_range() between multiple = mountpoints of the same fs type >>=20 >> The usual sort of sequencing previously reported to get to this >> point. Media update starts with the rewind to the checkpoint in >> hopes of avoiding oddities from the later failure. >>=20 >> . . . : >>=20 >> [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 414 Failed: 0 Skipped: 39 Ignored: 335 = Fetched: 0 Tobuild: 33800 Time: 00:30:41 >>=20 >>=20 >> So 414 and and still building. >>=20 >> More later. (It may be a while.) >>=20 >=20 > [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 2013 Failed: 2 Skipped: 179 Ignored: 335 = Fetched: 0 Tobuild: 32059 Time: 01:42:47 >=20 > and still going. (FYI: The failures are expected.) >=20 > After a while I might stop it and start over with a non-debug > kernel installed instead. I did ^C after 2.5 hr (with 2447 built): ^C[02:30:05] Error: Signal SIGINT caught, cleaning up and exiting [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [sigint:] Queued: = 34588 Built: 2447 Failed: 5 Skipped: 226 Ignored: 335 Fetched: = 0 Tobuild: 31575 Time: 02:29:59 [02:30:05] Logs: = /usr/local/poudriere/data/logs/bulk/main-amd64-bulk_a-default/2023-09-08_1= 6h31m51s [02:30:05] Cleaning up [02:38:04] Unmounting file systems Exiting with status 1 I'll switch it over to a non-debug kernel and, probably, world and setup/run another test. . . . (time goes by) . . . Hmm. This did not get sent when I wrote the above. FYI, non-debug test status: [main-amd64-bulk_a-default] [2023-09-08_19h51m52s] [parallel_build:] = Queued: 34588 Built: 2547 Failed: 5 Skipped: 239 Ignored: 335 = Fetched: 0 Tobuild: 31462 Time: 01:59:58 I may let it run overnight. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Sep 9 16:32:33 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjdnJ3WC0z4tFMM for ; Sat, 9 Sep 2023 16:32:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjdnH4646z4fKl for ; Sat, 9 Sep 2023 16:32:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=UgzQdwsK; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694277169; bh=weTyd1bs9Rvq/AfYBi6SheE/T1juWcs47saskxKNi5A=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=UgzQdwsKr4C2C9zP1tQFW2MwW2VQqAt7+PpF3eyqV37+pVXuq7l+MKqCsRnfajrm07I5JQnzamUSuqzWQkJEuxSRtxUS+Yqd6YypBeEsQvHj1ygSG5BHXNt2rreaNlidBkvaonrn9G57fg6xJZgJwtDSmorT1qkyfSjdC5XY5SYPOWGRZoVBmOjmWo7W5Y275Jn2DPGUvKvvYrInDjGjlqKzdBk5bPIuwqDsnZ6aCuA0vtAzDD99ODqrYCd0zAjEL/QYckwitBsI4TC9T+nCORuC/S1ZvD1UA85tB3likTPG0ccqqTo496Fufzd+hgYKUsK2E3pfBpIK5KxwgjtWiA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694277169; bh=IaNjl5Xyh86UoYs6V/SLmByy5XhBbxcbF0P12XlyEyD=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=lW32LKSFozRint7dznXgQZlZ0VHA6p6hSz/LvCb5X/qe9MngAfv09Dg/ydhTxtVwVV6eVifsvAbzMWqYT0f1E8Vxk2RVBHrd94b/lE+WTiVFaKv5NwJq0XNeDb3D1A7xAsTc25jagr2WV4POptTL1txRg7mGXbXj0gQ0CYAuGPMSHGqYXClqNw2gHv03W3YxwxOU4/hSiYcUGWozCTKet0fFj7zD5ACVmtUdZ2poAk+zzPqYGip3cSc52ylcofal0NR1vl0drs+oIzYOTBEPsTX7ZbloEfn0UKYvb74R9lQoaIsUV1WplXb5S8AtZsj2BU+VTKU02VUda7Se99aRVA== X-YMail-OSG: BguVTfQVM1mUJxxH9Q_s3TcW_raiwcFx13_4kwzgeFoeHvotY1jIut6eAne_2Zm EwGgQEHF7EQc7UBvj2MM.PSc462TISmnLQ4HdQ1ekzmpbjgK_zGCI_1x6egKXs.y2pJG7vY7C5qk SArlWNlFzgt0kg2JF5YwNjcE3RyoFdbNl1ehRAJCQcAt7lKDr2Oypc3X44IQGz3IcMLTWjkU_8EX Kn6zisshRNFCKTgCnFoDkOlkQkQCnc37hH1bWDwg_Q42I9PdvcIVaK6HrzhVqmynwmbcoiRGbPob .zdiYxR9DlNJ3WSQJWn67c051qmWJouvGqNibEEEi8cBM_NJ_eZXShNvmbQieVyAlYHpmnsV85X_ hi48q3x8l7XxJQtNYjkSKt2vwWExMwRtzbQzbW80hTIU7C2Nbke2RWWE1KojAE5xCVsyM9j8jBXR rfCeayyRxIcMmBk1CoHiiKgmvsc.Om1ViD9Lh7SCwz4ddoexBR.TdyI4_v.DlKomFh3mQg34w3hW IaEzs39KQUtEREFFVkjrFVdyaY.osjX_kxjYU7GdYXYRsAGQKQa0J6G.gb9TSIm6cDiObqn0AY.D 81bQLPFTv_Z.6wRpQQYBsRAxzmpBrY8dU2c_3FasnNXcUp0DjOWN2FrhZWVfKexLfv08v1B4bwWL pyfSl.6aVr_qXaqk_nnhJ6wtJrrBWv0sCbfAq9aWIdFvvi_i7ksEwU3uTFHpbZCqvWcPGmbCIURw XhDxc0fe2BAfjd81pHFw1JBoCs93nUzWAV6G3pM9SKxf8RosaQUgSlQ1MUqK26_vTC.fQg4ZVCTo tQ_PXD.JtkutQhinlnbgrj23JYcTSWly5czKW.Us0Ev_YJdMCIP.Nh6RMZYgLtb4RDyLAvuQXzFD 1ahUaoYET9JX1OmJoE_rmWAZpzAaQlSM3zUS.xQDaKZ7GK.bsZ8Nv0UvwqAoPZB0mXzpyoV1A18. 0eeCSj4vwQl0tYDYXRAISsH0Aqxi6pc.MlE1dOkMdNUsx_L4AHXQ08VRVMsbqhc2DwEvgrvdmcO5 z7KX3O2jVf6KbmK2BILxzqieAfg9WKEOxHmnORAy35NKjhi7.qob84Otdpff_wQgVoaEoYG6.Gom JxtOkHzsBPi6Uu3IBWuXAf0qkt2geawYdLL7c1ZvDnLBQRr9khz8MTphOeI19JY6mGlnaJyqQgvb n1bB8mXJXqHiukgHeuExGUtqxXNRSJZhnTKvRS4wjzEhNxmeGv4swtIVDA6h_m0WoaN0v5LmdWug sL9RZVporD7zfcpJPG1WJS1hJhDKkK_SeMab91CXsjt1aaYB7elBHAOdl7Mxck5cb6aSAQf9iMFF MZue4xZ6s4R5SyjDlxfu5iTxPQ0CfeCj_Vp9SFCgAyEowW0eUgbNy1cf_8NrFNOPTUDTfR1zhqGa 5roc22EjKm2d5eyGAR45by5Kf3aGzLgLkfnHLsLUNc_eRnDmG4tt1SzjqH9C5fYK0ozHOTNpbVTn 1WHep7Cf.V.OMkNwV_XhmEiv2FL.4WXixOaLcW0DvPQ_28bnl58yR2eDqzXqXw0GQ.JhMrRBV0B1 MNYFS7qrbza3W83VWMPgz3N1yYvsQ_vkrvXw1hUfAsPcR8iV1DNI4TKoniOA8O4VPtlPZG.l6cWS K6Yjkzhqa2VJXOqQU7FdqpMSkKL4oZBmJf8jMc4qY7pBA7wIpiJaR4pENTI.eBPbq.0mfcxag5ad Z1BnAT35hEsij4xtGiUmk0sj8Sz03OzFbn9SPGDnHvIO8JzSOdnC9ZRHcj9_9YS9OoQt5GgujJeT bGTbe.zaHjI0_063hlWY4d7ZppPHcS0TkcKW0BvNE9WF4OFQgijPdkhr0qsr.sYyO22nOSCFgDb6 UQlB1QL2zaMvNwweswFFg1GEWYDxIPD1N.3f2FIUAczGSzhnGbw2DNipZ1U84iDjLgjYAu0cWbGH fJGmuK_bfpjWfIgGvScq3tLJ6KA0qMeUsOv4VYggrLxtqpVZ7hl4QJapM4gn7qt97y3DvjoFH5F. VXkwnwK4dmQesVvatqSMmrc1XGClFjT.G.dSnOaa20AkkE.qIlFMDoXfe3bVoCBBG3rbuxEIVYS_ cIUFfW8Ji7ot9NhO.zPoZ0zcMHmehOPUwo67LNQSyETH.OeaZxYmYvi09JKRboJpBKtbv01XLRm0 gfHWD2XT5xffFVetspjU7N79qsfVPn4UGu9MnpLJxXSc1Nu5nF9ktP65qHvDYrkSHwQYz87clnVz 7aJq0Rdc7TJk9vbjQfE8mXIkA6w7ADVe59NOR0vxN38AuwTHac41Tnpm36OcKGlDR8wJRzNj3E.u CwTWKyXDSxA-- X-Sonic-MF: X-Sonic-ID: 38ec0d9c-0e85-44c9-974b-f419a2f472c3 Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Sat, 9 Sep 2023 16:32:49 +0000 Received: by hermes--production-gq1-6b7c87dcf5-sv5pn (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID b392d06b9743fbafddba95b9ff9f6f4a; Sat, 09 Sep 2023 16:32:44 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: main [and, likely, stable/14]: do not set vfs.zfs.bclone_enabled=1 with that zpool feature enabled because it still leads to panics From: Mark Millard In-Reply-To: <8746A218-F83A-40E7-95F8-5EC1E36411C1@yahoo.com> Date: Sat, 9 Sep 2023 09:32:33 -0700 Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , Pawel Jakub Dawidek Content-Transfer-Encoding: quoted-printable Message-Id: <1B343698-6865-4761-B514-1539AAE291BC@yahoo.com> References: <7CE2CAAF-8BB0-4422-B194-4A6B0A4BC12C@yahoo.com> <08B7E72B-78F1-4ACA-B09D-E8C34BCE2335@yahoo.com> <20230907184823.GC4090@FreeBSD.org> <4f4e2b68-57e0-a475-e2bd-1f2b8844ebfe@FreeBSD.org> <354C5B8C-4216-4171-B8C2-8E827817F8E5@yahoo.com> <8B8B3707-4B37-4621-8124-D6A77CAF6879@yahoo.com> <15df58d3-4603-132f-112e-d10a6d4419bf@FreeBSD.org> <2a25427c-5a61-3f72-4e31-b7666741d38d@FreeBSD.org> <63717d32-f340-1320-3335-85135d1b62bc@FreeBSD.org> <05C47E15-640D-41AD-9C4C-73A1D5041CF4@yahoo.com> <8746A218-F83A-40E7-95F8-5EC1E36411C1@yahoo.com> To: Martin Matuska , Alexander Motin , Glen Barber X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.992]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.147:from]; BLOCKLISTDE_FAIL(0.00)[98.137.69.147:server fail]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.147:from]; TO_DN_ALL(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RjdnH4646z4fKl On Sep 8, 2023, at 21:54, Mark Millard wrote: > On Sep 8, 2023, at 18:19, Mark Millard wrote: >=20 >> On Sep 8, 2023, at 17:03, Mark Millard wrote: >>=20 >>> On Sep 8, 2023, at 15:30, Martin Matuska wrote: >>>=20 >>>> I can confirm that the patch fixes the panic caused by the provided = script on my test systems. >>>> Mark, would it be possible to try poudriere on your system with a = patched kernel? >>>=20 >>> . . . >>>=20 >>> On 9. 9. 2023 0:09, Alexander Motin wrote: >>>> On 08.09.2023 09:52, Martin Matuska wrote: >>>>> . . . >>>>=20 >>>> Thank you, Martin. I was able to reproduce the issue with your = script and found the cause. >>>>=20 >>>> I first though the issue is triggered by the `cp`, but it appeared = to be triggered by `cat`. It also got copy_file_range() support, but = later than `cp`. That is probably why it slipped through testing. This = patch fixes it for me: https://github.com/openzfs/zfs/pull/15251 . >>>>=20 >>>> Mark, could you please try the patch? >>>=20 >>> If all goes well, this will end up reporting that the >>> poudriere bulk -a is still running but has gotten past, >>> say, 320+ port->package builds finished (so: more than >>> double observed so far for the panic context). Later >>> would be a report with a larger figure. A normal run >>> I might let go for 6000+ ports and 10 hr or so. >>>=20 >>> Notes as I go . . . >>>=20 >>> Patch applied, built, and installed to the test media. >>> Also, booted: >>>=20 >>> # uname -apKU >>> FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT amd64 1500000 = #75 main-n265228-c9315099f69e-dirty: Thu Sep 7 13:28:47 PDT 2023 = root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-dbg-clang/usr/main-src/amd64.amd= 64/sys/GENERIC-DBG amd64 amd64 1500000 1500000 >>>=20 >>> Note that this is with a debug kernel (-dbg- in path and -DBG in >>> the GENERIC* name). Also, the vintage of what it is based on has: >>>=20 >>> git: 969071be938c - main - vfs: copy_file_range() between multiple = mountpoints of the same fs type >>>=20 >>> The usual sort of sequencing previously reported to get to this >>> point. Media update starts with the rewind to the checkpoint in >>> hopes of avoiding oddities from the later failure. >>>=20 >>> . . . : >>>=20 >>> [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 414 Failed: 0 Skipped: 39 Ignored: 335 = Fetched: 0 Tobuild: 33800 Time: 00:30:41 >>>=20 >>>=20 >>> So 414 and and still building. >>>=20 >>> More later. (It may be a while.) >>>=20 >>=20 >> [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [parallel_build:] = Queued: 34588 Built: 2013 Failed: 2 Skipped: 179 Ignored: 335 = Fetched: 0 Tobuild: 32059 Time: 01:42:47 >>=20 >> and still going. (FYI: The failures are expected.) >>=20 >> After a while I might stop it and start over with a non-debug >> kernel installed instead. >=20 > I did ^C after 2.5 hr (with 2447 built): >=20 > ^C[02:30:05] Error: Signal SIGINT caught, cleaning up and exiting > [main-amd64-bulk_a-default] [2023-09-08_16h31m51s] [sigint:] Queued: = 34588 Built: 2447 Failed: 5 Skipped: 226 Ignored: 335 Fetched: = 0 Tobuild: 31575 Time: 02:29:59 > [02:30:05] Logs: = /usr/local/poudriere/data/logs/bulk/main-amd64-bulk_a-default/2023-09-08_1= 6h31m51s > [02:30:05] Cleaning up > [02:38:04] Unmounting file systems > Exiting with status 1 >=20 > I'll switch it over to a non-debug kernel and, probably, world > and setup/run another test. >=20 > . . . (time goes by) . . . >=20 > Hmm. This did not get sent when I wrote the above. FYI, non-debug > test status: >=20 > [main-amd64-bulk_a-default] [2023-09-08_19h51m52s] [parallel_build:] = Queued: 34588 Built: 2547 Failed: 5 Skipped: 239 Ignored: 335 = Fetched: 0 Tobuild: 31462 Time: 01:59:58 >=20 > I may let it run overnight. I finally stopped it at 7473 built (a little over 13 hrs elapsed): ^C[13:08:30] Error: Signal SIGINT caught, cleaning up and exiting [main-amd64-bulk_a-default] [2023-09-08_19h51m52s] [sigint:] Queued: = 34588 Built: 7473 Failed: 23 Skipped: 798 Ignored: 335 Fetched: = 0 Tobuild: 25959 Time: 13:08:26 [13:08:30] Logs: = /usr/local/poudriere/data/logs/bulk/main-amd64-bulk_a-default/2023-09-08_1= 9h51m52s [13:08:31] Cleaning up [13:17:10] Unmounting file systems Exiting with status 1 In part that was more evidence for deadlocks at least being fairly rare as well. None of the failed ones looked odd. (A fair portion are because the bulk -a was mostly doing WITH_DEBUG=3D builds. Many upstreams change library names, some other file names, or paths used for debug builds and ports generally do not cover well building the debug builds for such. I've used these runs to extend my list of exceptions that avoid using WITH_DEBUG .) So no evidence of corruptions. (I do not normally do bulk -a builds. The rare bulk -a runs are normally to check that my configuration of a builder machine still works reasonably --beyond building just the few hundred ports that I normally build. So I should be able to build most any combination that I decide to try.) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Sep 9 21:38:09 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RjmYq5y0nz4sr0Q; Sat, 9 Sep 2023 21:38:23 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RjmYq2P6kz4Ks3; Sat, 9 Sep 2023 21:38:23 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-f53.google.com with SMTP id 2adb3069b0e04-5007abb15e9so5415937e87.0; Sat, 09 Sep 2023 14:38:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694295501; x=1694900301; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kVph2YuRV82Uh17RiOx48L7cg2YuVco2JEzPRbuVLQQ=; b=iKRxO9f+3aJxeohNID4k+RLmt3kfZ44B37WijpmJCUgbusE4LSKn2ZArC7J8shP/Cz cRoUeTMDsPH3fjReTeczpQhb0w+ab46MKHnOdzo/WOoC73fOlTxYR/uil86Q9aNTVY0s 3zzZTOEvaFJrm7Q2/35Y8uYhDxDYK8XdMuvJH2/4DwhbLH/kURSVt+Lrfi+4aYl1fpo6 k9Y0dIiZOyk/LnMCOSDct8NCAnjRZrvuUzYACzSfpwtOS3vL6YXAwcHoQTHciR1StHfH 8k9YC+AryouQWbP1+AlaqK6kElMAOw/0+31hahViZSzw0LMMm+zXQLXCnwQVt24W/7O7 ESHg== X-Gm-Message-State: AOJu0YwSJpAViDI/UYXZRex3C8QycYJIA23hFTSEGeI1sqc6MvSYkMt7 1dnfYsWqn/cBlpqHknQB9RjHiG3AhroaSH8KeT4mWvvUCtvZUg== X-Google-Smtp-Source: AGHT+IFhChV7QQNrG1bdNoQSmK0Q5sqbFycs5tTvMVOsN4jIH/qiHx2YepTlkHXJeuDtBS0suU3s8eVe2KPk6dJiab0= X-Received: by 2002:a05:6512:3c8c:b0:500:7f51:d129 with SMTP id h12-20020a0565123c8c00b005007f51d129mr5351370lfv.34.1694295500717; Sat, 09 Sep 2023 14:38:20 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <20230908233308.GS1219@FreeBSD.org> In-Reply-To: <20230908233308.GS1219@FreeBSD.org> From: Ed Maste Date: Sat, 9 Sep 2023 17:38:09 -0400 Message-ID: Subject: Re: FreeBSD 14.0-BETA1 Now Available To: Glen Barber Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, FreeBSD Release Engineering Team Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Queue-Id: 4RjmYq2P6kz4Ks3 On Fri, 8 Sept 2023 at 19:33, Glen Barber wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > The first BETA build of the 14.0-RELEASE release cycle is now available. > ... > The freebsd-update(8) utility supports binary upgrades of amd64 and i386 > systems running earlier FreeBSD releases. Note, aarch64 binary updates > are expected to be available starting with BETA2, due to a configuration > error. Systems running earlier FreeBSD releases can upgrade as follows: > > # freebsd-update upgrade -r 14.0-BETA1 Upgrading with FreeBSD-update is failing with: # freebsd-update install src component not installed, skipped Creating snapshot of existing boot environment... done. Installing updates...install: ///usr/include/c++/v1/__string exists but is not a directory install: ///usr/include/c++/v1/__string/char_traits.h: Not a directory install: ///usr/include/c++/v1/__string/extern_template_lists.h: Not a directory Reported by Vedran Miletic in PR273661. We'll provide an update with additional information and possible workarounds, when available.