From nobody Sat Aug 30 13:50:45 2025 X-Original-To: freebsd-pkgbase@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cDc3v3nTVz65TGw for ; Sat, 30 Aug 2025 13:51:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-54.consmr.mail.gq1.yahoo.com (sonic316-54.consmr.mail.gq1.yahoo.com [98.137.69.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4cDc3v214Tz3NXK for ; Sat, 30 Aug 2025 13:51:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1756561860; bh=cHjB1RaIgyzs8gAC1/vm1TQifK8+qZ9kY3of8TjWeWQ=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=jPGOibzlxAOBYodJGGD1p8OVMgs8jxtG6a9ZF8zfgZS9bi8dc2Z72RJ8hQlZ0Aytpq6LDvK/gdHQ8nhWCElYryegH1t0SN3QTQjWMEIlCLu4zVlv3WopUZYn+sos5W8S0FXJgzb4lsnfMyZomexY3fzGsRw0RTKTNDxjXAa2O7oqaRF/eceDwjDOPSBpmAW5XBHf3qDvXSnCEnR6A1SlmnAPfS2cGzG9hPkp2DUWISOYfnCh6tCNWNoedfkGYRvM6FcJD0F+ilGNuchFZRagvBEpFWf/0RJy6vpcyNLmvmSqZvmdr7ioVZ4UxDWTbdqD7nTZbme9uO44bDLN4gCVEg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1756561860; bh=b15sXwFOMQRssYf8d3CjkPIN3vag72ORq/HMFyq9Wrb=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=B5z6GZlDInw3HDUSe71WcdAozXD6ENyPJb2Nv+gD6KbTh9cEIm2i0xA4021gjSpXVZ1pjU04ixiBz+3+u8SLqH3nSkbwBvQIoMR3fHmOVfrdSVUxs8pyy2fz2Am6eprhH/GpRs0Rv5rspJCKahTZm76+BBWIe1UE+kg8vc8KVRfgLyiAXlegNYr0oj+3WDHIjzIfxFQDrkxKRad2sVlcekWRN22zpoQukf+Mra04BQycRJT/PaItX4PfIHOsZxkYopU5TnsvI5eLsX50J7n7bhmMvkGkRe4sy1Azf2q8a9lRm4UsgeGjm0ZMOQQ0XSjHMbJ+4XtAkwJS+IXnkTfG7g== X-YMail-OSG: ZJYgGnQVM1lxfk7PHbUgU_.KlaNBPjL26biSCO74wBcVe6JhV6jvdCbLCDEeUWp J.Nk4o1IhWpHkxGL.NROyoKIFTiV4cmtS3QrGmh2ErjQEKLPeNejhVvHUz5aiar0PRW6dHN4_C56 D.IEmoTsClm57B_x44wB4AjOfRSi0lyay.QSLhqkQbsMs1fuqFbPa_AORdDbSByJpq1j8ahFd0YE nJbRGTAISSA2VqkIMNnkpmi05H.yrfjAVSCE_oU0j0TC4oWfSSnDXoERjkEikk7UcKx5OPcoCmbs eHdXPVbDtKqA3NvDe5koIKAkn3hBD_BHs4U8_lqDXQEsgqYfQ7Qvn_ODKYca9VKnkS5W_VQcF.ed vjtYLWtdxPXu97BVrwZrhBCKZk3vJmsV_AulQP3mHLLvEVCYWn9fEqyI_TDcDchGNUJHbLwdRNGC PQ5TN6M1TBT0nwTFkwGBnIvs0X4I.0qoQPZJziyQMaB._QKPm5fMxeL6HCMeSWwxyTNEI55Sfq3O RaUmTMLp4EwCLcQ3tUEYbLWm5p23SoXpAlm2JuHYDUgydVQY60jylLo5iAwFKiRFzBzHs_0Kjnfi dfFXdOj.IoSzINEGJsVxozOfQCKa8DAdZBxzWnaRuXxg8gr70.KvOQuteiOpwY9Sx2TdDCG1D4I. NXy8E756cv90VAtb2PJuqbVM8XbNrZ0hj0jvN1DBKC1ozSLBzMizyREjZCDx280v4AKwMWHgJpiW it75Eej3Aw9yZH2dIpFClUdb8nmLg2RHC7ZQ6AtOj8As7ZNojgowXHzFeCfCSGrbiB9WIoCBFxkw 9WnnxEuhKadPx.Mf8dFrm4kZsxMex_FcDPg722.jW4QLx13m39pSUObVeHzUTL3tVuIOCk3rSnkm sX2AxxhDONsd2Anf5hb0zO2jD2DjrmDrq3Ebyn48tNuSEaBbcNKRPk_N4YfGImK3FAoRCZpnwDCI yUrKgKJJZ.KlhTl4u7oOFH67yZUHeNBYV8ME69U81sYZlec.jDDsuG.nALZY3hysK4wkOajHUOZd Cer4o_F7qiHNmZgCGfeaZUZNYTU4otdz2WMec2Q9W.4pZK9RQJS.GuZ54Jky9hkOcaSopvsZTBf0 xSv203c4roKKopksU1IzqaLCn830ghXv4zDda67tSsO_FdbyjPMcmBj5jgQy9uhDY4s_bj5xOyG0 cFFX9aFCdwsnJkH6zKNu3YptD0ODLtlbiha6j5WIAHH_ZYylkyoFHc5KE_TgaaJVikd_iIgSMiAZ imZH1ruPKnOo4kU5kJ7EWRUhub1WAVukPYSjHAAXShV2rzZfXRy0n6tXaDo7m8O78YZLZUCib_go z5X_XLMe.Gva_1y0AAAp73OCgloOf4B_rKYRGG2LLNCZkLxuqb7KnZshvVAezujErwSnY4rxASGO OYa23tfefz9TF5U_R_pqpbY19i2IVAJNqeGTN8GTbbrXaVr54Koy6jwwHWWP2doCejeiiHnl5Uun hW0IWvQiyaUOw_BgONbACV7dYdHVwtY6ysbJQdpDr.7sIOGC8FMyZSex6SIp0_2eeZ2J_8LTU6VG 6wRh3P2F9HcXRQkt1ouQR6jk.k29lPE7tyFdbtaV_enmwttrN_cIG5Fx1cfPOvUF1gTuOzFvaQ.l W1oFYdkAadUgrJElfMO_xVkAFJwtUZaz8vh_aSK7KkKlQ3gav7gXxNbCf1YTCBGmkb62Qx8XkAAa z8KFOUthvhETKYlDUT.PdVrKtEwtwsyK0iebH6gChnz3j6yC4.inBoH27SOcGzda0yFFQd48xmEI 3x19P4JXBMoDM8hJURQhAUMF33BmB77quxNDhxaSZ.s5ANkW1W6pFY8V5Om2Aow_zRvQplfjYD96 A6SFKvyHAilsYY5v3PR6KwX_fC62J43cXguVjQyn2az1kwWWVo24v9Oko1zM6AWEdEvp.1mCYHxq Utj8bqWGZCSV4cx90N8rQsJaFYtJLwp6AtqF1GKe460kNCVdtJx.hMRxbE16q2YM3XjYJt8pD3IW dyOCWajj7BapbyzJrnlhGZsQLDBUA6eYbWOzB9uLXsWXcSbjdnpO5fRWVTUNpJn6emUYHXgYvIle p_W2FnpA2Sc9d.1LnMgezzwoPbQNWyeGNz4zDBzt3Rj7FrMustGGDTNkKiQHs43tj1xM8PDBydkN WlN.9B02JMSq5kRS4TLNmcuO1YKnzWCwrbQbRJr5MmoKTntkyCiEjyUdLHBVZky4OH9kfZZulzv5 BFTk71eeUze6DYEZbEb6wugbSt3D8ro0BfLcUbuQqVSpGN225_ynLHt7ennKcJ4ZFf0puY60Y7O4 7ISA- X-Sonic-MF: X-Sonic-ID: 9b71223a-34fd-45da-a671-90e27105c424 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sat, 30 Aug 2025 13:51:00 +0000 Received: by hermes--production-gq1-7bfc77444d-fxzkx (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 797c8e7ae4813f2732c7583802092bd2; Sat, 30 Aug 2025 13:50:56 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Packaging the FreeBSD base system List-Archive: https://lists.freebsd.org/archives/freebsd-pkgbase List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-pkgbase@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Subject: Re: pkg upgrade, vfs.zfs.arc.free_target, vfs.zfs.arc.sys_free, vm.pageout_oom_seq From: Mark Millard X-Priority: 5 (Lowest) In-Reply-To: <29ff7f7e-12ef-4d10-8ca4-15ae35ce6bae@gmail.com> Date: Sat, 30 Aug 2025 06:50:45 -0700 Cc: freebsd-pkg@freebsd.org, freebsd-pkgbase@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <19251E67-BC92-4D71-9E71-6C7546A46D8C@yahoo.com> References: <0dbf8c95-7697-4887-a890-335c3ccd80f1@gmail.com> <29ff7f7e-12ef-4d10-8ca4-15ae35ce6bae@gmail.com> To: Graham Perrin X-Mailer: Apple Mail (2.3826.700.81) X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4cDc3v214Tz3NXK On Aug 30, 2025, at 02:19, Graham Perrin wrote: > On 30/08/2025 03:46, Graham Perrin wrote: >=20 >> =E2=80=A6 >>=20 >> pkg upgrade -fUy >>=20 >> a) failed at least once with 12 GB memory without tuning >>=20 >> b) succeeded at least once with 13 GB without tuning >>=20 >> c) succeeded at least twice with 2 GB, ARC tuned. >>=20 >> =E2=80=A6 >>=20 >> Temporarily set before (c): >>=20 >> sysctl vfs.zfs.arc.free_target=3D256000 >=20 > Maybe preferable, an alternative to tuning the ZFS ARC: >=20 > vm.pageout_oom_seq=3D120 Such assignments of larger than default (12) values should help avoid failures, for sure. (It is not biased to controlling other performance characteristics to be better for the context.) > First test: 2 GB given to the VirtualBox guest, /boot/loader.conf used = for the tuning. Success. >=20 > Second test: 1 GB given, temporarily tuned in single user mode before = an exit to multi-user. Success. >=20 > Ref: = = =E2=80=93 again, thanks to Mark Millard. >=20 >=20 > The physical storage that I use for the virtual disks is slow, even = when connected to super speed USB (5Gb/s). An old StoreJet Transcend = mobile HDD. The varying slowness might explain why upgrades fails for = me (pkg killed, failed to reclaim memory) more often than for other = testers. >=20 > During and after a simple ten-step upgrade from FreeBSD-ports: >=20 > grahamperrin@mowa219-gjp4 ~> zpool iostat Transcend 5 > capacity operations bandwidth > pool alloc free read write read write > ---------- ----- ----- ----- ----- ----- ----- > Transcend 777G 151G 36 65 4.01M 5.00M Hmm. 151G/(777G+151G) approx=3D 0.163 for the fraction free. That is well outside the recommended free fraction range for efficient ZFS use. The Design and Implementation of the FreeBSD operation System, second edition, page 548, 2nd bullet: "Like all non-overwriting filesystems, ZFS operates best when a least a quarter of its disk pool is free. Write throughput becomes poor when the pool gets too full. By contrast, UFS can run well to 95 percent full and acceptably to 99 percent full." Also, page 549: "ZFS was designed to manage and operate enormous filesystems easily, which it does well. Its design assumed that it would have many fast 64-bit CPUs with large amounts of memory to support these enormous filesystems. When these resources are available, it works extremely well. However, it is not designed for or well suited to run on resource-constrained systems using 32-bit CPUs with less than 8 Gbytes of memory and one small, nearly full disk, which is typical of many embedded systems." Note: I take that last to mean that even one of: ) 32-bit CPUs ) few 64-bit CPUs ) slow 64-bit CPUs ) less than 8 Gbytes of memory (or insufficient memory more generally) ) nearly full disk (slow storage would make that worse) can lead to the "not well suited" status for ZFS use. Use of ZFS inside virtual machines with any such characteristics would not be exempted from that potential status. Also on Page 548: "ZFS caches its data in its ARC that is not part of the unified-memory cache managed by the virtual memory. The result is that when mmap is used on a ZFS file, . . . This approach provides coherency between memory-mapped and I/O access at the expense of wasted memory due to having 2 copies of the file in memory and extra overhead caused by the need to copy the contents between the two copies." (sendfile also mentioned as having a similar issue.) > Transcend 777G 151G 34 120 5.28M 9.44M > Transcend 777G 151G 79 80 8.04M 5.97M > Transcend 777G 151G 82 42 1.72M 3.99M > Transcend 777G 151G 166 18 3.23M 2.11M > Transcend 777G 151G 157 74 3.89M 5.59M > Transcend 777G 151G 64 91 4.48M 9.01M > Transcend 777G 151G 8 27 904K 2.27M > Transcend 777G 151G 0 18 16.0K 1.36M > Transcend 777G 151G 0 0 0 0 > Transcend 777G 151G 0 0 0 0 > Transcend 777G 151G 0 12 819 218K > Transcend 777G 151G 0 0 0 0 > Transcend 777G 151G 0 0 0 0 > Transcend 777G 151G 0 0 15.2K 0 > Transcend 777G 151G 0 13 0 369K > Transcend 777G 151G 0 11 0 139K > Transcend 777G 151G 0 0 0 0 > ^C=E2=8F=8E > grahamperrin@mowa219-gjp4 ~ [SIGINT]> zfs version > zfs-2.3.1-1ubuntu2 > zfs-kmod-2.3.1-1ubuntu2 > grahamperrin@mowa219-gjp4 ~> lsb_release -a > No LSB modules are available. > Distributor ID: Ubuntu > Description: Ubuntu 25.04 > Release: 25.04 > Codename: plucky > grahamperrin@mowa219-gjp4 ~> =3D=3D=3D Mark Millard marklmi at yahoo.com