From owner-freebsd-current@freebsd.org Sat Dec 28 05:59:57 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 599B21DC18B for ; Sat, 28 Dec 2019 05:59:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-22.consmr.mail.gq1.yahoo.com (sonic317-22.consmr.mail.gq1.yahoo.com [98.137.66.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47lChX2S2Pz3Hbl for ; Sat, 28 Dec 2019 05:59:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: .MfLf2cVM1m_PPwTZGTzH2sVeykiagPnhV9nU4B25yLE0k3Nakqs6up54RnkdHW .wxWBTTrK285b7pU5f_A24umRJv4Qx3UkjA7D1O77kEYssolRz5j2589HQ7ordS2fFqUMYXdg76t wxoiKPVUzAe8bJ8jCBerMKQf3l0pcD.pAkDVXEh71qXPybR.YbZrsfc5fPSqq2C0fTaF8LiTB.i. 1Jx_OMUmBn3nS2SUfQOE.WCu70bHgpbX3RzqU420iRrtzIv6OqlhPrlKSrLji1yeazxO9CCkZngd C3T0vq.uCkbMnByq1crBoLA6P1hU7bSrlifrPGjHLWLNWZFEQS.1zQvIrp1PX7lyDJ61JxHGVi6e TpyLYeYHa_vUagITj3lVxmkDdzPhSkII0nPYATf78fJ7U4hsDDLXi8fnaf23wfpO8eTlm8ewMIxP 0Tl8orazv7KxNPxr7T5d1afXDlzuM4_6ywsEzRsXLAQDOwWDCIS_zH9zv761FmPXX0C9kGlwoQWI ZWsP3XAhT2hP9tc5K6BkXUFyfKKNMyrIFsWbvCXlp9r07KRzk8NAFcHJzsAnm4gFcCo8_1NXv8Pg a45JofkzQlBf7qYv44hHeoaHbgR0CFlgKHPz3EpdYfwsi.VahzfN1aIdUjVyAdJdbUau6aDtwGVt HegXQFQsoITn3aeKsbQPUq6zS6FRxCoEqM7eTzfct6E6ZPkNlYkWwySWYlUeyIH9RaiId5Ihni8a EfXIDmkVL7gwBYhZgJgd7fYj1DphbpfHaxuZYv7NSEVLZvhSx2oh7BuoyiFuK7TA8KzWz0n2hkeF 2HSby9RclI27gu_qKz827QfLyG9EBbnRZ5tra38WCK8zyNHOI0usoBX6gu6cUNN2g6Rh_rbmyAOX 0_0E6Zn8Im0rJFIBfBfUOceIXhg7UfMyFIT4rFBmuWhmx_fzcuk8duoe9TIhNUlgv9OUiAfG9lvQ cjRxD_6dkWWlIEj9am_nzy89Gp5vXvcF6MUdgrObd1RgxDkzb6cFppt1dIA9reUUeAU.Yo1V4PAt UlUNO9NFsJauXgaCkRe0A3dtkoW_v.qeVoD2seVtciN6fVY4G2W7btN2BW8qJyDwtw28aijjI3GD 6qefSrfIsCfdfQJQstfd0VdHbxTg9EDW5eWHfXOUxqA3UjOzd7Olb8yUX1wX6BLOOdg0VHUNWKcA OZgq90QktGXjU3UrSHOSircQSKQSomN.Gdprl.cSoFm3f3hYxUGEctTxPVUOq4NVbqXrWxKzse0E Vo.IsLM4epUQeWUPvjUqrkFw3jFrvn2d20rRMMTjg6i5AU4G9FImWNiYxbVitX8PvCAICfA..Scf YlupuuW39HeouC2x25tiDonud3PEOU41zJT__z9cz0xovdosX3IYeHJokk30J6BpNVvMnSz32BQE qysojJ.nHO3HHOgNb7dL.voNZ8d4I1Pai5ZtnzX4- Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Sat, 28 Dec 2019 05:59:54 +0000 Received: by smtp410.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 5727daf17009ee2142fefff3f58f77f2; Sat, 28 Dec 2019 05:59:49 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.40.2.2.4\)) Subject: head -r356066 reaching kern.ipc.nmbclusters on Rock64 (CortexA53 with 4GiByte of RAM) while putting files on it via nfs: some evidence Message-Id: <65659DD9-335A-40C7-A6DE-B71019BCD6E6@yahoo.com> Date: Fri, 27 Dec 2019 21:59:49 -0800 Cc: FreeBSD Current To: freebsd-arm X-Mailer: Apple Mail (2.3608.40.2.2.4) References: <65659DD9-335A-40C7-A6DE-B71019BCD6E6.ref@yahoo.com> X-Rspamd-Queue-Id: 47lChX2S2Pz3Hbl X-Spamd-Bar: - X-Spamd-Result: default: False [-1.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.45)[-0.452,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_GOOD(-0.10)[text/plain]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.76)[-0.761,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[148.66.137.98.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ip: (6.67), ipnet: 98.137.64.0/21(0.89), asn: 36647(0.71), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Dec 2019 05:59:57 -0000 The following sort of sequence leads to the Rock64 not responding on the console or over ethernet, after notifying of nmbclusters having been reached. (This limits what information I have of what things were like at the end.) This is for a head -r356066 based non-debug-build context. The sequence leading to the hangup was: # mount -onoatime,hard,intr 192.168.1.???:/ /mnt # tar -cf /mnt/usr/obj/clang-cortexA53-installkernel.tar -C = /usr/obj/DESTDIRs/clang-cortexA53-installkernel/ . # tar -cf /mnt/usr/obj/clang-cortexA53-installworld.tar -C = /usr/obj/DESTDIRs/clang-cortexA53-installworld/ . # tar -cf /mnt/usr/obj/clang-cortexA53-installworld-poud.tar -C = /usr/obj/DESTDIRs/clang-cortexA53-installworld-poud/ . (Hung up during this last one, after reporting on the console that it had reached kern.ipc.nmbclusters.) cp -aRx of directory trees copied to such a mount and the like also lead to such. I've not seen such an issue on the Cortex-A7 (armv7, 2 GiByte), CortexA57 (aarch64, 8 GiByte), CortexA72 (aarch64, 16 GiBYte), or powerpc64 (16 GiByte) contexts I've done similarly with. For reference, after power-off/power-on and retrying just the last tar, which worked, shows what I report below. Before showing the more complete list of sysctl -a output that mentions "cluster", here are large valued items compared to the other example contexts I've been using: . . . vm.uma.mbuf_cluster.stats.frees: 557 vm.uma.mbuf_cluster.stats.allocs: 63807 vm.uma.mbuf_cluster.stats.current: 63250 . . . vm.uma.mbuf_cluster.limit.items: 63739 . . . vm.uma.mbuf_cluster.keg.pages: 31870 . . . # sysctl -a | grep clust kern.ipc.nmbclusters: 84351 kern.geom.raid.raid1e.rebuild_cluster_idle: 100 kern.geom.raid.raid1.rebuild_cluster_idle: 100 vm.cluster_anon: 1 vm.uma.mbuf_cluster.stats.xdomain: 0 vm.uma.mbuf_cluster.stats.fails: 0 vm.uma.mbuf_cluster.stats.frees: 557 vm.uma.mbuf_cluster.stats.allocs: 63807 vm.uma.mbuf_cluster.stats.current: 63250 vm.uma.mbuf_cluster.domain.0.wss: 1 vm.uma.mbuf_cluster.domain.0.imin: 0 vm.uma.mbuf_cluster.domain.0.imax: 0 vm.uma.mbuf_cluster.domain.0.nitems: 0 vm.uma.mbuf_cluster.limit.sleeps: 0 vm.uma.mbuf_cluster.limit.sleepers: 0 vm.uma.mbuf_cluster.limit.max_items: 84351 vm.uma.mbuf_cluster.limit.items: 63739 vm.uma.mbuf_cluster.keg.efficiency: 98 vm.uma.mbuf_cluster.keg.free: 1 vm.uma.mbuf_cluster.keg.pages: 31870 vm.uma.mbuf_cluster.keg.align: 7 vm.uma.mbuf_cluster.keg.ipers: 2 vm.uma.mbuf_cluster.keg.ppera: 1 vm.uma.mbuf_cluster.keg.rsize: 2048 vm.uma.mbuf_cluster.keg.name: mbuf_cluster vm.uma.mbuf_cluster.bucket_size_max: 253 vm.uma.mbuf_cluster.bucket_size: 251 vm.uma.mbuf_cluster.flags: 0x2008 vm.uma.mbuf_cluster.size: 2048 vm.phys_pager_cluster: 1024 vfs.ffs.maxclustersearch: 10 Here are figures for a couple of the other contexts, after their tars-via-nfs: =46rom the Cortex-A7 (armv7, 2GiByte) context: # sysctl -a | grep cluster kern.ipc.nmbclusters: 26086 vm.cluster_anon: 1 vm.uma.mbuf_cluster.stats.xdomain: 0 vm.uma.mbuf_cluster.stats.fails: 0 vm.uma.mbuf_cluster.stats.frees: 1689 vm.uma.mbuf_cluster.stats.allocs: 4472 vm.uma.mbuf_cluster.stats.current: 2783 vm.uma.mbuf_cluster.domain.0.wss: 0 vm.uma.mbuf_cluster.domain.0.imin: 125 vm.uma.mbuf_cluster.domain.0.imax: 125 vm.uma.mbuf_cluster.domain.0.nitems: 125 vm.uma.mbuf_cluster.limit.sleeps: 0 vm.uma.mbuf_cluster.limit.sleepers: 0 vm.uma.mbuf_cluster.limit.max_items: 26086 vm.uma.mbuf_cluster.limit.items: 3539 vm.uma.mbuf_cluster.keg.efficiency: 98 vm.uma.mbuf_cluster.keg.free: 41 vm.uma.mbuf_cluster.keg.pages: 1790 vm.uma.mbuf_cluster.keg.align: 3 vm.uma.mbuf_cluster.keg.ipers: 2 vm.uma.mbuf_cluster.keg.ppera: 1 vm.uma.mbuf_cluster.keg.rsize: 2048 vm.uma.mbuf_cluster.keg.name: mbuf_cluster vm.uma.mbuf_cluster.bucket_size_max: 253 vm.uma.mbuf_cluster.bucket_size: 103 vm.uma.mbuf_cluster.flags: 0x2008 vm.uma.mbuf_cluster.size: 2048 vm.phys_pager_cluster: 1024 vfs.ffs.maxclustersearch: 10 The Cortext-A57 (aarch64, 8 GiByte) context: # sysctl -a | grep clust kern.ipc.nmbclusters: 168310 kern.geom.raid.raid1e.rebuild_cluster_idle: 100 kern.geom.raid.raid1.rebuild_cluster_idle: 100 vm.cluster_anon: 1 vm.uma.mbuf_cluster.stats.xdomain: 0 vm.uma.mbuf_cluster.stats.fails: 0 vm.uma.mbuf_cluster.stats.frees: 8678 vm.uma.mbuf_cluster.stats.allocs: 10702 vm.uma.mbuf_cluster.stats.current: 2024 vm.uma.mbuf_cluster.domain.0.wss: 0 vm.uma.mbuf_cluster.domain.0.imin: 25 vm.uma.mbuf_cluster.domain.0.imax: 25 vm.uma.mbuf_cluster.domain.0.nitems: 25 vm.uma.mbuf_cluster.limit.sleeps: 0 vm.uma.mbuf_cluster.limit.sleepers: 0 vm.uma.mbuf_cluster.limit.max_items: 168310 vm.uma.mbuf_cluster.limit.items: 2069 vm.uma.mbuf_cluster.keg.efficiency: 98 vm.uma.mbuf_cluster.keg.free: 1 vm.uma.mbuf_cluster.keg.pages: 1035 vm.uma.mbuf_cluster.keg.align: 7 vm.uma.mbuf_cluster.keg.ipers: 2 vm.uma.mbuf_cluster.keg.ppera: 1 vm.uma.mbuf_cluster.keg.rsize: 2048 vm.uma.mbuf_cluster.keg.name: mbuf_cluster vm.uma.mbuf_cluster.bucket_size_max: 253 vm.uma.mbuf_cluster.bucket_size: 5 vm.uma.mbuf_cluster.flags: 0x2008 vm.uma.mbuf_cluster.size: 2048 vm.phys_pager_cluster: 1024 vfs.ffs.maxclustersearch: 10 (The Context-A57 had more than just 3 tars done over nfs.) The problem seems somewhat specific to Rock64-like contexts, not that I know in what specific respect(s) for that "like". =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)