From nobody Thu Sep 9 23:35:11 2021 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A3E9D17BE8F9 for ; Thu, 9 Sep 2021 23:35:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4H5Fjm1hjLz4Xh0 for ; Thu, 9 Sep 2021 23:35:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1631230517; bh=AUUXBf9+kC39TDAz3dZZlFwo5dS5R7FTF9baOH2TFeA=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=MEbyxIII6vqJ9l7k67UjX5njASvMDDVXjAhyxC+S3rExMbSzSsCtJ1eDLX1JC4ZD/72cLEpgN8Iv6d20Zis8W0OC+hYvEjaztUjGakpdrkKG8y5hVBGKQaWmGmv/vyEnpNKduYVNsmZVQSfXcYqgG3JNrXdYFGr7zQTfvdrVmD9pSfgeNwvRCcp42C1GD+rWenCxiUChPaM4HDWY1x+U5f8eY2eTwbwpELiZ4QW687V0RbxW85DwoYKgFgBnprI7FraeVgOIpz/WhahMpw1RNPfFsxdDD2P+2/Rq2/uOIt5s/gGLSKT/4S8T8+E42pOjM6et/ff2NwnkZ7v4GvRtLA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1631230517; bh=EHmsBXtqrUNEClKX6PmkmsU8KAIbzS5ItN+Bgrclnu/=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Ehk7hUh6v8gToTzlOREYGjUzr27FR6cnfqKIjGYfEqfwA/ZM7OLnBUoHcE5SJxpn6/DdPhm7QQpissBZ8lHHwCvT3D7m/9mzzseA9ehJuaRgT1f+1coP6vXLw4CDl37NepgU52AMGwgdZqm6GN65D/dXZlrZPceD+lQKmzQKh+dFdNWpAYhVzur/YDBkuCoEDqlpRQx0mJD9iEOZNOWc3Ca3cnSDsfAEqBNX9IOjWqlYSnH8D5I8J7tO3zk2hOt1QJBQF+p2ONEFeDKWA2H32YC80dxz9NEBdAKNDGEKk9pvtnGzsfMfGTRcg0iYM9Mr5N7DsCl+01Xzc9D2Gihm1Q== X-YMail-OSG: F5zo5NMVM1kFYGPhN.Ti_ChpVLo1OL0RTJyV9BcH50ci2BznYh_jul_aDjQ9pqm 8qIDkEbuf3SOobHGUovk1DrD27i37zqx6ygpeV6RQUZF1WaH_TWzndk.46KwwdXjnunckq6z5UAk c6pdE6bUAzzlQIwYYDevezXdSaUQXEp4BiyJ9QSDortkc62ukXHELFNsY7OI1qdp6gdb87SOziG7 8r2M3TUbHnZNrbPhLDAJgQVAkcrHi4XS_ZBIKVBAjlu0n_lCwv5pZYknB2qSTGLfFT7N9L5Qf1DK MC_Sl0BQkhLbimjB1X2nmM_vqzxeA01k8SMekGeHbUnZEZKghoL26vb_.1FffZ3SoVoRBYS2Lpiw CihDMaixudgQLkXiEB_Q87DAG22ENpUL1V0saZAjwJ.taQ0JskEXF0Hh_RvUBGUQEX1GMQ.16Iim awr.cFdnBqzLeWtPPMIHCv9PFHC4eD6tFjF6E735QNBKlt7JwR_4qJoBsPIAb8ib9eNdphcC4dlI pcZtSHC8dPag7w50ME7TYU2wOOZYDW5.pm5H12TNlc31Q4qMteunJSnGSfKmiZ8KE2qpG_yfgNOP Z8ltNjoHFMznwRv_5UxLjkc4MNXk_QquNznAJi1MteqOIb4rDtcX.AdPDQI5EGVFfJNIHylobjU0 3epIDWDSkDbeCRDgrspZnPb2dUHuL6CmCE2YD0OxzQQ9j0RIY_NLWe142OlIWfACtxm6zY2rh6FR b.3uEHKFJz.O8vO2CTAxeV.n_XIgm43QWEwj66pETiIM1FFyTeh6Jfn2AFMJzVYj8BDic1BhpQv0 eOc3uXDlORP.DqCsScio_PqSa56AkCKixTrPB9V5v9KifObgmoJY6JUWn64bny25qqBjyXbmVNS0 REtS9Ezr_Y0ldX7zfv8JWsaysiZN9aNsSPZ28Rk2yk.eTd1tGvvNYuU588I_siXQ5Jh7v44Do2lc EtNN4JrQhStQzUg83o3eUBf8dBURHndkcgAuAdAXdRwmgna7Km5hgX_TEcQgVbGBC3skWYAU0jQs R.16kWicWLhuBsMTUBlg.1svY5RVrPoEaIQbnm.eEPRKUwsis0MTV_ZI5Apuj36cty5tlNz0_CxO dUXYdl4Z5TiIRLKmJg.0g9L17j5UQld7nEE6mSx1yIwhrFDP5trm4akBjZNNMJUDI0vyk3zLdD7P tZiCumPJ6X9KCX0h.AVd8gRKMfz_63OXCB2UgoRX8ppZQ5H0eKeDA3hi1jB4IA17bCqdB9nFWpdL OTH_I3oXscq2fWfWNPcgfG4R.PITg7_WvYFyxOy.98q.I1YYv.bGagyQRkmWnkif3naRxV1JK5c4 a_g3f2HcPoByE.jdRwH_zYgwHH8vxB3.Nc3t_ORhjjhIv9ugoklpLLZ8tIERDscDQuD15IJQtT8D hexbq9e61fxQEoh1RxCv3u6osBLUF0LlmuoHUs1NkqsLg6Khq30iNiXZcXjAC98LaZj1QQb4Kail 4Fl.YqSxW45SOu5NsN4YxpHzdr4uSjRP5jqFC1I34ywQyUWfS.an5jfotXIFpXaawg0TWhBZg39U 5PTQTs9AwptQ9u3M9NkFoWWGg5QIq5U1z2XOEDdeL12Q.5kHYupVyEoPdouDyfJc9YIDWOGoRaCo CwhG3k6U_zttax6c1sEoLlVaK1n2Z.Hik.pvEUBr2WBtfyVx9nRBJevvWjqlQzs3US9KD_761woZ 3GVomv8fTRgaxNAZiClLKY1XlFVh79.8GtgGSmTW8iWgz4xVauhRIxKMOavp46ka2613MvKhLAwN LRPgeWlbVQB62C623rF9DIzNB14MjS1WXosNUrWN6XIiBN7pE01QqOIvnBqiuQsjebu9ML5dL0ka mY2rftnMxr.yWdVrTXyC9EPHtqsm58rs0Zo6105ohGFyFkGCPUXt6DLAJbT6.ICiq0BPviZaCobR 4kGKPgBm_6V11vS8XqDsIZpGJg1g2NPvPqaS9BQGhd.TBt.ZCk4ihm4R_uTgl8LJIioe_uQjiTZ2 im0Ot5VzK4iHzBYFD2QZLqIn5oXDUYL2okXRur63PRlXs6D61VFYU9Q02TWm_1GU2TCXoFWmx_P5 5X3SaOOnEGKh5LFEa056mkTAZyPFqlpwF9UCDTKc3BrxNXbR7ZHQR3CGyRkOeVKpQ0jmo5Hubffc CqvjwjBQDBDZw4wOIyEKVYLsidEkUL6YUUrdjmBYUHqTF.rT12gza0KvCCRkZW2WCryI3XH9mjdS AiPxG36qW99b43GJ3iyUX_ZBfGElmhcyDmW7pwEWMhfh_kdC7SlKVa0QvnAis9X.dtfZmivXsYzz cepnjko7eAB1xLKUhkafWsRt2rbPQN0SaHcYsaQ23IHxLawZmmQCFjtVGlCWz83kHxQnzM_fCHl4 B4ZY- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Thu, 9 Sep 2021 23:35:17 +0000 Received: by kubenode513.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 391d86cceeac2fe449ae22f0f7af2acc; Thu, 09 Sep 2021 23:35:12 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: poudriere-devel for UFS vs. ZFS: time for "Starting/Cloning builders", aarch64 example (HoneyComb with 16 Cortex-A72's) Message-Id: Date: Thu, 9 Sep 2021 16:35:11 -0700 Cc: FreeBSD Toolchain To: freebsd-ports@freebsd.org, Bryan Drewery X-Mailer: Apple Mail (2.3654.120.0.1.13) References: X-Rspamd-Queue-Id: 4H5Fjm1hjLz4Xh0 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=MEbyxIII; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_DN_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-toolchain X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N The following is from the same system but different boot media selected = in EDK2's UEFI/ACPI. A summary for a 16-core HoneyComb Cortex-A72 system is UFS: 80sec+ ZFS: 10sec or so. I am limited to at most 16 cores for = aarch64. HoneyComb (16 cpu) UFS (so: NO_ZFS=3Dyes) with USE_TMPFS=3D"data" (main = system) : [00:00:28] Building 476 packages using 16 builders [00:00:28] Starting/Cloning builders [00:01:49] Hit CTRL+t at any time to see build progress and stats [00:01:49] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.17.1 HoneyComb (16 cpu) ZFS with USE_TMPFS=3D"data" (main system) : [00:00:13] Building 475 packages using 16 builders [00:00:13] Starting/Cloning builders [00:00:23] Hit CTRL+t at any time to see build progress and stats [00:00:23] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.17.1 Both of these were after recently booting. Past experience is that UFS gets more than proportionally worse as the freeBSD cpu count goes up. But I've only had 4 and 16 for aarch64 and until recently had no ZFS context. I only had 4 (2 sockets/2 cores each) back in my old PowerMac "Quad Core" usage days but it was demonstrable back then as well. (The Quad Core PowerMacs bit the dust.) Past investigations suggested that th parallel cpdup's were spending much time spinning but not makeing progress, ending up showing getblk status during the spinning. I have at times modified the common.sh script to have the: # Jail might be lingering from previous build. Already = recursively # destroyed all the builder datasets, so just try stopping the = jail # and ignore any errors stop_builder "${id}" mkdir -p "${mnt}" clonefs ${MASTERMNT} ${mnt} prepkg code instead executed in a sequential loop just before the "parallel_start" . This helped cut the time for a UFS context by avoiding (busy) wait time. For reference (the ZFS example matches but for node names: # uname -apKU FreeBSD CA72_UFS 14.0-CURRENT FreeBSD 14.0-CURRENT #13 = main-n249019-0637070b5bca-dirty: Sat Sep 4 18:12:39 PDT 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400032 1400032 # cd /usr/ports # ~/fbsd-based-on-what-commit.sh=20 branch: main merge-base: b0c4eaac2a3aa9bc422c21b9d398e4dbfea18736 merge-base: CommitDate: 2021-09-07 21:55:24 +0000 b0c4eaac2a3a (HEAD -> main, freebsd/main, freebsd/HEAD) = security/suricata: Add patch for upstream locking fix n557269 (--first-parent --count for merge-base) But the issue is not new. The effect is visible on the ThreadRipper 1950X (32 FreeBSD cpus): (Again, one system but different boot media selected, also the same main and ports vintages.) ThreadRipper 1950X (32 cpu) UFS with USE_TMPFS=3Dyes (releng/13.0) : [00:00:11] Building 111 packages using 32 builders [00:00:11] Starting/Cloning builders [00:00:51] Hit CTRL+t at any time to see build progress and stats [00:00:51] [01] [00:00:00] Building devel/meson | meson-0.59.1 ThreadRipper 1950X (32 cpu) ZFS with USE_TMPFS=3Dyes (main) : [00:00:07] Building 535 packages using 32 builders [00:00:07] Starting/Cloning builders [00:00:18] Hit CTRL+t at any time to see build progress and stats [00:00:18] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.17.1 So UFS about 40 sec vs. ZFS about 11 sec. (I never did the contrasting ZFS case for old PowerMacs and no longer have access to such.) I'm guessing that the UFS context would not scale well to having 2 or more times as many freebsd cpus. (I do not have access to such.) I expect that other times when other parallel cpdup activity by poudriere is involved for UFS also ends up with the "getblk status" issue in some way. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)